Trellis decoding method and device

Abstract

The present application discloses a mesh decoding method and device, which relates to the field of mesh encoding and decoding technology, and helps to achieve spatiotemporal consistency between two-dimensional texture maps of multi-frame three-dimensional networks, thereby improving the encoding and decoding performance of three-dimensional meshes. In the present application, encoding and decoding are collectively referred to as decoding. The mesh decoding method includes: determining the mesh elements contained in the first patch in the current mesh according to the mesh elements contained in the reference block patch in the reference mesh of the current mesh; reconstructing the current textured mesh according to the first patch and the two-dimensional texture map corresponding to the first patch.

Description

Technical Field

[0001] The present application relates to the technical field of trellis coding and decoding, and in particular to a trellis decoding method and device. Background Art

[0002] A 3D mesh with texture is a common digital representation of real-world objects. The basic principle is: when rendering a 3D mesh, a 2D texture map (atlas) is mapped onto the 3D mesh to enhance the realism of the rendering. The texture map usually refers to the texture on the surface of an object, and also includes the color pattern on the smooth surface of the object. A 3D mesh usually refers to the vertices, edges, and faces of a 3D shape composed of multiple mesh elements. Mesh elements can be triangles, polygons, etc. In layman's terms, it is to paste a 2D patterned image onto a 3D mesh to enhance the realism of the display. See Figure 1 , is a schematic diagram of a 3D grid with a texture map. Figure 1 Figure (a) is a schematic diagram of a 3D mesh of a human body, Figure (b) is a schematic diagram of a 2D texture map of the human body, and Figure (c) is a schematic diagram of a 3D mesh with a texture map. Specifically, by mapping the 2D texture map in Figure (b) to the 3D mesh map of the human body in Figure (a), a more realistic 3D image of the human body in Figure (c) can be obtained.

[0003] In the process of encoding a multi-frame three-dimensional mesh, if the two-dimensional texture maps of the multi-frame three-dimensional mesh have spatiotemporal consistency, then the compression efficiency can be improved when these two-dimensional texture maps are subsequently subjected to video compression encoding, thereby improving the video encoding and decoding performance, and further improving the encoding and decoding performance of the three-dimensional mesh. Among them, a multi-frame two-dimensional texture map with spatiotemporal consistency refers to a multi-frame texture map that has spatial consistency in a time series. Specifically, the position of an expanded image of one or more blocks (patches) of one frame of the multi-frame three-dimensional mesh in the two-dimensional texture map of the frame mesh corresponds to (such as being the same or similar to) the position of an expanded image of a corresponding patch of another frame mesh in the texture map of the other frame mesh. Among them, a patch includes one or more mesh elements. "Identical" includes being exactly the same or substantially the same. See. Figure 2 , is a schematic diagram of a texture map of a multi-frame three-dimensional grid. Figure 2 Figure (a) shows the texture map of multi-frame 3D meshes arranged arbitrarily in space, that is, the texture map of multi-frame 3D meshes without spatiotemporal consistency; Figure 2 Figure (b) shows the texture map of a multi-frame 3D mesh with spatiotemporal consistency.

[0004] Therefore, how to achieve spatiotemporal consistency between the two-dimensional texture maps of multi-frame three-dimensional networks, thereby improving the encoding and decoding performance of three-dimensional meshes, has become a technical problem that needs to be solved urgently. Summary of the invention

[0005] The embodiments of the present application provide a mesh decoding method and device, which helps to achieve spatiotemporal consistency between two-dimensional texture images of a multi-frame three-dimensional network, thereby improving the encoding and decoding performance of the three-dimensional mesh. In the embodiments of the present application, encoding and decoding are collectively referred to as decoding.

[0006] In a first aspect, a grid decoding method is provided, comprising: first, determining the grid elements contained in the first patch in the current grid according to the grid elements contained in the reference block patch in the reference grid of the current grid; wherein the grid elements contained in the reference patch correspond to the grid elements contained in the first patch; and then reconstructing the current textured grid according to the first patch and the two-dimensional texture map corresponding to the first patch. This technical solution performs patch division on the current grid based on the correspondence between the grid elements contained in the reference grid and the grid elements contained in the current grid, which helps to achieve spatiotemporal consistency between the patch division results of the multi-frame grids, thereby helping to achieve spatiotemporal consistency between the two-dimensional texture maps of the multi-frame three-dimensional network, thereby improving the encoding and decoding performance of the three-dimensional grid.

[0007] Optionally, the reference grid refers to a grid that is referenced (or based on) in the process of performing patch division on the current grid (specifically, the initial patch division process). Grid elements with corresponding relationships in different grids indicate the same part in the different grids.

[0008] In a possible design, the method further includes: obtaining the correspondence between the grid elements in the current grid and the grid elements in the reference grid. Based on this, the grid elements included in the first patch in the current grid are determined according to the grid elements included in the reference patch in the reference grid of the current grid, including: determining the grid elements included in the first patch according to the correspondence between the grid elements in the current grid and the grid elements in the reference grid, and the grid elements included in the reference patch. In this way, according to the correspondence between the grid elements in the current grid and the grid elements in the reference grid, the grid elements in the first patch in the current grid can be quickly determined, so that the patch division results have temporal and spatial consistency, thereby improving the encoding and decoding performance of the three-dimensional grid.

[0009] In one possible design, the current grid includes a first grid element, and obtaining the correspondence between the grid elements in the current grid and the grid elements in the reference grid specifically includes: determining the grid element in the reference grid that corresponds to the first grid element based on the voxel cube corresponding to the first grid element.

[0010] In a possible design, the reference grid and the current grid are both reference grids, and according to the voxel cube corresponding to the first grid element, the grid element in the reference grid and corresponding to the first grid element is determined, including: according to the voxel cube corresponding to the first grid element, a first preset range is obtained in the current grid, and a second preset range is obtained in the reference grid. The first preset range is a range containing the voxel cube in the current grid, and the position of the second preset range in the reference grid is the same as the position of the first preset range in the current grid. Then the second grid element in the candidate grid element set is used as the grid element corresponding to the first grid element. Among them, the candidate grid element set is a set composed of grid elements within the second preset range; the second grid element is the grid element in the candidate grid element set that is closest to the target grid element, and the distance between the second grid element and the target grid element is less than or equal to the preset distance; the position of the target grid element in the reference grid is the same as the position of the first grid element in the current grid. In this way, through the correspondence between the grid elements and the voxel cubes, the correspondence between the grid elements can be quickly determined, and the parallelism is high, which can greatly reduce the calculation time.

[0011] In one possible design, the current grid also includes a third grid element, and the method further includes: if there is no corresponding relationship between the third grid element and each grid element in the reference grid, then the third grid element is used as the second patch in the current grid. In this case, reconstructing the current textured grid according to the first patch and the two-dimensional texture map corresponding to the first patch may include: reconstructing the current textured grid according to the first patch, the two-dimensional texture map corresponding to the first patch, the second patch, and the two-dimensional texture map corresponding to the second patch.

[0012] In a possible design, the method further includes: constructing an optimization function according to the patches divided by the first N frames of the current grid; wherein the optimization function includes at least one of the following constraints: the consistency of the patches divided by the multi-frame grids, the visibility of the texture gaps, the degree of easy flattening of the patches, the degree of easy packing of the patches, and the spatial utilization of the texture map; N≥1, and N is an integer. Optionally, the optimization function may also include at least two of the constraints. Then, according to the optimization function, the patches divided by the current grid are redivided to obtain the redivided patches. Based on this, the current textured grid is reconstructed according to the first patch and the two-dimensional texture map corresponding to the first patch, specifically including: reconstructing the current textured grid according to the redivided patch and the two-dimensional texture map corresponding to the redivided patch. In this way, by redividing the patch, it is helpful to obtain a more reasonable patch division result, and it is helpful to make the generated two-dimensional texture map have the characteristics of low distortion, few seams and / or high texture space utilization, thereby helping to further improve the encoding and decoding performance of the three-dimensional grid.

[0013] In one possible design, the method further includes: determining the texture coordinates of the fifth mesh element according to the texture coordinates of the fourth mesh element. The fourth mesh element is a mesh element in the reference patch, the fifth mesh element is a mesh element in the first patch, and the fourth mesh element corresponds to the fifth mesh element. In this case, reconstructing the current textured mesh according to the first patch and the two-dimensional texture map corresponding to the first patch specifically includes reconstructing the current textured mesh according to the first patch, the two-dimensional texture map corresponding to the first patch, and the texture coordinates of the fifth mesh element.

[0014] The fifth grid element may be the same as or different from the first grid element. The fifth grid element is different from the third grid element. When the fifth grid element is the first grid element, the fourth grid element may be the second grid element.

[0015] In a possible design, the current grid and the reference grid are different grids to be decoded, or the current grid and the reference grid are reference grids of different grids to be decoded.

[0016] In one possible design, the decoding method is an encoding method, which also includes: generating a code stream, the code stream including first indication information, and the first indication information is used to indicate a voxel cube corresponding to the first grid element.

[0017] In one possible design, the decoding method is an encoding method, which further includes: generating a code stream, the code stream including second indication information. When the current grid is the grid to be encoded, the second indication information is used to indicate the geometric information of the grid to be encoded; or when the current grid is a reference grid of the grid to be encoded, the second indication information is used to indicate the geometric information and deformation field parameters of the reference grid of the grid to be encoded.

[0018] In one possible design, the method further includes: parsing the code stream to obtain first indication information, the first indication information being used to indicate a voxel cube corresponding to the first grid element. Accordingly, according to the voxel cube indicated by the first indication information, a grid element in the reference grid corresponding to the first grid element is determined. In this possible design, the grid decoding method is specifically a grid decoding method.

[0019] In one possible design, the method further includes: parsing the code stream to obtain second indication information. Wherein, when the current grid is the grid to be decoded, the second indication information is used to indicate the geometric information of the grid to be decoded; or when the current grid is the reference grid of the grid to be decoded, the second indication information is used to indicate the geometric information and deformation field parameters of the reference grid of the grid to be decoded. In this case, reconstructing the current textured grid according to the first patch and the two-dimensional texture map corresponding to the first patch specifically includes: reconstructing the current textured grid according to the information indicated by the second indication information, the first patch and the two-dimensional texture map corresponding to the first patch. In this possible design, the grid decoding method is specifically a grid decoding method.

[0020] In a second aspect, a grid decoder is provided, the decoder comprising a module (or unit) for executing the method in any possible design of the first aspect above.

[0021] In a third aspect, a trellis decoding device is provided, the decoding device comprising a memory and a processor. The processor calls a program code stored in the memory to execute part or all of the steps of the method in any possible design of the first aspect.

[0022] In a fourth aspect, a computer-readable storage medium is provided, which stores a program code, wherein the program code includes instructions for executing part or all of the steps of the method in any possible design in the first aspect.

[0023] It should be understood that the beneficial effects of any of the grid decoding devices, grid decoders and computer-readable storage media provided above can correspond to the beneficial effects of the method embodiments provided in the corresponding aspects above, and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] Figure 1 is a schematic diagram of a textured three-dimensional grid that can be used in an embodiment of the present application;

[0025] Figure 2 is a schematic diagram of a texture map of a multi-frame three-dimensional grid that can be used in an embodiment of the present application;

[0026] Figure 3 is a schematic block diagram of a grid decoding system that can be used in an example of an embodiment of the present application;

[0027] Figure 4 A schematic diagram of the relationship between a to-be-decoded grid and a reference grid of the to-be-decoded grid that can be used in an embodiment of the present application;

[0028] Figure 4A A schematic diagram of the relationship between a voxel cube and a grid that can be used in an embodiment of the present application;

[0029] Figure 5 is a schematic block diagram of an encoder that can be used in an example of an embodiment of the present application;

[0030] Figure 6 A schematic diagram of a grid to be encoded, a schematic diagram of each patch in the grid to be encoded, an expanded diagram of each patch in the grid, and a schematic diagram of the expanded diagram of the grid to be encoded that can be used in an embodiment of the present application;

[0031] Figure 7 is a schematic diagram of a texture mapping that can be used in an embodiment of the present application;

[0032] Figure 8 is a schematic block diagram of a decoder that can be used in an example of an embodiment of the present application;

[0033] Fig. 9 A schematic diagram of a flow chart of a grid decoding method provided in an embodiment of the present application;

[0034] Fig.10 A schematic diagram of dividing a patch provided in an embodiment of the present application;

[0035] Fig.11 A schematic diagram of generating texture coordinates according to an embodiment of the present application;

[0036] Fig.12 A schematic flow chart of a method for searching a mesh element corresponding to a mesh element in a current mesh provided in an embodiment of the present application;

[0037] Fig.13 A schematic diagram of a process of finding a grid element corresponding to a grid element in a current grid provided by an embodiment of the present application;

[0038] Fig.14 A schematic diagram of another grid decoding method flow chart provided in an embodiment of the present application;

[0039] Fig.15 A schematic diagram of a patch chain provided in an embodiment of the present application;

[0040] Fig.16 A schematic diagram of re-dividing the patch initially divided into the current grid provided in an embodiment of the present application;

[0041] Fig.17 A schematic diagram of an initial patch partition and a repartition patch according to an embodiment of the present application;

[0042] Fig.18 A schematic diagram of the structure of a grid decoder provided in an embodiment of the present application;

[0043] Fig.19 A schematic diagram of the structure of a grid encoder provided in an embodiment of the present application;

[0044] Fig. 20 A schematic diagram of the structure of a grid decoder provided in an embodiment of the present application. DETAILED DESCRIPTION

[0045] The term "at least one" in the embodiments of the present application includes one or more. "Multiple" refers to two or more. For example, at least one of A, B and C includes: A exists alone, B exists alone, A and B exist at the same time, A and C exist at the same time, B and C exist at the same time, and A, B and C exist at the same time. In the description of the present application, "multiple" refers to two or more than two. In addition, in order to facilitate the clear description of the technical scheme of the embodiments of the present application, in the embodiments of the present application, the words "first", "second" and the like are used to distinguish the same items or similar items with substantially the same functions and effects. Those skilled in the art will understand that the words "first", "second" and the like do not limit the quantity and execution order, and the words "first", "second" and the like do not limit them to be necessarily different.

[0046] Figure 3 1 is a schematic block diagram of a three-dimensional grid decoding system 1 that can be used in an example of an embodiment of the present application. The term "grid decoding" generally refers to grid encoding or grid decoding. The encoder 100 of the grid decoding system 1 can encode the grid to be encoded according to any grid encoding method proposed in the present application. The decoder 200 of the grid decoding system 1 can decode the grid to be decoded according to the grid decoding method corresponding to the grid encoding method used by the encoder proposed in the present application.

[0047] like Figure 3As shown, the grid decoding system 1 includes a source device 10 and a destination device 20. The source device 10 generates encoded grid data. Therefore, the source device 10 can be referred to as a grid encoding device. The destination device 20 can decode the encoded grid data generated by the source device 10. Therefore, the destination device 20 can be referred to as a grid decoding device. Various implementations of the source device 10, the destination device 20, or both may include one or more processors and a memory coupled to the one or more processors. The memory may include, but is not limited to, a random access memory (RAM), a read-only memory (ROM), an electrically erasable programmable read-only memory (EEPROM), a flash memory, or any other medium that can be used to store desired program code in the form of instructions or data structures that can be accessed by a computer, as described herein.

[0048] Source device 10 and destination device 20 may comprise a variety of devices, including desktop computers, mobile computing devices, notebook (e.g., laptop) computers, tablet computers, set-top boxes, telephone handsets such as so-called “smart” phones, televisions, cameras, display devices, digital media players, video game consoles, in-vehicle computers, or the like.

[0049] The destination device 20 may receive the encoded mesh data from the source device 10 via the link 30. The link 30 may include one or more media or devices capable of moving the encoded mesh data from the source device 10 to the destination device 20. In one example, the link 30 may include one or more communication media that enable the source device 10 to send the encoded mesh data directly to the destination device 20 in real time. In this example, the source device 10 may modulate the encoded mesh data according to a communication standard (e.g., a wireless communication protocol), and may send the modulated mesh data to the destination device 20. The one or more communication media may include wireless and / or wired communication media, such as a radio frequency (RF) spectrum or one or more physical transmission lines. The one or more communication media may form part of a packet-based network, such as a local area network, a wide area network, or a global network (e.g., the Internet). The one or more communication media may include routers, switches, base stations, or other equipment that facilitates communication from the source device 10 to the destination device 20.

[0050] In another example, the encoded data may be output from the output interface 140 to the storage device 40. Similarly, the encoded mesh data may be accessed from the storage device 40 via the input interface 240. The storage device 40 may include any of a variety of distributed or locally accessed data storage media, such as a hard drive, a Blu-ray disc, a digital versatile disc (DVD), a compact disc read-only memory (CD-ROM), flash memory, volatile or non-volatile memory, or any other suitable digital storage media for storing encoded mesh data.

[0051] In another example, the storage device 40 may correspond to a file server or another intermediate storage device that may hold the encoded mesh data generated by the source device 10. The destination device 20 may access the stored mesh data from the storage device 40 via streaming or downloading. The file server may be any type of server capable of storing the encoded mesh data and sending the encoded mesh data to the destination device 20. Example file servers include a network server (e.g., for a website), a file transfer protocol (FTP) server, a network attached storage (NAS) device, or a local disk drive. The destination device 20 may access the encoded mesh data through any standard data connection, including an Internet connection. This may include a wireless channel (e.g., a Wi-Fi connection), a wired connection (e.g., a digital subscriber line (DSL), a cable modem, etc.), or a combination of the two that is suitable for accessing the encoded mesh data stored on the file server. The transmission of the encoded mesh data from the storage device 40 may be a streaming transmission, a download transmission, or a combination of the two.

[0052] Figure 3 The trellis decoding system 1 described in is merely an example, and the technology of the present application may be applicable to trellis decoding (e.g., trellis encoding or trellis decoding) devices that do not necessarily include any data communication between the trellis encoding device and the trellis decoding device. In other examples, data is retrieved from local storage, streamed over a network, and the like. The trellis encoding device may encode data and store the data to a memory, and / or the trellis decoding device may retrieve data from a memory and decode the data. In many examples, encoding and decoding are performed by devices that do not communicate with each other but only encode data to a memory and / or retrieve data from a memory and decode data.

[0053] exist Figure 3In an example, the source device 10 includes a data source 120, an encoder 100, and an output interface 140. In some examples, the output interface 140 may include a modulator / demodulator (modem) and / or a transmitter (or transmitter). The data source 120 may include a mesh capture device (e.g., a camera), a mesh archive containing previously captured mesh data, a mesh feed interface for receiving mesh data from a mesh content provider, and / or a computer graphics system for generating mesh data, or a combination of these sources of mesh data. The data input to the data source 120 may be pose information, RGBs information, depths information, etc. of the camera to generate input data for the encoder 100, such as a textured mesh to be encoded.

[0054] Encoder 100 may encode mesh data from data source 120. In some examples, source device 10 sends the encoded mesh data directly to destination device 20 via output interface 140. In other examples, the encoded mesh data may also be stored on storage device 40 for later access by destination device 20 for decoding and / or playback.

[0055] exist Figure 3 In an example of , the destination device 20 includes an input interface 240, a decoder 200, and a display device 220. In some examples, the input interface 240 includes a receiver and / or a modem. The input interface 240 may receive the encoded mesh data via the link 30 and / or from the storage device 40. The display device 220 may be integrated with the destination device 20 or may be external to the destination device 20. In general, the display device 220 displays the decoded mesh data. The display device 220 may include a variety of display devices, such as a liquid crystal display (LCD), a plasma display, an organic light-emitting diode (OLED) display, or other types of display devices.

[0056] although Figure 3 1, but in some aspects, encoder 100 and decoder 200 may be integrated with an audio encoder and decoder, respectively, and may include appropriate multiplexer-demultiplexer (MUX-DEMUX) units or other hardware and software to handle the encoding of both audio and video in a common data stream or in separate data streams. In some examples, the MUX-DEMUX units may conform to the ITU H.223 multiplexer protocol, or other protocols such as the user datagram protocol (UDP), if applicable.

[0057] The encoder 100 and the decoder 200 can each be implemented as any of a variety of circuits such as the following: one or more microprocessors, digital signal processors (digital signal processing, DSP), application specific integrated circuits (application specific integrated circuit, ASIC), field programmable gate arrays (field-programmable gate array, FPGA), discrete logic, hardware or any combination thereof for performing the method provided by the embodiment of the present application. If the present application is partially implemented in software, the device can store instructions for the software in a suitable computer-readable storage medium (for example, a non-volatile computer-readable storage medium), and can use one or more processors to execute the instructions in hardware to implement the technology provided by the embodiment of the present application. Any of the foregoing contents (including hardware, software, a combination of hardware and software, etc.) can be regarded as one or more processors. Each of the encoder 100 and the decoder 200 can be included in one or more encoders or decoders, and any of the encoders or decoders can be integrated as a part of a combined encoder / decoder (codec) in a corresponding device.

[0058] The present application may generally refer to encoder 100 as "signaling" or "sending" certain information to another device, such as decoder 200. The term "signaling" or "sending" may generally refer to the transmission of syntax elements and / or other data used to decode the compressed grid data. This transmission may occur in real time or near real time. Alternatively, this communication may occur over a period of time, such as when the syntax elements are stored in the encoded bitstream to a computer-readable storage medium at the time of encoding, and the decoding device may then retrieve the syntax elements at any time after the syntax elements are stored to such medium.

[0059] Below, the nouns involved in this application are explained:

[0060] Reference grid: In the process of dynamic 3D grid reconstruction, the grid information collected in the frame before the grid to be decoded in the grid sequence and the grid information collected in the frame to be decoded is "de-motioned" (that is, the grid information collected in each frame is transformed so that the motion corresponding to the transformed grid information is the same as the motion of a specific frame grid in the grid sequence (such as the first frame grid)), and then merged together to form a grid. Among them, the grid sequence is a sequence of multiple frames of grids that are continuous in time. As an example, in the grid sequence, the second reference grid and the first reference grid are similar in shape. Among them, the first reference grid is the reference grid of the first grid, and the second reference grid is the reference grid of the second grid. The second grid is the grid that follows the first grid in time.

[0061] Optionally, the grid sequence may be one or more groups of frames (GOF), where a GOF is a sequence of N-frame grids that are continuous in time, where N is an integer greater than 1. The embodiment of the present application does not limit the specific value of N, such as N=32 or 64. Alternatively, the grid sequence may be a sequence of partially continuous grids in a GOF.

[0062] Deformation field parameters: The reference grid is transformed into the grid to be decoded through non-rigid deformation, where the non-rigid deformation can be represented by deformation field parameters.

[0063] Specifically, in the same grid sequence, the same frame grid and its corresponding reference grid have the same topological structure, that is, the same frame grid and its corresponding reference grid have the same grid elements, the same number of grid elements, the same number of vertices, and the same connection relationship between grid elements. The reference grids of different grids to be decoded have similar geometric shapes, but the number of grid elements and the number of vertices contained can be the same or different. Figure 4 As shown, the grid n to be decoded and the reference grid of the grid n to be decoded have the same topological structure; the reference grid of the grid n-1 to be decoded and the reference grid of the grid n to be decoded have a similar geometric shape.

[0064] Voxel cube: A fixed spatial region where a three-dimensional grid is located is uniformly divided, and each sub-cube obtained is called a voxel cube. The fixed spatial region contains the entire three-dimensional grid. In one example, a voxel cube includes one or more grid elements. Figure 4A As shown, each small cube in the figure is a voxel cube.

[0065] like Figure 5 , which is a schematic block diagram of an encoder 100 that can be used in an example of an embodiment of the present application. Figure 5 In the example, the encoder 100 may include: a patch information generation module 101, a geometric information generation module 102, a texture map generation module 103, an image or video based encoding module 104, an auxiliary information encoding module 105, a geometric information encoding module 106, a multiplexing module 107 and a reconstruction module 108. Among them:

[0066] The patch information generation module 101 is used to divide the grid to be encoded into multiple patches by a certain method, obtain relevant information of the generated patches, map each divided patch to a two-dimensional plane to obtain an expanded view of the patch, and then pack the expanded views of each divided patch to generate an expanded view of the grid to be encoded. Optionally, the grid to be encoded can be a textured grid to be encoded. This application does not make specific restrictions. A patch is an area composed of connected grid elements, and the spatial positions of grid elements in the same patch are adjacent. A patch may include one or more grid elements. The grid elements may be triangles, polygons, etc. The relevant information of the patch may include but is not limited to at least one of the following information: the number of patches into which the grid to be encoded is divided, the expanded view of each patch generated by mapping from three-dimensional space to two-dimensional space, the corresponding relationship between grid elements, etc. Part of the relevant information, such as the corresponding relationship between grid elements, can be sent to the auxiliary information encoding module 105 as auxiliary information for encoding (i.e., compression encoding).

[0067] like Figure 6 As shown, it is a schematic diagram of a grid to be encoded, a patch of the grid to be encoded, an expanded diagram of each patch in the grid, and an expanded diagram of the grid to be encoded that can be used in an embodiment of the present application. Figure 6 Figure (a) is a schematic diagram of a textured grid to be encoded. Figure 6 Figure (b) is a schematic diagram of a grid to be encoded without texture. Figure 6 Figure (c) is based on Figure 6 The schematic diagram of each patch contained in the to-be-encoded grid obtained by (b) in FIG. Among them, a connected region represents a patch. Figure 6 Figure (d) in the figure is Figure 6 Schematic diagram of the expanded view of each patch in the grid to be encoded obtained after each patch shown in Figure (c) is mapped to a two-dimensional plane. Figure 6 Figure (e) is based on Figure 6 (d) is a schematic diagram of the expanded graph of all patches packaged or the expanded graph of the grid to be encoded.

[0068] Texture map generation module 103: used to generate a two-dimensional texture map of the mesh to be encoded based on the expanded image of the mesh to be encoded, the texture information, the geometric information of the mesh to be encoded generated by the geometric information generation module 102, and the output information of the reconstruction module 108. Figure 6 As shown in Figure (f) in the figure. Among them, Figure 6 Figure (f) is based on Figure 6Then, the generated two-dimensional texture map and texture coordinates are sent to the image or video-based encoding module 104 for image or video-based encoding. In one example, the geometric information includes the geometric information of the grid to be encoded; or, the geometric information of the reference grid (canonical mesh) of the grid to be encoded and the deformation field parameter (warping field parameter). Please refer to the following for the output information of the reconstruction module 108, which will not be repeated here. Texture coordinates refer to the corresponding area position information of the grid elements in the three-dimensional grid on the two-dimensional texture map. Specifically, Figure 7 Taking the triangular face shown on the three-dimensional grid as a grid element as an example, the triangular face is mapped to the two-dimensional texture map through texture coordinates (that is, the vertex coordinates of the three corners in the figure are (0.5, 0.8), (0.3, 0.2) and (0.7, 0.2) respectively).

[0069] In some embodiments of the present application, regarding texture coordinates, the texture map generation module 103 may not need to send the texture coordinates of each grid in the grid to be encoded to the image or video-based encoding module 104, but only sends the texture coordinate update amount to the image or video-based encoding module 104. The texture coordinate update amount may be the texture coordinates of the new grid element. The new grid element is a grid element in the current grid that has no corresponding relationship in the reference grid. The current grid may be a grid to be encoded or a reference grid of a grid to be encoded. Specifically, please refer to the following for the implementation method of determining the corresponding relationship between grid elements.

[0070] The texture map and texture coordinates generated by the texture map generation module 103 are sent to the image or video based encoding module 104 for encoding. Subsequently:

[0071] On the one hand, the image or video based encoding module 104 , the auxiliary information encoding module 105 and the geometric information encoding module 106 send the obtained encoding results (ie, code streams) to the multiplexing module 107 to be merged into one code stream, which can be sent to the output interface 140 .

[0072] On the other hand, the merged code stream obtained based on the multiplexing module 107 is sent to the reconstruction module 108 for grid reconstruction to reconstruct the current textured grid, and then obtain the texture coordinates and geometric information of the reconstructed current textured grid as input to the texture map generation module in the next frame of the grid to be encoded.

[0073] Specifically, the merged code stream obtained by the reconstruction module 108 is decoded to obtain a decoded two-dimensional texture map, geometric information of the current grid, and auxiliary information of the current grid. The texture coordinates of the current grid are obtained using the decoded geometric information and auxiliary information as input information for generating a two-dimensional texture map for the next frame of the grid to be encoded.

[0074] Understandably, Figure 5 The encoder 100 shown is only an example. In a specific implementation, the encoder 100 may include a Figure 5 More or fewer modules are shown in the embodiment of the present application.

[0075] like Figure 8 , which is a schematic block diagram of a decoder 200 that can be used in an example of an embodiment of the present application. Figure 8 In the example of , the decoder 200 may include: a demultiplexing module 201, an image or video based decoding module 202, a geometric information decoding module 203, an auxiliary information decoding module 204 and a reconstruction module 205. Among them:

[0076] The demultiplexing module 201 is used to send the input code stream (i.e., the merged code stream) to the corresponding decoding module. Specifically, the code stream containing the encoded texture map is sent to the image or video based decoding module 202; the code stream containing the encoded geometric information is sent to the geometric information decoding module 203, and the code stream containing the encoded auxiliary information is sent to the auxiliary information decoding module 204.

[0077] The image or video based decoding module 202 is used to decode the received encoded texture map; and then send the texture map information obtained by decoding to the grid texture information reconstruction module 205.

[0078] The geometry information decoding module 203 is used to decode the received encoded geometry information; and then send the decoded geometry information to the mesh texture information reconstruction module 205 .

[0079] The auxiliary information decoding module 204 is configured to decode the received encoded auxiliary information and send information indicating the auxiliary information obtained by decoding to the mesh texture information reconstruction module 205 .

[0080] The mesh texture information reconstruction module 205 is used to reconstruct the texture information of the mesh according to the received texture map information, geometric information and auxiliary information to obtain a reconstructed current textured mesh.

[0081] Understandably, Figure 8 The decoder 200 shown is only an example. In a specific implementation, the decoder 200 may include Figure 8More or fewer modules are shown in the embodiment of the present application.

[0082] The following is a description of the grid decoding method provided in the embodiment of the present application. Figure 3 In the grid decoding system shown, any grid decoding method described below can be executed by the source device 10 in the grid decoding, and more specifically, by the encoder 100 in the source device 10.

[0083] like Fig. 9 As shown, it is a flowchart of a grid decoding method provided in an embodiment of the present application. Fig. 9 The method shown may include the following steps:

[0084] S101: The decoder obtains the correspondence between the grid elements in the reference grid and the grid elements in the current grid.

[0085] The reference grid is a grid that is referenced (or based on) in the process of patching the current grid (specifically, the initial patching process). Optionally, the reference grid and the current grid have similar contents. For example, the reference grid and the current grid are grids representing different poses of the same person. For another example, the reference grid and the current grid both contain the same landscape area.

[0086] In one implementation, the current grid may be any frame grid to be decoded other than the first frame grid in the grid sequence. In this case, the reference grid may be any frame grid before the grid to be decoded. For example, the reference grid is the previous frame grid of the current grid.

[0087] In another implementation, the current grid may be a reference grid of any frame grid to be decoded other than the first frame grid in the grid sequence. In this case, the reference grid may be a reference grid of any frame grid before the grid to be decoded. For example, the reference grid is a reference grid of a frame grid before the current grid. It should be noted that, for ease of description, the specific examples below are all described by taking the current grid and the reference grid as the reference grid as an example.

[0088] It can be understood that for any frame or multiple frames in a grid sequence (such as the first frame grid in the grid sequence, etc.), the decoder can use the patch division method in the prior art (for example, random sampling block division method, semantics-preserving division method, or patch block flattening method, etc.) to patch the one or multiple frame grids.

[0089] The mesh elements with corresponding relationships in different meshes indicate the same part in the different meshes. For example, assuming that the reference mesh and the current mesh are different meshes representing the same person, then the mesh element representing the nose of the person in the reference mesh has a corresponding relationship with the mesh element representing the nose of the person in the current mesh.

[0090] When both the reference grid and the current grid are reference grids, the grid elements with corresponding relationships in different grids indicate the same part in the different grids, specifically including: the grid elements with corresponding relationships in different grids indicate the same position in the different grids. For example, assuming that the reference grid and the current grid are reference grids representing the same person, then the position of the grid element representing the tip of the person's index finger in the reference grid is the same as the position of the grid element representing the tip of the person's index finger in the current grid.

[0091] In the specific implementation process, the reference grid may contain grid elements corresponding to some or all grid elements in the current grid, or may not contain grid elements corresponding to any grid element in the current grid. It can be understood that, for a given current grid, the higher the similarity between the content represented by the reference grid and the current grid, the more grid elements corresponding to the grid elements contained in the current grid can be found in the reference grid by the decoder.

[0092] Optionally, S101 may include: the decoder searches (such as serially or in parallel) whether each grid element in the current grid has a corresponding grid element in the reference grid, thereby obtaining the correspondence between the grid elements in the reference grid and the grid elements in the current grid.

[0093] The embodiment of the present application does not limit how the decoder obtains the correspondence between the grid elements in the reference grid and the grid elements in the current grid. Specifically, the decoder can obtain the correspondence through methods in the prior art. For example, the correspondence is obtained by establishing a tree structure through an approximate nearest neighbor (ANN) library. In addition, the embodiment of the present application also provides a method for obtaining the correspondence based on a voxel cube. Specifically, based on the voxel cube corresponding to any grid element in the current grid, the grid element corresponding to the grid element in the reference grid is searched. For example, refer to the following Fig.12 and Fig.13 .

[0094] Optionally, if the correspondence between mesh elements in the reference mesh and mesh elements in the current mesh is obtained based on voxel cubes, then:

[0095] When the decoder is an encoder, the method further includes: the encoder generates a code stream, the code stream includes first indication information, and the first indication information is used to indicate the voxel cube corresponding to each grid element in the grid to be encoded. In addition, based on this implementation, the embodiment of the present application does not limit the order between S102~S105 and the step of the encoder generating the code stream. For example, S102~S105 can be executed first, and then the encoder generates the code stream, or the encoder generates the code stream first and then S102~S105, or the encoder generates the code stream during the execution of S102~S105.

[0096] Correspondingly, when the decoder is a decoder, the decoder receives the bitstream transmitted by the encoder and parses the bitstream to obtain the first indication information for indicating the voxel cube corresponding to the grid element. Based on this, S101 may include: the decoder determines the correspondence between the grid elements in the reference grid and the grid elements in the current grid according to the voxel cube corresponding to at least one grid element (such as each grid element) in the grid to be encoded indicated by the first indication information.

[0097] It is understandable that if S101 of acquiring the correspondence between the grid elements in the reference grid and the grid elements in the current grid is implemented based on a method in the prior art, then the encoder may not need to encode the first indication information into the bitstream.

[0098] S102: The decoder divides the current grid into at least one patch according to the correspondence between the grid elements in the reference grid and the grid elements in the current grid.

[0099] Optionally, the decoder may search for grid elements in the current grid that correspond to each grid element included in the reference patch, based on the correspondence between the grid elements in the reference grid and the grid elements in the current grid (this step is marked as step 1). The reference patch is any patch in the reference grid. If found, the set of grid elements found is taken as a patch (marked as the first patch). Some or all of the grid elements included in the reference patch correspond one-to-one to the grid elements included in the first patch. If not found, another grid element in the reference grid is taken as the reference patch, and the process returns to step 1. This continues until all patches in the reference grid are traversed (i.e., the method described in this paragraph is executed for each patch in the reference grid).

[0100] If there are no remaining mesh elements in the current mesh after traversing all patches in the reference mesh, the process of patching the current mesh ends. If there are still mesh elements in the current mesh that do not correspond to any mesh element in the reference mesh after traversing all patches in the reference mesh (marked as remaining mesh elements), each remaining mesh element is treated as a patch in the current mesh. At this point, the process of patching the current mesh ends.

[0101] In order to facilitate understanding of this optional implementation, the following Fig.10 This optional implementation is described.

[0102] Fig.10 Figure (a) in FIG. 1 mainly shows the correspondence between the grid elements in the reference grid and the grid elements in the current grid, as well as the patch division of the reference grid. Fig.10 Figure (b) in the figure mainly shows the patch division of the current grid determined based on Figure a.

[0103] exist Fig.10 In , each node represents a grid element. Fig.10 The first columns in (a) and (b) of FIG. 1 represent the reference grid, including grid elements 11 to 18; the second columns represent the current grid, including grid elements 21 to 27. On the time axis t (i.e., time sequence), the reference grid is located before the current grid.

[0104] exist Fig.10 In the example, for different grid elements in the same frame grid, if they are connected by a vertical edge, it means that the different grid elements are spatially adjacent, that is, they can be used as grid elements in the same patch. Of course, in the specific implementation process, whether to use these different grid elements as grid elements in the same patch needs to be determined based on the optional implementation method in S102. For example, grid element 11 and grid element 12 are spatially adjacent grid elements.

[0105] exist Fig.10 In the example, for different grid elements in different frame grids, if they are connected by a horizontal edge, it means that there is a corresponding relationship between the different grid elements. For example, there is a corresponding relationship between grid element 12 and grid element 21.

[0106] exist Fig.10 In the figure, the mesh elements in the same dashed box represent the mesh elements in the same patch. Fig.10 In Figure (a), mesh element 11 forms a patch; mesh element 12 and mesh element 13 form a patch.

[0107] based on Fig.10 As shown in Figure (a), the decoder can perform the following steps 1 to 5 to implement patch partitioning of the current grid to obtain Fig.10 The patch division is shown in Figure (b).

[0108] Step 1: Search the current grid for a mesh element corresponding to mesh element 11 contained in patch 11. The result of this search is that no mesh element is found.

[0109] Step 2: Search for mesh elements corresponding to mesh elements 12 and 13 contained in patch 12 in the current mesh. The result of this search is that only mesh element 21 corresponding to mesh element 12 is found. Therefore, mesh element 21 can be used as a patch in the current mesh (marked as patch 21).

[0110] Step 3: Search for mesh elements corresponding to mesh elements 14, 15 and 16 contained in patch 13 in the current grid. The result of this search is that mesh elements 22, 23 and 24 corresponding to mesh elements 14, 15 and 16 are found respectively. Therefore, mesh elements 22, 23 and 24 can be used as a patch in the current grid (marked as patch 22).

[0111] Step 4: Search for mesh elements corresponding to mesh elements 17 and 18 contained in patch 14 in the current mesh. The result of this search is that mesh elements 26 and 27 corresponding to mesh elements 17 and 18 respectively are found. Therefore, mesh elements 26 and 27 can be used as a patch in the current mesh (marked as patch 23).

[0112] The above steps 1 to 4 may be executed in series or in parallel. Moreover, the embodiment of the present application does not limit the execution order of steps 1 to 4.

[0113] Step 5: After executing the above steps 1 to 4, there are remaining mesh elements in the current mesh, namely, mesh element 25. In this case, mesh element 25 can be used as a patch in the current mesh (marked as patch 24).

[0114] At this point, the current grid is divided into patches 21 to 24.

[0115] S103: The decoder obtains a two-dimensional texture map corresponding to each patch in the at least one patch. The specific implementation of this step can refer to the prior art and will not be described in detail here.

[0116] S104: The decoder obtains the texture coordinates of each mesh element in the current mesh.

[0117] Optionally, for any mesh element in the current mesh (marked as the first mesh element), if there is a mesh element corresponding to the first mesh element in the reference mesh, the texture coordinates of the first mesh element are determined according to the texture coordinates of the mesh element corresponding to the first mesh element. For example, the texture coordinates of the first mesh element are obtained by barycentric interpolation of the texture coordinates of the mesh element corresponding to the first mesh element. Fig.11 As shown, the grid element F in the current grid n,i The grid element F in the reference grid n-1,j Corresponding (as shown in step ①), therefore, the grid element F in the reference grid can be n-1,j The texture coordinates of each vertex Generate the grid element F in the current grid by barycentric interpolation n,i The texture coordinates of each vertex (As shown in step ②) In addition, if there is no mesh element corresponding to the first mesh element in the reference mesh, the texture coordinates of the first mesh element can be obtained based on the method in the prior art.

[0118] Based on this optional implementation, when the decoder is an encoder, the encoder may not need to encode the texture coordinates of the first mesh element into the bitstream, so that the bitstream transmission overhead can be saved. In addition, based on this optional implementation, the embodiment of the present application does not limit the order between S102~S103 and S104. For example, S102~S103 can be executed first and then S104, or S104 can be executed first and then S102~S103, or S104 can be executed during the execution of S102~S103.

[0119] Of course, as an alternative implementation of this optional implementation, the decoder may determine the texture coordinates of the first mesh element directly based on the method provided in the prior art instead of determining the texture coordinates of the first mesh element based on whether the corresponding mesh element can be found in the reference mesh. In this case, the order between S101 to S103 and S104 is not limited in the embodiment of the present application.

[0120] S105: The decoder reconstructs the current textured mesh according to the two-dimensional texture map corresponding to each patch in the at least one patch and the texture coordinates of the mesh elements (such as each mesh element) in the current mesh.

[0121] Specifically, the decoder reconstructs the current textured mesh according to the first patch, the two-dimensional texture map corresponding to the first patch, and the texture coordinates of each mesh element in the first patch. Alternatively, the decoder reconstructs the current textured mesh according to the first patch, the two-dimensional texture map corresponding to the first patch, the texture coordinates of each mesh element in the first patch, the second patch, the two-dimensional texture map corresponding to the second patch, and the texture coordinates of each mesh element in the second patch.

[0122] The specific implementation method of reconstructing the current textured grid can refer to the prior art or the above Figure 5 Notes on refactoring in .

[0123] Optionally, when the decoder is an encoder, the code stream generated by the encoder also includes second indication information. Wherein, when the current grid is the grid to be encoded, the second indication information is used to indicate the geometric information of the grid to be encoded; or, when the current grid is the reference grid of the grid to be encoded, the second indication information is used to indicate the geometric information and deformation field parameters of the reference grid of the grid to be encoded. In addition, based on this optional implementation method, the embodiment of the present application does not limit the execution order between the encoder generating the code stream and steps S101 to S104. For example, the code stream can be generated after executing steps S101 to S104, or the code stream can be generated first and then steps S101 to S104 can be executed, or the code stream can be generated during the execution of S101 to S104.

[0124] Correspondingly, when the decoder is a decoder, the decoder receives the bitstream transmitted by the encoder and parses the bitstream to obtain the second indication information. The above step S105 may include: the decoder reconstructs the current textured mesh according to the geometric information indicated by the second indication information, the at least one patch and the two-dimensional texture map corresponding to the patch.

[0125] The grid decoding method provided in this embodiment performs patch division on the current grid according to the correspondence between the grid elements in the reference grid and the grid elements in the current grid, as well as the patch division result of the reference grid. If the continuous multi-frame grids in the grid sequence are all patch divided based on this method, it is helpful to achieve the temporal and spatial consistency between the patch division results of the continuous multi-frame grids. For example, Fig.17 Figure (a) is a schematic diagram of the reference grid partition patch; Fig.17 Figure (b) is a schematic diagram of the patch division result obtained by dividing the current grid based on the patch division result of the reference grid; Fig.17Figure (c) is a schematic diagram of the result of partitioning patches of consecutive multiple frames of meshes based on the method in Figure (b). Based on the result of partitioning patches of consecutive multiple frames of meshes, it helps to generate a two-dimensional texture map with spatio-temporal consistency, thereby helping to improve the compression efficiency of the two-dimensional texture map, and further improving the encoding and decoding performance of the three-dimensional mesh.

[0126] Next, in combination with Fig.12 and Fig.13 This paper describes a method for finding a mesh element corresponding to a mesh element in the current mesh provided by an embodiment of the present application. This method can be a specific implementation manner of the above S101.

[0127] Fig.12 It is a schematic flowchart of a method for finding a mesh element corresponding to a mesh element in the current mesh. Fig.13 Based on Fig.12 The provided method is a schematic diagram of the process of finding a mesh element corresponding to a mesh element in the current mesh. Fig.12 and Fig.13 It is described by taking "the mesh sequence includes meshes 1 to N, the reference mesh is the (n - 1)-th frame mesh in the mesh sequence, the current mesh is the n-th frame mesh in the mesh sequence, N and n are both positive integers, and 1 < n ≤ N" as an example.

[0128] Fig.12 The method shown includes the following steps:

[0129] S101-1: The decoder obtains the voxel cube ni corresponding to the mesh element F F n,i represents the i-th mesh element in the n-th frame mesh. This step can correspond to Fig.13 step ① in

[0130] Among them, the embodiment of the present application does not specifically limit the method for obtaining the voxel cube corresponding to the mesh element. Specifically, the voxel cube corresponding to the mesh element can be obtained by a method in the prior art. For example: The Marching Cubes algorithm can be used to generate a three-dimensional mesh from the truncated signed distance function (TSDF). Each mesh element in the three-dimensional mesh belongs to a unique voxel cube. Among them, TSDF is a method for implicitly representing a three-dimensional model.

[0131] S101-2: The decoder according to the voxel cube Determine a first preset range, and determine a second preset range in the reference grid based on the first preset range. The position of the first preset range in the current grid is the same as the position of the second preset range in the reference grid. Search for grid elements included in the grid of the n-1th frame within the second preset range. In this embodiment, the grid elements included in the second preset range are referred to as a candidate grid element set. This step may correspond to Fig.13 Steps ② and ③ in .

[0132] The first preset range is the voxel cube contained in the current grid. range.

[0133] Optionally, the first preset range includes a voxel cube And with voxel cube Adjacent voxel cubes located above, below, left, right, in front of, and behind the voxel cube. Fig.13 In the example, mark the preset range as a voxel cube collection

[0134] Of course, the specific implementation is not limited thereto, for example, the first preset range may include a range defined by the voxel cube as the center point and a radius of R, wherein R is greater than 0. It can be understood that the smaller the preset range value is, the more accurate the corresponding relationship found in the reference grid is.

[0135] Specifically, Fig.13 As shown in steps ② and ③, find the voxel cube set in the reference grid All the grid elements contained in the set, all the found grid elements constitute the candidate grid element set

[0136] S101-3: The decoder determines whether the candidate grid element set is a non-empty set.

[0137] If yes, then execute S101-4. If no, then execute S101-7.

[0138] S101-4: The decoder determines the grid element F in the candidate grid element set that is closest to the target grid element. n-1,j Among them, the position of the target grid element in the reference grid is the same as the grid element F n,i Same as in the current grid. F n-1,j Represents the j-th grid element in the n-1-th frame grid, where j is an integer greater than or equal to 1.

[0139] If the candidate grid element set includes a target grid element, the target grid element is used as the grid element F n,iIf the candidate grid element set does not contain the target grid element, the grid element F is determined according to the following steps: n,i The corresponding grid element.

[0140] S101-5: Decoder determines grid element F n-1,j Whether the distance to the target mesh element is less than or equal to the preset threshold.

[0141] If yes, then execute S101-6. If no, then execute S101-7.

[0142] The embodiments of the present application do not limit the specific value of the preset threshold. It can be understood that the more grid elements in the current grid have a corresponding relationship with the reference grid, the more grid elements in the current grid have texture coordinates that do not need to be transmitted in the bitstream, thereby saving the bitstream transmission overhead; based on this, the value of the preset threshold can be larger. On the other hand, the larger the value of the preset threshold, the lower the accuracy of the grid elements with a corresponding relationship determined based on the preset threshold. Therefore, in a specific implementation, the value of the preset threshold can be determined based on factors such as the accuracy of the corresponding relationship between grid elements in different frames and the saving of bitstream transmission overhead.

[0143] S101-6: Decoder determines grid element F n-1,j is the grid element F n,i The corresponding grid element.

[0144] After executing S101-6, the process ends.

[0145] S101-7: Decoder determines grid element F n,i There is no corresponding mesh element in the reference mesh.

[0146] After executing S101-7, the process ends.

[0147] The method provided in this embodiment is based on obtaining the correspondence between the grid elements in the current grid and the corresponding voxel cubes to obtain the correspondence between the grid elements in the current grid and the grid elements in the reference grid. Compared with the method of obtaining the correspondence between the grid elements in the prior art (for example: ANN), the correspondence between the grid elements can be quickly obtained, thereby quickly generating a two-dimensional texture map with temporal and spatial consistency.

[0148] Below, through Fig.14 Another grid decoding method provided in an embodiment of the present application is described. In this embodiment, a grid sequence is described as a sequence of N grid frames that are continuous in time, where N is an integer greater than 1. Fig.14 The methods shown include:

[0149] S201-S202: refer to the above S101-S102. Of course, the embodiments of the present application are not limited thereto.

[0150] S203: The decoder obtains a patch chain composed of patches divided by the previous R frame grids of the current grid. The current grid is the nth frame grid in the grid sequence, and n and R are both integers, and 1 <n<N,R≥2。

[0151] A patch chain refers to a set of patches with corresponding relationships in the continuous previous R frame grids in the current grid. Assuming that the previous R frame grid includes grid 1 and grid 2, grid 1 and grid 2 are adjacent, grid 1 includes patch 1, grid 2 includes patch 2, and there is a corresponding relationship between the grid elements contained in patch 1 and the grid elements contained in patch 2, then there is a corresponding relationship between patch 1 and patch 2. Assuming that the previous R frame grid also includes grid 3, and grid 2 and grid 3 are adjacent, grid 3 includes patch 3, and there is a corresponding relationship between patch 2 and patch 3, then there is a corresponding relationship between patch 1, patch 2 and patch 3, and the set consisting of patch 1, patch 2 and patch 3 is called a patch chain. For example, Fig.15 As shown in the figure, pathc1, patch2 and patch3 in the dashed box form a patch chain. Fig.15 The example of R=3 is used for explanation.

[0152] S204: The decoder constructs an optimization function according to the patches divided by the previous R frame grid of the current grid, wherein the optimization function includes at least one of the following constraints: consistency of the patches divided by the multi-frame grids, visibility of texture gaps, ease of patch flattening, ease of patch packing, and spatial utilization of the texture map.

[0153] Optionally, give an optimization function, e.g., E total =f(x*E coherence ,y*E seamless ,z*E parameterization ), where E coherence It is used to measure the consistency of the patches divided by the multi-frame grid (specifically the previous R frame grid of the current grid in this example), E seamless Used to measure the visibility of texture gaps, E parameterization It is used to measure how easy it is to flatten a patch. x, y and z are the weights of the corresponding parameters, x>0, y>0, z>0.

[0154] S205: The decoder re-divides at least one patch divided into the current grid (hereinafter referred to as the patch initially divided into the current grid) according to the optimization function and the patch chain formed by the previous R frame grid of the current grid to obtain a re-divided patch.

[0155] like Fig.16 As shown in FIG. 1 , a schematic diagram of re-dividing the patch of the current grid initially divided is provided in an embodiment of the present application. Among them, the patch result of the current grid initial division is as follows: Fig.16 As shown in Figure (a). Specifically, mesh element 11 is patch1, mesh element 12 is patch2, mesh elements 13 and 14 constitute patch3, mesh element 15 is patch4, and mesh element 16 is patch5. Based on this, according to the constructed optimization function, patch4 and patch5 respectively constituted by mesh elements 15 and 16 can be optimized into a patch6. Of course, depending on the weights of the parameters in the constructed optimization function, the optimization results may also be different. For example, if more attention is paid to fewer seams in the two-dimensional texture image, the parameter E is given. seamless A larger weight is used to obtain a two-dimensional texture map generated by the re-divided patch with fewer seams. This embodiment of the present application does not specifically limit this.

[0156] Optionally, when the constraint of the constructed optimization function includes the consistency of the patches divided by the multi-frame grids, step S205 may include: merging multiple patch chains that meet the preset conditions into one patch chain, and re-dividing the patch of the initial division of the current grid according to the merged patch division result to obtain the re-divided patch. The preset conditions include: the patches belonging to the same frame in the multiple patch chains are spatially adjacent, and the grid elements contained in the patches belonging to adjacent grids in the same patch chain correspond one to one. Fig.16 As shown in : the first patch chain is spatially adjacent to the second patch chain, and the mesh elements contained in the patches of adjacent meshes in the same patch chain (e.g., the first patch chain or the second patch chain) correspond one to one. Therefore, the first patch chain can be merged with the second patch chain to obtain a third patch chain, as shown in Fig.16 As shown in Figure (b), patch2 and patch3 in the current grid are redivided according to the merged third patch chain, and finally patch2 and patch3 are merged into one patch to obtain patch7.

[0157] In this embodiment, the initial divided patches of the current grid are re-divided.

[0158] Optionally, in the specific implementation process, when the current grid is initially divided into patches, the decoder usually has cached the patch results of the initial division of the grids of the previous frames. At this time, when performing patch re-division, the grids of the previous frames and the results of the initial division of the current grid can be combined for re-division, so as to serve as a reference for dividing the next frame of the grid to be decoded into patches. Fig.17 Figure (c) is a schematic diagram of the result of the initial partition of a 4-frame grid. After redividing it, we get Fig.17 The result of patch redivision in Figure (d) is that not only the patch of the 4th frame grid in the figure is redivisioned, but also the 3 frames before the 4th frame grid are redivisioned.

[0159] S206: The decoder obtains the two-dimensional texture map corresponding to each patch in the re-divided patch and the texture coordinates of each mesh element in the current mesh.

[0160] Specifically, the specific implementation method of obtaining the two-dimensional texture map corresponding to each patch in this step can refer to the prior art. The implementation method of obtaining the texture coordinates of each grid element in the current grid can refer to the implementation method in S104 above, which will not be repeated here.

[0161] S207: The decoder reconstructs the current textured mesh according to the two-dimensional texture map corresponding to each patch in the re-divided patch and the texture coordinates of the mesh elements (such as each mesh element) in the current mesh.

[0162] The specific implementation method of reconstructing the current textured grid may refer to the prior art.

[0163] The beneficial effects of the trellis decoding method provided in this embodiment can be Fig. 9 Beneficial effects of the embodiment shown. In addition, in this embodiment, the patch initially divided into the current grid is re-divided, which helps to make the generated two-dimensional texture map have the characteristics of low distortion, few seams and / or high texture space utilization, thereby helping to further improve the encoding and decoding performance of the three-dimensional grid.

[0164] The above mainly introduces the solution provided by the embodiment of the present application from the perspective of the method. In order to achieve the above functions, it includes hardware structures and / or software modules corresponding to the execution of each function. Those skilled in the art should easily realize that, in combination with the modules and algorithm steps of each example described in the embodiments disclosed herein, the present application can be implemented in the form of hardware or a combination of hardware and computer software. Whether a function is executed in the form of hardware or computer software driving hardware depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to exceed the scope of the present application.

[0165] The embodiment of the present application can divide the encoder / decoder into functional modules according to the above method example. For example, each functional module can be divided corresponding to each function, or two or more functions can be integrated into one processing module. The above integrated module can be implemented in the form of hardware or in the form of software functional modules. It should be noted that the division of modules in the embodiment of the present application is schematic and is only a logical function division. There may be other division methods in actual implementation.

[0166] like Fig.18 , which is a schematic block diagram of a grid decoder 180 provided in an embodiment of the present application. The grid decoder 180 can be used to execute any grid decoding method provided in an embodiment of the present application, such as Fig. 9 or Fig.14 The grid decoding method shown.

[0167] The grid decoder 180 may include a first determination module 1801 and a reconstruction module 1802. Specifically: when the grid decoder 180 is a grid encoder, the operation performed by the first determination module 1801 may be specifically as described above. Figure 4 The operations performed by the patch information generation module 101 in the encoder 100 can be specifically performed by Figure 4 The reconstruction module 108 in the encoder 100 executes. When the trellis decoder 180 is a trellis decoder, the operations executed by the first determination module 1801 and the reconstruction module 1802 may be specifically executed by the reconstruction module 205 of the above decoder.

[0168] In some embodiments, the first determination module 1801 is used to determine the mesh elements included in the first patch in the current mesh according to the mesh elements included in the reference patch in the reference mesh of the current mesh. The mesh elements included in the reference patch correspond to the mesh elements included in the first patch. The reconstruction module 1802 is used to reconstruct the current textured mesh according to the first patch and the two-dimensional texture map corresponding to the first patch. For example, in combination with Fig. 9 The first determination module 1801 can be used to execute S101-S102, and the reconstruction module 1802 can be used to execute S105.

[0169] Optionally, the first determination module 1801 is further used to obtain the correspondence between the mesh elements in the current mesh and the mesh elements in the reference mesh. When the first determination module 1801 determines the mesh elements included in the first patch in the current mesh according to the mesh elements included in the reference patch in the reference mesh of the current mesh, it is specifically used to: determine the mesh elements included in the first patch according to the correspondence between the mesh elements in the current mesh and the mesh elements in the reference mesh, and the mesh elements included in the reference patch. For example, in combination Fig.12 , the first determining module 1801 can be used to execute S101 - 1 to S101 - 7.

[0170] Optionally, the current grid includes a first grid element, and the first determination module 1801, when executing the acquisition of the correspondence between the grid elements in the current grid and the grid elements in the reference grid, is specifically used to: determine the grid element in the reference grid that corresponds to the first grid element based on the voxel cube corresponding to the first grid element.

[0171] Optionally, both the reference grid and the current grid are reference grids; when the first determination module 1801 determines the grid element in the reference grid and corresponding to the first grid element according to the voxel cube corresponding to the first grid element, it is specifically used to: obtain a first preset range in the current grid and a second preset range in the reference grid according to the voxel cube corresponding to the first grid element. The first preset range is a range in the current grid that includes the "voxel cube corresponding to the first grid element", and the position of the second preset range in the reference grid is the same as the position of the first preset range in the current grid. Then, the second grid element in the candidate grid element set is used as the grid element corresponding to the first grid element. The candidate grid element set is a set of grid elements within the second preset range; the second grid element is the grid element in the candidate grid element set that is closest to the target grid element, and the distance between the second grid element and the target grid element is less than or equal to the preset distance; the position of the target grid element in the reference grid is the same as the position of the first grid element in the current grid.

[0172] Optionally, the first determination module 1801 is further configured to: if the current grid also includes a third grid element, and if the third grid element does not have a corresponding relationship with any grid element in the reference grid, then the third grid element is used as the second patch in the current grid. The reconstruction module 1802 is specifically configured to: reconstruct the current textured grid according to the first patch, the two-dimensional texture map corresponding to the first patch, the second patch, and the two-dimensional texture map corresponding to the second patch. For example, in combination with Fig. 9 , the first determination module 1801 can be used to execute S101 and S102. The reconstruction module 1802 can execute S105. For another example, the first determination module 1801 can be used to execute steps 1 to 5.

[0173] Optionally, the first determination module 1801 is used to construct an optimization function according to the patches divided by the first N frames of the current grid; then, according to the optimization function, the patches divided by the current grid are re-divided to obtain the re-divided patches. The optimization function includes at least one of the following constraints: the consistency of the patches divided by the multi-frame grids, the visibility of the texture gaps, the ease of flattening the patches, the ease of packing the patches, and the spatial utilization of the texture map; N≥1, and N is an integer; the patches divided by the current grid include the first patch. Optionally, the optimization function may also include at least two of the constraints. The reconstruction module 1802 is specifically used to reconstruct the current textured grid according to the re-divided patches and the two-dimensional texture map corresponding to the re-divided patches. For example, combined with Fig.14The first determining module 1801 can be used to execute S201 to S205. The reconstructing module 1802 can be used to execute S206 and S207.

[0174] Optionally, the mesh decoder 180 further includes a second determination module 1803, which is used to determine the texture coordinates of the fifth mesh element according to the texture coordinates of the fourth mesh element. The fourth mesh element is a mesh element in the reference patch, the fifth mesh element is a mesh element in the first patch, and the fourth mesh element corresponds to the fifth mesh element. The reconstruction module 1802 is specifically used to reconstruct the current textured mesh according to the first patch, the two-dimensional texture map corresponding to the first patch, and the texture coordinates of the fifth mesh element. For example, in combination Fig. 9 , the reconstruction module 1402 can be used to perform step S104. Specifically, when the grid decoder 180 is a grid encoder, the operation performed by the second determination module 1803 can be the above Figure 4 When the grid decoder 180 is a grid decoder, the operation performed by the second determination module 1803 can be performed by the texture map generation module 103 in the encoder 100. Figure 8 The reconstruction module 205 in the decoder 200 is executed.

[0175] Optionally, the current grid and the reference grid are different grids to be decoded, or the current grid and the reference grid are reference grids of different grids to be decoded.

[0176] Optionally, when the grid decoder 180 is specifically a grid encoder, such as Fig.19 As shown, the grid encoder may further include an encoding module 1804. The encoding module 1804 is used to: generate a code stream, the code stream including first indication information, the first indication information is used to indicate the voxel cube corresponding to the first grid element. For example, the operation performed by the encoding module 1804 may be the same as the above Figure 4 The auxiliary information encoding module 105 in the encoder 100 is executed.

[0177] Optionally, when the grid decoder 180 is a grid encoder, the encoding module 1804 is used to: generate a code stream, the code stream includes the second indication information. When the current grid is the grid to be encoded, the second indication information is used to indicate the geometric information of the grid to be encoded; or when the current grid is the reference grid of the grid to be encoded, the second indication information is used to indicate the geometric information and deformation field parameters of the reference grid of the grid to be encoded. For example, the operation performed by the encoding module 1804 can be the above Figure 4 The geometric information encoding module 106 in the encoder 100 is executed.

[0178] Optionally, when the grid decoder 180 is specifically a grid decoder, such as Fig. 20As shown, the grid decoder may further include a decoding module 1805. The decoding module 1805 is used to: parse the bitstream to obtain first indication information, where the first indication information is used to indicate the voxel cube corresponding to the first grid element. The first determination module 1801 is used to determine the grid element in the reference grid and corresponding to the first grid element according to the voxel cube indicated by the first indication information. For example, the operation performed by the decoding module 1805 may be the same as that described above. Figure 8 The auxiliary information decoding module 204 in the decoder 200 is executed.

[0179] Optionally, when the grid decoder 180 is a grid decoder, the decoding module 1805 is used to: parse the code stream to obtain second indication information. Wherein, when the current grid is the grid to be decoded, the second indication information is used to indicate the geometric information of the grid to be decoded; or when the current grid is the reference grid of the grid to be decoded, the second indication information is used to indicate the geometric information and deformation field parameters of the reference grid of the grid to be decoded. The reconstruction module 1802 is specifically used to reconstruct the current textured grid according to the information indicated by the second indication information, the first patch and the two-dimensional texture map corresponding to the first patch. For example, the operation performed by the decoding module 1805 can be the above Figure 8 The geometric information decoding module 203 in the decoder 200 is executed.

[0180] It can be understood that each module in the grid decoder 180 provided in the embodiment of the present application is a functional body for implementing the various execution steps included in the corresponding method provided above, that is, a functional body capable of fully implementing each step in the embodiment of the present application and the extension and deformation of these steps. For details, please refer to the introduction of the corresponding method above. For the sake of brevity, this article will not go into details.

[0181] Those skilled in the art will appreciate that the functions described in conjunction with the various illustrative logic blocks, modules, and algorithm steps disclosed herein can be implemented in hardware, software, firmware, or any combination thereof. If implemented in software, the functions described in the various illustrative logic blocks, modules, and steps can be stored or transmitted as one or more instructions or codes on a computer-readable medium and executed by a hardware-based processing unit. Computer-readable media can include computer-readable storage media, which corresponds to tangible media, such as data storage media, or includes any communication media that facilitates the transfer of computer programs from one place to another (e.g., according to a communication protocol). In this way, computer-readable media can generally correspond to (1) computer-readable storage media (e.g., tangible computer-readable storage media that can be non-transitory), or (2) communication media, such as signals or carrier waves. Data storage media can be accessible by one or more computers or one or more processors to execute instructions, codes for implementing the techniques described in this application. A computer program product can include computer-readable media.

[0182] As an example and not limitation, such computer-readable storage media may include RAM, ROM, EEPROM, CD-ROM or other optical disk storage, magnetic disk storage or other magnetic storage, flash memory, or any other medium that can be used to store the desired program code in the form of instructions or data structures and can be accessed by a computer. Also, any connection is properly referred to as a computer-readable medium. For example, if instructions are transmitted from a website, server or other remote source using a coaxial cable, fiber optic cable, twisted pair, digital subscriber line (DSL), or wireless technologies such as infrared, radio and microwave, then the coaxial cable, fiber optic cable, twisted pair, DSL, or wireless technologies such as infrared, radio and microwave are included in the definition of media. However, it should be understood that computer-readable storage media and data storage media do not include connections, carrier waves, signals or other temporary media, but are actually directed to non-temporary tangible storage media. As used herein, disks and optical disks include compact disks (CDs), laser optical disks, optical optical disks, digital versatile disks (DVDs), and Blu-ray disks, where disks typically reproduce data magnetically, while optical disks reproduce data optically using lasers. Combinations of the above should also be included within the scope of computer-readable media.

[0183] Instructions for implementing the methods described in the present application may be executed by one or more processors, such as one or more digital signal processors (DSPs), general-purpose microprocessors, application-specific integrated circuits (ASICs), field programmable logic arrays (FPGAs), or other equivalent integrated or discrete logic circuits. Thus, the term "processor" as used herein may refer to any of the aforementioned structures or any other structures suitable for implementing the techniques described herein. Additionally, in some aspects, the functions described by the various illustrative logic blocks, modules, and steps described herein may be provided within dedicated hardware and / or software modules configured for encoding and decoding, or incorporated in a combined codec. Moreover, the techniques may be fully implemented in one or more circuits or logic elements.

[0184] The techniques of the present application may be implemented in a variety of devices or apparatuses, including a wireless handset, an integrated circuit (IC), or a set of ICs (e.g., a chipset). Various components, modules, or units are described in this application to emphasize functional aspects of devices for performing the disclosed techniques, but do not necessarily need to be implemented by different hardware units. In fact, as described above, the various units may be combined in a codec hardware unit in conjunction with appropriate software and / or firmware, or provided by interoperating hardware units (including one or more processors as described above).

[0185] In the above embodiments, the description of each embodiment has different emphases. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0186] The above are only exemplary embodiments of the present application, but the protection scope of the present application is not limited thereto. Any changes or substitutions that can be easily thought of by a person skilled in the art within the technical scope disclosed in the present application should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. A trellis decoding method, characterized in that: include: Obtaining the correspondence between the grid elements in the current grid and the grid elements in the reference grid; Determine the mesh elements included in the first patch in the current mesh according to the correspondence between the mesh elements in the current mesh and the mesh elements in the reference mesh, and the mesh elements included in the reference block patch; Wherein, the mesh elements included in the reference patch correspond to the mesh elements included in the first patch; Reconstruct a current textured mesh according to the first patch and the two-dimensional texture map corresponding to the first patch.

2. The method according to claim 1, characterized in that The current grid includes a first grid element; and obtaining a correspondence between the grid elements in the current grid and the grid elements in the reference grid includes: A mesh element in the reference mesh corresponding to the first mesh element is determined according to the voxel cube corresponding to the first mesh element.

3. The method according to claim 2, characterized in that The reference grid and the current grid are both reference grids; and determining, according to the voxel cube corresponding to the first grid element, a grid element in the reference grid and corresponding to the first grid element, comprises: According to the voxel cube corresponding to the first grid element, a first preset range is obtained in the current grid; wherein the first preset range is a range in the current grid that includes the voxel cube; Acquire a second preset range in the reference grid; wherein the position of the second preset range in the reference grid is the same as the position of the first preset range in the current grid; A second grid element in the candidate grid element set is used as the grid element corresponding to the first grid element; wherein the candidate grid element set is a set of grid elements within the second preset range; the second grid element is the grid element in the candidate grid element set that is closest to the target grid element, and the distance between the second grid element and the target grid element is less than or equal to the preset distance; the position of the target grid element in the reference grid is the same as the position of the first grid element in the current grid.

4. The method according to any one of claims 1 to 3, characterized in that The current grid includes a third grid element, and the method further includes: If there is no corresponding relationship between the third grid element and any grid element in the reference grid, taking the third grid element as the second patch in the current grid; The reconstructing the current grid with texture according to the first patch and the two-dimensional texture map corresponding to the first patch includes: The current mesh with texture is reconstructed according to the first patch, the two-dimensional texture map corresponding to the first patch, the second patch, and the two-dimensional texture map corresponding to the second patch.

5. The method according to any one of claims 1 to 3, characterized in that The method further comprises: Constructing an optimization function according to the patches divided by the previous N frames of the current grid; wherein the optimization function includes at least one of the following constraints: consistency of the patches divided by the multi-frame grids, visibility of texture gaps, ease of patch flattening, ease of patch packing, and spatial utilization of texture maps; N≥1, where N is an integer; According to the optimization function, the patches into which the current grid is divided are re-divided to obtain re-divided patches; wherein the patches into which the current grid is divided include the first patch; The reconstructing the current grid with texture according to the first patch and the two-dimensional texture map corresponding to the first patch includes: The current grid with texture is reconstructed according to the re-divided patch and the two-dimensional texture map corresponding to the re-divided patch.

6. The method according to any one of claims 1 to 3, characterized in that The method further comprises: Determining the texture coordinates of the fifth mesh element according to the texture coordinates of the fourth mesh element; wherein the fourth mesh element is a mesh element in the reference patch, the fifth mesh element is a mesh element in the first patch, and the fourth mesh element corresponds to the fifth mesh element; The reconstructing the current grid with texture according to the first patch and the two-dimensional texture map corresponding to the first patch includes: The current mesh with texture is reconstructed according to the first patch, the two-dimensional texture map corresponding to the first patch, and the texture coordinates of the fifth mesh element.

7. The method according to any one of claims 1 to 3, characterized in that The current grid and the reference grid are different grids to be decoded; Alternatively, the current grid and the reference grid are reference grids of different grids to be decoded.

8. The method according to claim 2 or 3, characterized in that: The decoding method is an encoding method, and the method further comprises: A code stream is generated, where the code stream includes first indication information, where the first indication information is used to indicate a voxel cube corresponding to the first grid element.

9. The method according to any one of claims 1 to 3, characterized in that The trellis decoding method is a trellis encoding method; the method further comprises: Generate a code stream, the code stream comprising second indication information; when the current grid is the grid to be encoded, the second indication information is used to indicate the geometric information of the grid to be encoded; or, when the current grid is the reference grid of the grid to be encoded, the second indication information is used to indicate the geometric information and deformation field parameters of the reference grid of the grid to be encoded.

10. The method according to claim 2 or 3, characterized in that: The trellis decoding method is a trellis decoding method, and the method further comprises: Parsing the bitstream to obtain first indication information; the first indication information is used to indicate the voxel cube corresponding to the first grid element; The step of determining a grid element in the reference grid and corresponding to the first grid element according to the voxel cube corresponding to the first grid element comprises: A mesh element in the reference mesh and corresponding to the first mesh element is determined according to the voxel cube indicated by the first indication information.

11. The method according to any one of claims 1 to 3, characterized in that The trellis decoding method is a trellis decoding method, and the method further comprises: Parsing the bitstream to obtain second indication information; wherein, when the current grid is a grid to be decoded, the second indication information is used to indicate geometric information of the grid to be decoded; or when the current grid is a reference grid of the grid to be decoded, the second indication information is used to indicate geometric information and deformation field parameters of the reference grid of the grid to be decoded; The reconstructing the current grid with texture according to the first patch and the two-dimensional texture map corresponding to the first patch includes: Reconstruct the current grid with texture according to the information indicated by the second indication information, the first patch and the two-dimensional texture map corresponding to the first patch.

12. A trellis decoder, characterized in that: include: A first determination module: used to obtain the correspondence between the grid elements in the current grid and the grid elements in the reference grid; The first determination module is further configured to determine the mesh elements included in the first patch in the current grid according to the correspondence between the mesh elements in the current grid and the mesh elements in the reference grid, and the mesh elements included in the reference block patch; Wherein, the mesh elements included in the reference patch correspond to the mesh elements included in the first patch; Reconstruction module: used to reconstruct the current grid with texture according to the first patch and the two-dimensional texture map corresponding to the first patch.

13. The trellis decoder according to claim 12, characterized in that The current grid includes a first grid element; When executing the acquisition of the correspondence between the grid elements in the current grid and the grid elements in the reference grid, the first determination module is specifically used to: determine the grid element in the reference grid corresponding to the first grid element based on the voxel cube corresponding to the first grid element.

14. The trellis decoder according to claim 13, characterized in that The reference grid and the current grid are both reference grids; when the first determining module performs the step of determining the grid element in the reference grid and corresponding to the first grid element according to the voxel cube corresponding to the first grid element, it is specifically used to: According to the voxel cube corresponding to the first grid element, a first preset range is obtained in the current grid; wherein the first preset range is a range in the current grid that includes the voxel cube; Acquire a second preset range in the reference grid; wherein the position of the second preset range in the reference grid is the same as the position of the first preset range in the current grid; A second grid element in the candidate grid element set is used as the grid element corresponding to the first grid element; wherein the candidate grid element set is a set of grid elements within the second preset range; the second grid element is the grid element in the candidate grid element set that is closest to the target grid element, and the distance between the second grid element and the target grid element is less than or equal to the preset distance; the position of the target grid element in the reference grid is the same as the position of the first grid element in the current grid.

15. The trellis decoder according to any one of claims 12 to 14, characterized in that: The current grid includes a third grid element, and the first determining module is further used to: if there is no corresponding relationship between the third grid element and any grid element in the reference grid, use the third grid element as a second patch in the current grid; The reconstruction module is specifically used to reconstruct the current grid with texture according to the first patch, the two-dimensional texture map corresponding to the first patch, the second patch, and the two-dimensional texture map corresponding to the second patch.

16. The trellis decoder according to any one of claims 12 to 14, characterized in that: The first determination module is further used to: construct an optimization function according to the patches divided by the first N frames of the current grid; wherein the optimization function includes at least one of the following constraints: consistency of the patches divided by the multi-frame grids, visibility of texture gaps, ease of patch flattening, ease of patch packing, and spatial utilization of the texture map; N≥1, where N is an integer; according to the optimization function, re-divide the patches divided by the current grid to obtain re-divided patches; wherein the patches divided by the current grid include the first patch; The reconstruction module is specifically used to reconstruct the current grid with texture according to the re-divided patch and the two-dimensional texture map corresponding to the re-divided patch.

17. The trellis decoder according to any one of claims 12 to 14, characterized in that: The trellis decoder further comprises: A second determining module: used to determine the texture coordinates of the fifth mesh element according to the texture coordinates of the fourth mesh element; wherein the fourth mesh element is a mesh element in the reference patch, the fifth mesh element is a mesh element in the first patch, and the fourth mesh element corresponds to the fifth mesh element; The reconstruction module is specifically used to reconstruct the current grid with texture according to the first patch, the two-dimensional texture map corresponding to the first patch, and the texture coordinates of the fifth grid element.

18. The trellis decoder according to any one of claims 12 to 14, characterized in that: The current grid and the reference grid are different grids to be decoded; Alternatively, the current grid and the reference grid are reference grids of different grids to be decoded.

19. The trellis decoder according to claim 13 or 14, characterized in that: The trellis decoder is a trellis encoder, and the trellis encoder further comprises: Coding module: used for generating a code stream, wherein the code stream includes first indication information, and the first indication information is used for indicating a voxel cube corresponding to the first grid element.

20. The trellis decoder according to any one of claims 12 to 14, characterized in that: The trellis decoder is a trellis encoder, and the trellis encoder further comprises: Coding module: used to generate a code stream, the code stream includes second indication information; when the current grid is the grid to be encoded, the second indication information is used to indicate the geometric information of the grid to be encoded; or, when the current grid is the reference grid of the grid to be encoded, the second indication information is used to indicate the geometric information and deformation field parameters of the reference grid of the grid to be encoded.

21. The trellis decoder according to claim 13 or 14, characterized in that: The trellis decoder is a trellis decoder, and the trellis decoder further comprises: Decoding module: used for parsing the code stream to obtain first indication information; the first indication information is used for indicating the voxel cube corresponding to the first grid element; The first determination module is specifically configured to determine, according to the voxel cube indicated by the first indication information, a grid element in the reference grid corresponding to the first grid element.

22. The trellis decoder according to any one of claims 12 to 14, characterized in that: The trellis decoder is a trellis decoder, and the trellis decoder further comprises: Decoding module: used for parsing the code stream to obtain second indication information; wherein, when the current grid is the grid to be decoded, the second indication information is used to indicate the geometric information of the grid to be decoded; or when the current grid is the reference grid of the grid to be decoded, the second indication information is used to indicate the geometric information and deformation field parameters of the reference grid of the grid to be decoded; The reconstruction module is specifically used to reconstruct the current grid with texture according to the information indicated by the second indication information, the first patch and the two-dimensional texture map corresponding to the first patch.

23. A trellis decoding device, characterized in that: The invention comprises a memory and a processor; the memory is used to store program code; the processor is used to call the program code to execute the method according to any one of claims 1 to 11.

24. A computer-readable storage medium, characterized in that: The method comprises a program code, which, when executed on a computer, causes the computer to execute the method according to any one of claims 1 to 11.

Images