Encoding method and device, decoding method and device, encoding end and decoding end

Abstract

The invention discloses a coding method and device, a decoding method and device, a coding end and a decoding end, and belongs to the technical field of video processing.The coding method comprises the steps that the coding end reconstructs a basic grid to obtain a reconstructed basic grid; the coding end obtains texture information corresponding to the reconstructed basic grid according to the reconstructed basic grid; the encoding end encodes the texture information; and the encoding end sends an encoded code stream to a decoding end, wherein the code stream at least comprises a basic grid code stream and an auxiliary information code stream.

Description

TECHNICAL FIELD

[0001] The present application belongs to the technical field of video processing, and particularly relates to an encoding method and device, a decoding method and device, an encoding end and a decoding end. BACKGROUND

[0002] The existing encoding scheme is to obtain the texture coordinates of the original base mesh by parameterizing the original base mesh. This way will make the texture coordinates of the decoded base mesh inconsistent with the texture coordinates transmitted by the encoding end at the decoding end. SUMMARY

[0003] The embodiments of the present application provide an encoding method and device, a decoding method and device, an encoding end and a decoding end to ensure that the decoding end can obtain the texture coordinates corresponding to the decoded base mesh.

[0004] In a first aspect, an encoding method is provided, and the method comprises:

[0005] The encoding end reconstructs a base mesh to obtain a reconstructed base mesh;

[0006] The encoding end obtains texture information corresponding to the reconstructed base mesh according to the reconstructed base mesh;

[0007] The encoding end encodes the texture information;

[0008] The encoding end sends the encoded code stream to a decoding end, and the code stream at least includes a base mesh code stream and auxiliary information code stream.

[0009] In a second aspect, an encoding device is provided, and the device comprises:

[0010] A first obtaining module is configured to reconstruct a base mesh to obtain a reconstructed base mesh;

[0011] A second obtaining module is configured to obtain texture information corresponding to the reconstructed base mesh according to the reconstructed base mesh;

[0012] An encoding module is configured to encode the texture information;

[0013] A sending module is configured to send the encoded code stream to a decoding end, and the code stream at least includes a base mesh code stream and auxiliary information code stream.

[0014] In a third aspect, a decoding method is provided, and the method comprises:

[0015] A decoding end decodes a received code stream to obtain a base mesh with texture coordinates, and the code stream at least includes a base mesh code stream and auxiliary information code stream;

[0016] The decoding end removes repeated points from the base mesh with the texture coordinates to obtain a decoded mesh.

[0017] In a fourth aspect, a decoding apparatus is provided, comprising:

[0018] a decoding module configured to decode a received bitstream to obtain a base mesh with texture coordinates, the bitstream comprising at least a base mesh bitstream and an auxiliary information bitstream;

[0019] a processing module configured to remove repeated points from the base mesh with the texture coordinates to obtain a decoded mesh.

[0020] In a fifth aspect, an encoding end is provided, comprising a processor and a memory, the memory storing programs or instructions executable on the processor, the programs or instructions being executed by the processor to implement the steps of the method according to the first aspect.

[0021] In a sixth aspect, an encoding end is provided, comprising a processor and a communication interface, wherein the processor is configured to reconstruct a base mesh to obtain a reconstructed base mesh, obtain texture information corresponding to the reconstructed base mesh according to the reconstructed base mesh, encode the texture information, and send an encoded bitstream to a decoding end, the bitstream comprising at least a base mesh bitstream and an auxiliary information bitstream.

[0022] In a seventh aspect, a decoding end is provided, comprising a processor and a memory, the memory storing programs or instructions executable on the processor, the programs or instructions being executed by the processor to implement the steps of the method according to the third aspect.

[0023] In an eighth aspect, a decoding end is provided, comprising a processor and a communication interface, wherein the processor is configured to decode a received bitstream to obtain a base mesh with texture coordinates, the bitstream comprising at least a base mesh bitstream and an auxiliary information bitstream, and remove repeated points from the base mesh with the texture coordinates to obtain a decoded mesh.

[0024] In a ninth aspect, a codec system is provided, comprising an encoding end and a decoding end, the encoding end being configured to implement the steps of the method according to the first aspect, and the decoding end being configured to implement the steps of the method according to the third aspect.

[0025] In a tenth aspect, a readable storage medium is provided, the readable storage medium storing programs or instructions executable on a processor, the programs or instructions being executed by the processor to implement the steps of the method according to the first aspect or the third aspect.

[0026] In an eleventh aspect, a chip is provided, the chip comprising a processor and a communication interface, the communication interface and the processor being coupled, the processor being configured to run a program or an instruction to implement steps of the method according to the first aspect or the third aspect.

[0027] In a twelfth aspect, a computer program / program product is provided, the computer program / program product being stored in a storage medium, the computer program / program product being executed by at least one processor to implement steps of the method according to the first aspect or the third aspect.

[0028] In the embodiments of the present application, the basic mesh is reconstructed to obtain a reconstructed basic mesh; then, texture information corresponding to the reconstructed basic mesh is obtained according to the reconstructed basic mesh; then, the texture information is encoded; finally, the encoded bitstream is sent to a decoding end; so that the reconstructed basic mesh is used to obtain the texture information in a parameterized manner when the texture information is encoded at the encoding end, which is consistent with the texture information decoding at the decoding end using the decoded basic mesh, so as to ensure that the decoding end can obtain the texture coordinates corresponding to the decoded basic mesh. BRIEF DESCRIPTION OF DRAWINGS

[0029] Figure 1 FIG. 1 is a schematic diagram of a coding system provided by the embodiments of the present application;

[0030] Figure 2 FIG. 2 is a structural schematic diagram of an encoder provided by the embodiments of the present application;

[0031] Figure 3 FIG. 3 is a structural schematic diagram of a decoder provided by the embodiments of the present application;

[0032] Figure 4 FIG. 4 is a flow schematic diagram of an encoding method according to the embodiments of the present application;

[0033] Figure 5 FIG. 5 is a flow schematic diagram of obtaining a reconstructed basic mesh;

[0034] Figure 6 FIG. 6 is a schematic diagram of mesh simplification;

[0035] Figure 7 FIG. 7 is a schematic diagram of basic mesh generation;

[0036] Figure 8 FIG. 8 is a schematic diagram of Tetris arrangement;

[0037] Figure 9 FIG. 9 is a schematic diagram of an Ortho Atlas parameterization basic process;

[0038] Figure 10 FIG. 10 is an Ortho Atlas partition generation process;

[0039] Figure 11 is a schematic diagram of a parameterized mesh adjustment process;

[0040] Figure 12 is a schematic diagram of an overall encoding process;

[0041] Figure 13 is a schematic diagram of a subdivision deformation;

[0042] Figure 14 is a schematic diagram of a displacement vector computation process;

[0043] Figure 15 is a schematic diagram of a decoding method according to an embodiment of the application;

[0044] Figure 16 is a schematic diagram of an overall decoding process;

[0045] Figure 17 is a schematic diagram of a module of an encoding apparatus according to an embodiment of the application;

[0046] Figure 18 is a schematic diagram of a structure of an encoding end according to an embodiment of the application;

[0047] Figure 19 is a schematic diagram of a module of a decoding apparatus according to an embodiment of the application;

[0048] Figure 20 is a schematic diagram of a structure of an electronic device according to an embodiment of the application. DETAILED DESCRIPTION

[0049] The technical solutions in the embodiments of the present application will be clearly described below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only some, but not all of the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by a person of ordinary skill in the art belong to the scope of protection of the present application.

[0050] The terms "first", "second", and the like in the present application are used to distinguish similar objects, and are not used to describe a specific order or sequence. It should be understood that the terms used in this way can be interchanged under appropriate circumstances, so that the embodiments of the present application can be implemented in an order other than that illustrated or described herein, and the objects distinguished by "first", "second" are usually a category, and are not limited to the number of objects, for example, the first object can be one or more. In addition, "or" in the present application means at least one of the connected objects. For example, "A or B" covers three schemes, namely, scheme one: including A and not including B; scheme two: including B and not including A; scheme three: including A and including B. The character " / " generally represents that the objects before and after are in an "or" relationship.

[0051] Figure 1Schematic diagram of a codec system provided in an embodiment of the present application. The technical solution of the embodiment of the present application relates to encoding and decoding (CODEC) (including encoding or decoding) of video data. The video data includes original unencoded video, encoded video, decoded (e.g., reconstructed) video, or syntax elements.

[0052] like Figure 1 As shown, the codec system includes a source device 100, which provides encoded video data to be decoded and displayed by a destination device 110. Specifically, the source device 100 provides the video data to the destination device 110 via a communication medium 120. The source device 100 and the destination device 110 may include any one or more of a desktop computer, a notebook (i.e., laptop) computer, a tablet computer, a set-top box, a mobile phone, a wearable device (e.g., a smart watch or a wearable camera), a television, a camera, a display device, an in-vehicle device, a virtual reality (VR) device, an augmented reality (AR) device, a mixed reality (MR) device, a digital media player, a video game console, a video conferencing device, a video streaming device, a broadcast receiver device, a broadcast transmitter device, a spacecraft, an aircraft, a robot, a satellite, and the like.

[0053] exist Figure 1 In the example of FIG, the source device 100 includes a data source 101, a memory 102, an encoder 200, and an output interface 104. The destination device 110 includes an input interface 111, a decoder 300, a memory 113, and a display device 114. The source device 100 represents an example of a video encoding device, and the destination device 110 represents an example of a video decoding device. In other examples, the source device 100 and the destination device 110 may not include Figure 1 Some components in, or can also include Figure 1 For example, source device 100 may receive video data from an external data source (such as an external camera). Similarly, destination device 110 may interface with an external display device rather than including an integrated display device. For another example, memory 102 and memory 113 may be external memories.

[0054] Although Figure 1 Source device 100 and destination device 110 are shown as separate devices, but in some examples, they may be integrated into one device. In such embodiments, the functions corresponding to source device 100 and the functions corresponding to destination device 110 may be implemented using the same hardware or software, or using separate hardware or software, or any combination thereof.

[0055] In some examples, the source device 100 and the destination device 110 can engage in one-way video transmission or two-way video transmission. If two-way video transmission, the source device 100 and the destination device 110 can operate in a substantially symmetrical manner, i.e., each of the source device 100 and the destination device 110 includes an encoder and a decoder.

[0056] The data source 101 represents a source of video data (i.e., raw, uncoded video data) and provides the encoder 200 with successive pictures containing the video data that the encoder 200 encodes. The data source 101 of the source device 100 can include a video capture device, such as a video camera, a video archive containing previously captured raw video, or a video feed interface to receive video from a video content provider. As another alternative, the data source 101 can generate computer graphics-based data as the source video, or a combination of live video, archived video, and computer-generated video. In these cases, the encoder 200 encodes the captured, pre-captured, or computer-generated video data. The encoder 200 can rearrange the pictures from the received order (sometimes referred to as "display order") into the encoding order. The encoder 200 can generate a bitstream including encoded video data. The source device 100 can then output the encoded video data via the output interface 104 onto the communication medium 120 for reception or retrieval by, e.g., the input interface 111 of the destination device 110.

[0057] The memory 102 of the source device 100 and the memory 113 of the destination device 110 represent general purpose memories. In some examples, the memory 102 can store raw video data from the data source 101, and the memory 113 can store decoded video data from the decoder 300. Additionally or alternatively, the memories 102, 113 can store software instructions that are executable by, e.g., the encoder 200 and the decoder 300, respectively. Although the memory 102 and the memory 113 are shown separately from the encoder 200 and the decoder 300 in this example, it should be understood that the encoder 200 and the decoder 300 can also include internal memories for functionally similar or equivalent purposes. If the encoder 200 and the decoder 300 are deployed on the same hardware device, the memory 102 and the memory 113 can be one and the same memory. Moreover, the memories 102, 113 can store encoded video data that is output from the encoder 200 and input to the decoder 300, for example. In some examples, portions of the memories 102, 113 can be allocated as one or more video buffers, e.g., to store raw, decoded, or encoded video data.

[0058] In some examples, source device 100 can output encoded data from output interface 104 to storage 113. Similarly, destination device 110 can access encoded data from storage 113 via input interface 111. Storage 113 or storage 102 can include any of a variety of distributed or locally accessed data storage media such as a hard drive, Blu-ray discs, Digital Versatile Discs (DVDs), Compact Disc Read-Only Memory (CD-ROMs), flash drive, volatile or non-volatile memory, or any other suitable digital storage media for storing encoded video data.

[0059] Output interface 104 can include any type of medium or device capable of sending encoded video data from source device 100 to destination device 110. For example, output interface 104 can include a transmitter or a transceiver, e.g., an antenna, configured to transmit encoded video data from source device 100 directly to destination device 110 in real-time. The encoded video data can be modulated according to a communication standard of a wireless communication protocol and transmitted to destination device 110.

[0060] Communication medium 120 can include transient media, such as wireless broadcasts or wired networks transmitted over physical lines. For example, communication medium 120 can include radio frequency (RF) spectrum or one or more physical transmission lines (e.g., cable). Communication medium 120 can form a portion of a packet-based network, such as a local area network, a wide area network, or a global network, such as the Internet. Communication medium 120 can also be in a form of storage media, such as a hard drive, flash drive, compact disc, digital video disc, Blu-ray disc, volatile or non-volatile memory, or any other suitable digital storage media for storing encoded video data.

[0061] In some embodiments, communication medium 120 can include routers, switches, base stations, or any other equipment that can be used to facilitate communication from source device 100 to destination device 110. For example, a server (not shown) can receive the encoded video from source device 100 and provide the encoded video data to destination device 110, e.g., via network transmission. The server can include a web server (e.g., for a website), a server configured to provide file transfer protocol services (such as File Transfer Protocol (FTP) or File Delivery Over Unidirectional Transport (FLUTE) protocol), a content delivery network (CDN) device, a Hypertext Transfer Protocol (HTTP) server, a Multimedia Broadcast Multicast Services (MBMS) or evolved MBMS (eMBMS) server, or a Network-attached storage (NAS) device, etc. The server can implement one or more HTTP streaming protocols, such as the MPEG Media Transport (MMT) protocol, Dynamic Adaptive Streaming over HTTP (DASH) protocol, HTTP Live Streaming (HLS) protocol, or Real Time Streaming Protocol (RTSP), etc.

[0062] Destination device 110 can access the encoded video data from a server, e.g., through a wireless channel (e.g., Wi-Fi connection) or wired connection (e.g., Digital subscriber line (DSL), cable modem, etc.) for accessing encoded video data stored on the server.

[0063] The output interface 104 and the input interface 111 may represent wireless transmitters / receivers, modems, wired networking components (e.g., Ethernet cards), wireless communication components operating according to the IEEE 802.11 standard or the IEEE 802.15 standard (e.g., ZigBee™), the Bluetooth standard, or other physical components. In examples where the output interface 104 and the input interface 111 include wireless components, the output interface 104 and the input interface 111 may be configured to communicate data, such as encoded video data, according to WIFI, Ethernet, a cellular network (such as 4G, LTE (Long Term Evolution), LTE-Advanced, 5G, 6G, etc.).

[0064] The technology provided in the embodiments of the present application can be applied to support video encoding and decoding in one or more multimedia applications such as: video conferencing, over-the-air television broadcasting, cable television transmission, satellite television transmission, Internet streaming video transmission, digital video encoded on a data storage medium, decoding of digital video stored on a data storage medium, or other applications.

[0065] The input interface 111 of the destination device 110 receives an encoded video bitstream from the communication medium 120. The encoded video bitstream may include syntax elements and encoded data units (e.g., sequences, groups of pictures, pictures, slices, blocks, etc.), wherein the syntax elements are used to decode the encoded data units to obtain decoded video data. The display device 114 displays the decoded video data to the user. The display device 114 may include a cathode ray tube (CRT), a liquid crystal display (LCD), a plasma display, an organic light-emitting diode (OLED) display, or other types of display devices.

[0066] The encoder 200 and the decoder 300 may be implemented as one or more of a variety of processing circuits, which may include a microprocessor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA), discrete logic, hardware, or any combination thereof. When the technology is implemented in whole or in part in software, the device may store instructions for the software in an appropriate non-transitory computer-readable storage medium and use one or more processors to execute the instructions in hardware to perform the technology provided in the embodiments of the present application.

[0067] The encoder 200 and the decoder 300 can be processed based on the following video coding and decoding standards: H.263, H.264, H.265 (also known as High Efficiency Video Coding (HEVC)), H.266 (also known as Versatile Video Coding (VVC), the second generation Moving Picture Experts Group 2 (MPEG-2), MPEG-4, VP8, VP9, ​​the first generation of Open Media Video 1 (Alliance for Open Media Video 1, AV1), the first generation of audio and video coding standard (Audio Videocoding Standard 1, AVS1), AVS2, AVS3 or the next generation video standard protocol, which is not specifically limited in the embodiments of the present application.

[0068] Generally, the encoder 200 and decoder 300 can perform block-based encoding and decoding of pictures. The term "block" generally refers to a structure that includes data to be processed (e.g., encoded, decoded, or otherwise used in the encoding or decoding process). For example, a block can include a two-dimensional matrix of samples of luma or chroma data. For example, the encoder 200 and decoder 300 can encode and decode video data represented in YUV format.

[0069] See also Figure 2 , which is a schematic diagram of the structure of the encoder 200 provided in an embodiment of the present application, and the encoder 200 can be Figure 1 In the encoder 200. Figure 2 In the example, the encoder 200 includes a memory 201, a coding parameter determination unit 210, a residual generation unit 202, a transform processing unit 203, a quantization unit 204, an inverse quantization unit 205, an inverse transform processing unit 206, a reconstruction unit 207, a filter unit 208, a decoded picture buffer (DPB) 209 and an entropy coding unit 220.

[0070] The memory 201 can store the video data to be encoded. For example, the encoder 200 can store the video data to be encoded. Figure 1 The data source 101 shown receives the video data and stores it. In some examples, the memory 201 can be on the same chip as the other components of the encoder 200 (e.g., Figure 2 ), or may be independent of the chips where these components are located.

[0071] The coding parameter determination unit 210 includes a mode selection unit 211, an inter prediction unit 212, and an intra prediction unit 213. The inter prediction unit 212 is configured to obtain a first prediction block of the current block in an inter prediction mode, the intra prediction unit 213 is configured to obtain a second prediction block of the current block in an intra prediction mode, and the mode selection unit 211 is configured to obtain a target prediction block according to the first prediction block and the second prediction block, and determine a final prediction mode. In addition, the coding parameter determination unit 210 can further include other functional units, such as a functional unit for determining a partitioning manner of a coding unit (CU), a functional unit for determining a transform type of residual data of the CU or a quantization parameter of the residual data of the CU, and the like.

[0072] For ease of description and understanding, the embodiments of the present application refer to a CU to be processed in a current picture as a current CU, and an image block to be processed in the current CU as a current block or a to-be-processed image block, for example, in encoding, referring to a block being currently encoded; in decoding, referring to a block being currently decoded.

[0073] The inter prediction unit 212 can include a motion estimation unit and a motion compensation unit. For inter prediction of the current block, the motion estimation unit can perform a motion search to identify one or more matching reference blocks in one or more reference pictures (e.g., one or more previously coded pictures stored in the DPB 209).

[0074] The motion estimation unit can form one or more motion vectors (MVs) of a position of a reference block in a reference picture relative to a position of the current block in the current picture. The motion compensation unit can obtain a prediction value of a precision indicated by the motion vector by interpolation.

[0075] The coding parameter determination unit 210 can provide the target prediction block to the residual generation unit 202. The residual generation unit 202 receives original uncoded video data of the current block from the memory 201, and calculates a residual between the current block and the target prediction block to obtain a residual block. In some examples, the functions of the residual generation unit 202 can be implemented using one or more subtractor circuits that perform binary subtraction.

[0076] As an example, the coding parameter determination unit 210 can provide syntax elements representing the coding parameters to the entropy encoding unit 220 for encoding. The coding parameters include one or more of a partitioning manner of the CU, a final prediction mode, a transform type of residual data of the CU, or a quantization parameter of the residual data of the CU, and the like.

[0077] The transform processing unit 203 applies transform to the residual block output by the residual generation unit 202 to obtain a transform coefficient block, which can include discrete cosine transform (DCT), integer transform, directional transform, or Karhunen-Loeve transform, among others. In some examples, the encoder 200 can not include the transform processing unit 203.

[0078] The quantization unit 204 can quantize the transform coefficients in the transform coefficient block according to a quantization parameter (QP) value associated with the current block, to produce a quantized transform coefficient block.

[0079] The inverse quantization unit 205 and the inverse transform processing unit 206 can apply inverse quantization and inverse transform, respectively, to the transform coefficient block to obtain a reconstructed residual block. The reconstruction unit 207 can generate a reconstructed block corresponding to the current block based on the reconstructed residual block and the target prediction block generated by the encoding parameter determination unit 210.

[0080] The filter unit 208 can perform one or more filter operations on the reconstructed block. For example, the filter unit 208 can be a deblocking filter (DBF), an adaptive loop filter (ALF), a sample adaptive offset (SAO) filter, among others. In some examples, the encoder 200 can not include the filter unit 208.

[0081] The encoder 200 stores reconstructed pictures resulting from the reconstructed blocks in the DPB 209. For example, in examples where the operation of the filter unit 208 is not needed, the reconstruction unit 207 can store the reconstructed blocks to the DPB 209. In examples where the operation of the filter unit 208 is needed, the filter unit 208 can store the filtered reconstructed blocks to the DPB 209. The inter prediction unit 212 obtains reconstructed pictures from the DPB 209 to perform inter prediction for blocks of subsequent pictures to be encoded. In some examples, the DPB 209 can be replaced by other types of memory.

[0082] The entropy encoding unit 220 can entropy encode syntax elements of other components in the encoder 200 to output encoded video data. For example, the entropy encoding unit 220 can entropy encode quantized transform coefficient blocks from the quantization unit 204. As another example, the entropy encoding unit 220 can entropy encode syntax elements from the encoding parameter determination unit 210 (e.g., motion information used for inter prediction or intra mode information used for intra prediction).

[0083] It is understandable that Figure 2 The composition of the encoder 200 is merely an illustration and does not constitute a limitation to the embodiments of the present application.

[0084] Figure 3 is a schematic diagram of the structure of the decoder 300 provided in an embodiment of the present application. The decoder 300 may be Figure 1 The decoder 300. Figure 3 In the example, the decoder 300 includes a coded picture buffer (CPB) 301, an entropy decoding unit 302, a prediction processing unit 310, an inverse quantization unit 303, an inverse transform processing unit 304, a reconstruction unit 305, a filter unit 306 and a DPB 307.

[0085] The entropy decoding unit 302 can receive the encoded video data from the CPB 301 and perform entropy decoding on the video data to obtain syntax elements, where the syntax elements indicate encoding parameters, and the encoding parameters include one or more of the CU partitioning method, the final prediction mode, the transform type of the CU's residual data, or the quantization parameter of the CU's residual data.

[0086] When the syntax element includes the final prediction mode, the prediction processing unit 310 obtains the final prediction mode. If the final prediction mode is an inter-frame prediction mode, the prediction block of the current CU can be obtained by the inter-frame prediction unit 311 of the prediction processing unit 310; if the final prediction mode is an intra-frame prediction mode, the prediction block of the current CU can be obtained by the intra-frame prediction unit 312 of the prediction processing unit 310. In some examples, the prediction processing unit 310 may also include a unit for performing prediction functions according to other prediction modes.

[0087] CPB 301 can be obtained from Figure 1 The communication medium 120 shown obtains the encoded video data and stores it. The DPB 307 is used to store the decoded pictures. Optionally, the CPB 301 and the DPB 307 can also be replaced with other types of memory, which is not specifically limited in this application. In some examples, the CPB 301 can be on the same chip as other components of the decoder 300 (as shown in the figure), or it can be independent of the chip where these components are located.

[0088] The decoder 300 can perform the reconstruction operation separately for each block. The entropy decoding unit 302 can entropy-decode the syntax elements of the quantized transform coefficients and the transform information (e.g., QP or transform mode indication) to obtain the quantized transform coefficients. The quantized transform coefficients are inverse quantized by the inverse quantization unit 303 to obtain a transform coefficient block including transform coefficients. The transform coefficient block is inverse transformed by the inverse transform processing unit 304 to generate a residual block corresponding to the current block, which is the inverse operation of the above-mentioned transform.

[0089] The reconstruction unit 305 can reconstruct the current block from the prediction block and the residual block. For example, the reconstruction unit 305 can add the samples of the residual block to the corresponding samples of the prediction block to reconstruct the current block.

[0090] The filter unit 306 can perform one or more filter operations on the reconstructed block. For example, the types of the filter unit 306 can refer to the types of the filter unit 208, which are not repeated here. In some examples, the operations of the filter unit 306 can be skipped.

[0091] The decoder 300 can store the reconstructed pictures resulting from the reconstructed blocks in the DPB 307. For example, in examples where the operations of the filter unit 306 are not performed, the reconstruction unit 305 can store the reconstructed blocks to the DPB 307. In examples where the operations of the filter unit 306 are performed, the filter unit 306 can store the filtered reconstructed blocks to the DPB 307. The decoder 300 can output decoded pictures (e.g., decoded video) from the DPB 307 for subsequent presentation to a display device, such as the display device 114 of FIG. 1. Figure 1

[0092] The encoding method provided by the embodiments of the present application can be executed by the encoding end, for example, the encoder 200 shown in FIG. 2. The decoding method provided by the embodiments of the present application can be executed by the decoding end, for example, the decoder 300 shown in FIG. 3. The encoding end and the decoding end can be realized by software, hardware or combination thereof, when they are realized by hardware, the encoding end can be referred to as an encoding end device or a video encoding device, and the decoding end can be referred to as a decoding end device or a video decoding device. Figure 1 Figure 2 Figure 1 Figure 3 The following first describes the technology related to the embodiments of the present application as follows.

[0093] The following first describes the technology related to the embodiments of the present application as follows.

[0094] ​​​​In recent years, with the rapid development of multimedia technology, the related research results are rapidly industrialized, and become an important part of people's life. Three-dimensional model is a new generation of digital media after audio, image, video. Three-dimensional mesh is a common way of three-dimensional model representation. Compared with traditional image, video and other multimedia, three-dimensional mesh model has stronger interactivity and realism, which makes it more and more widely used in business, manufacturing, construction, education, medicine, entertainment, art, military and other fields.

[0095] Although there are many kinds of three-dimensional mesh representation methods, triangular mesh is still the most common representation method. Three-dimensional mesh can be regarded as consisting of three basic elements: vertices, edges and faces. Vertices are the most basic elements in the mesh, which define the position in a three-dimensional space. Edges are the line segments connecting two vertices in the mesh. Faces can be regarded as polygons formed by the closed path of edges. For triangular mesh, each face is a triangle.

[0096] The information contained in the mesh is usually divided into three categories: geometric information, connection information and attribute information. Geometric information is the position of each vertex of the mesh in three-dimensional space. Connection information describes the association between elements in the mesh, i.e. the connection relationship between vertices. Attribute information is optional, which can associate attributes to the corresponding mesh elements (such as vertex color, normal vector, etc. can be associated with mesh vertices). Mesh parameterization can also be used to map the mesh from three-dimensional space to two-dimensional plane area, and this mapping relationship is usually described by a set of parameter coordinates, called UV coordinates or texture coordinates, which are associated with mesh vertices. This two-dimensional mapping can be used to represent high-resolution attribute information, such as texture, normal vector, etc.

[0097] In almost all application fields using three-dimensional mesh (such as computational simulation, entertainment, medical imaging, digital cultural relics, computer design, e-commerce, etc.), with people's increasing demand for three-dimensional mesh model in visual effect, the model is becoming more and more complex, and the precision of the model is also increasing, so the amount of data required to represent three-dimensional mesh is also increasing. The above problems lead to the increasing complexity of three-dimensional mesh processing, visualization, transmission and storage. Three-dimensional mesh compression can be regarded as a way to solve the above problems, which reduces the size of the model data, and is beneficial to the processing, storage and transmission of three-dimensional mesh, so it is necessary to propose an efficient and general three-dimensional mesh compression algorithm.

[0098] Recently, the international standardization organization MPEG in the field of audio and video coding compression begins to formulate a compression standard VDMC (Video-based dynamic mesh coding) for three-dimensional mesh, which is based on the existing V3C (Visual Volumetric Video-based Coding) standard specification, and the V3C standard provides a general method for compressing three-dimensional models, which can be in the form of point cloud, mesh or panoramic video. The compression method of three-dimensional mesh model is compatible with the standard, which helps to promote the method and its applicability. Therefore, it is of great significance to optimize the three-dimensional mesh coding method in VDMC and combine the optimization method with the V3C standard. One possible optimization method is the optimization of intra-frame base mesh coding. In the existing framework, the information of the base mesh will be divided into connection relationship, geometric information and attribute information for processing. For the geometric information coding of the base mesh of three-dimensional mesh coding, a geometric information coding method depending only on part of the connection relationship is proposed, which can support the parallel scheme of connection relationship and geometric information coding, and only one traversal can complete the coding of connection relationship and geometric information, which can reduce the time complexity.

[0099] V3C standard

[0100] The V3C standard provides a method for encoding and decoding various three-dimensional media through video or image coding technology. Specifically, it converts three-dimensional media content from three-dimensional representation to multiple two-dimensional representations (called V3C components) through projection and other methods before encoding, and then encodes the two-dimensional representations using existing video or image coding technology. The V3C components mainly include occupancy components, geometry components and attribute components. The occupancy component can represent which areas in the two-dimensional representation are associated with the data of the three-dimensional representation; the geometry component represents information related to the position of the three-dimensional data in space, and the attribute component can provide attribute information corresponding to the vertices, such as material, texture, etc. In addition, the components also contain information on how to reconstruct the three-dimensional model from these components, which is called atlas information.

[0101] Atlas information is used to associate all components, and additional information for reconstructing from two dimensions back to three dimensions is also included in the atlas component. The atlas is composed of multiple basic units, called patches. Each patch represents a block area in the available two-dimensional component and contains the information needed to project the area back to three-dimensional space.

[0102] VDMC

[0103] VDMC is a standard for compressing three-dimensional mesh established by MPEG, the main idea of which is to compress three-dimensional mesh by using the existing V3C standard. Since three-dimensional mesh needs to be encoded due to the existence of connection information, the specific encoding process is slightly different from V3C, and the syntax semantics and decoding operation of the decoding end of the V3C standard need to be extended to support the decoding and reconstruction of three-dimensional mesh.

[0104] At the encoding end, for the input mesh, first, simplification is performed by the simplification module, then new texture coordinates are generated by mesh parameterization, then the parameterized mesh is subdivided and deformed, that is, new vertices are inserted on the mesh according to a specific subdivision method, and the distance between the subdivided mesh vertices and the nearest neighbor points of the input mesh is calculated, which is called displacement information. Then, the vertex positions of the mesh before subdivision and deformation are adjusted according to the displacement information, and the adjusted mesh is called the base mesh, which is input into the base mesh encoding module for compression. When encoding the base mesh, it is divided into three types of sub-meshes for independent encoding. After the base mesh is encoded, it is reconstructed, and then the order of the displacement is adjusted according to the vertex order of the reconstructed base mesh. Then, the vertex displacement information after adjusting the order is first wavelet transformed, the transformed coefficients are quantized, then the quantized coefficients are arranged into a two-dimensional image according to a specific scanning order, and the two-dimensional image is encoded using a video encoder. Then, the reconstructed displacement information is applied to the subdivided base mesh to obtain the reconstructed subdivided and deformed mesh, and the mesh, the original input mesh and the corresponding texture map are input into the corresponding texture map conversion module to obtain the texture map corresponding to the reconstructed mesh, which is also encoded using a video encoder. The parameters used in the encoding process, such as the type of video encoder, the type of mesh encoder, the transformation parameters, the quantization parameters, etc. are transmitted to the decoding end through the auxiliary information.

[0105] At the decoding end, for the received code stream, the decoding end first demultiplexes each part of the code stream to obtain the base mesh code stream, the displacement video code stream, the texture map video code stream and the auxiliary information code stream. For the base mesh code stream, the mesh decoder indicated by the auxiliary information is used to decode the base mesh. The displacement video code stream and the texture map video code stream are decoded by the video decoder. For the displacement part, the displacement needs to be taken out from the image and subjected to the steps of inverse quantization and inverse transformation through the displacement decoding module after video decoding, and then applied to the subdivided base mesh to obtain the deformed mesh reconstructed at the decoding end. The texture map after decoding is the texture map corresponding to the reconstructed deformed mesh. The subsequent application or rendering module processes the reconstructed deformed mesh and the decoded texture map as input.

[0106] VDMC generally encodes base meshes in intra-frame and inter-frame modes. In intra-frame mode, the encoder processes the base mesh as follows: The input base mesh is encoded into three streams: a connection information stream, a geometry information stream, and an attribute information stream, through the connection relationship encoding module, the geometry information encoding module, and the attribute information encoding module. These streams are then mixed and output. The encoding of the geometry information and attribute information both references the order in which the connection relationships are traversed.

[0107] Corresponding to the encoder, the decoder's base mesh decompression process in intra-frame mode involves decompressing the input bitstream into three streams: a connectivity information stream, a geometry information stream, and an attribute information stream. These streams are then fed into the connectivity decoding module, the geometry decoding module, and the attribute decoding module, respectively. These decoders obtain the base mesh's connectivity, geometry, and attribute information, ultimately reconstructing the base mesh. Similar to the encoder, the decoding of the geometry and attribute information on the decoder also requires reference to the order in which the connectivity relationships are traversed.

[0108] On the encoding side, the Edgebreaker algorithm traverses the 3D mesh and represents the connections as five modes (C, L, E, R, and S) for efficient encoding. For geometric information, the geometric coordinates of the traversed vertices are predicted based on the order in which the connections are traversed, ultimately encoding the residual between the predicted and true coordinates.

[0109] On the decoder side, the connectivity relationships are reconstructed by traversing the connectivity sequence passed from the encoder. For geometric information, the mesh is traversed based on the fully reconstructed connectivity relationships, and the geometric positions of the vertices are predicted using the same prediction scheme as the encoder. The actual geometric coordinates of the vertices are then added to the residuals of the geometric coordinates passed from the encoder.

[0110] The following, in conjunction with the accompanying drawings, describes in detail the encoding and decoding methods, devices, encoding terminals, and decoding terminals provided by the embodiments of the present application through some embodiments and their application scenarios.

[0111] like Figure 4 As shown, the embodiment of the present application provides an encoding method, including:

[0112] Step 401: The encoder reconstructs the basic grid to obtain a reconstructed basic grid.

[0113] Step 402: the encoder obtains texture information corresponding to the reconstructed basic mesh according to the reconstructed basic mesh;

[0114] Step 403: the encoding end encodes the texture information;

[0115] Step 404, the encoding end sends the encoded code stream to the decoding end, and the code stream at least includes the base grid code stream and the auxiliary information code stream.

[0116] It should be noted that, in the embodiment of the application, the base grid is reconstructed to obtain a reconstructed base grid, then the texture information corresponding to the reconstructed base grid is obtained according to the reconstructed base grid, then the texture information is encoded, and finally the encoded code stream is sent to the decoding end, so that the texture information is obtained by using the reconstructed base grid for parameterization when the texture information is encoded at the encoding end, which is consistent with the texture information decoding using the decoded base grid at the decoding end, so as to ensure that the decoding end can obtain the texture coordinates corresponding to the decoded base grid.

[0117] Optionally, in an implementation, the specific implementation of reconstructing the base grid to obtain the reconstructed base grid includes:

[0118] The encoding end reconstructs the connection relationship and the geometric information of the base grid to obtain the reconstructed base grid.

[0119] It should be noted that, in the embodiment of the application, only the connection relationship and the geometric information are reconstructed when the reconstruction is performed.

[0120] For example, the embodiment of the application provides a way of reconstructing the base grid, as shown in Figure 5 The specific implementation process includes: first, the geometric information (here, the geometric information mainly includes geometric coordinates) of the base grid is quantized according to a preset quantization parameter, then the connection relationship and the geometric information of the quantized base grid are encoded, the connection relationship code stream and the geometric information code stream are mixed after encoding, the connection relationship code stream and the geometric information code stream of the base grid are obtained by demixing, the connection relationship and the geometric information are decoded respectively, and finally the geometric coordinates are dequantized to output the reconstructed base grid.

[0121] It should be noted that, here, only an example of obtaining the reconstructed base grid is provided, and the way of obtaining the reconstructed base grid in the embodiment of the application is not limited to this, and any implementation way of obtaining the reconstructed base grid belongs to the protection scope of the embodiment of the application.

[0122] Optionally, in an implementation, the base grid needs to be obtained before the base grid is reconstructed to obtain the reconstructed base grid, and the embodiment of the application provides two ways of obtaining the base grid, which are specifically described as follows.

[0123] The first way of obtaining is that the encoding end simplifies the original grid to obtain the base grid.

[0124] It should be noted that this implementation is to simplify the original mesh, and the simplified mesh is used as the base mesh.

[0125] Optionally, the mesh simplification in the embodiments of the present application can be performed in the following manner:

[0126] Mesh simplification is to simplify the current input mesh to a base mesh with relatively fewer points and faces, and to maintain the shape of the original mesh as much as possible. The focus of mesh simplification is the simplification operation and the corresponding error metric. A feasible mesh simplification operation is shown in FIG. 1, which combines the two vertices at the ends of an edge into one vertex and deletes the connection between the two vertices. This process is repeated in the entire mesh according to certain rules to reduce the number of faces and vertices of the mesh to the target value. Figure 6

[0127] In the simplification process, an error metric can be selected to optimize the simplification result. For example, the sum of the equation coefficients of all adjacent faces of a vertex can be selected as the error metric of the vertex, and the error metric of the corresponding edge is the sum of the error metrics of the two vertices of the edge. In other words, the error caused by the combination of an edge is the sum of the distances from the combined vertex to all adjacent planes of the original two vertices of the edge.

[0128] After the simplification operation and the corresponding error metric are determined, the mesh simplification is iteratively performed. First, the vertex error of the initial mesh is calculated to obtain the error of each edge. Then, each edge is arranged in ascending order of error, and the edge with the smallest error is selected for combination each time. At the same time, the position of the combined vertex is calculated, and the error of all edges related to the combined vertex is updated. That is, the order of edge arrangement is updated to ensure that each iteration is based on the global error metric. Through iteration, the number of faces of the mesh is simplified to meet the requirements of lossy coding.

[0129] Of course, other simplification methods can also be used in the embodiments of the present application, and any implementation scheme that can simplify the mesh based on the simplification purpose to obtain a base mesh falls within the protection scope of the present application.

[0130] The second obtaining manner is that the encoding end simplifies the original mesh to obtain a simplified mesh, and generates a mesh based on the simplified mesh to obtain a base mesh.

[0131] It should be noted that this implementation is to simplify the mesh first to obtain a simplified mesh, and then generate a base mesh based on the simplified mesh. This implementation can be understood as performing one more mesh generation process based on the first obtaining manner. Optionally, the process of obtaining the simplified mesh in this obtaining manner can be referred to the implementation above, and will not be described here.

[0132] Optionally, the mesh generation in the embodiments of the present application can be performed in the following manner:​

[0133] The specific process is as shown in the following figure. Figure 7 As shown in the figure, first, the reference mesh (which is obtained based on mesh simplification), the mapping mesh obtained by simplifying the mesh, and the simplified mesh are subdivided respectively, the mesh obtained after subdivision is subjected to initial deformation to generate an initial deformation mesh, and then the initial deformation mesh is subjected to iterative deformation. In the iterative deformation process, a certain error can be selected to optimize the initial deformation mesh. For example, the mean square error of the position coordinates of the initial deformation mesh and the reference mesh can be selected as the minimum criterion to optimize the position coordinates of the initial deformation mesh. After smoothing and other operations on the optimized initial deformation mesh, the final deformation mesh is output. At this time, the displacement information between the final deformation mesh and the subdivided reference mesh is used as the output of the iterative deformation to adjust the vertex position of the simplified mesh. The simplified mesh after adjusting the position is the generated base mesh.

[0134] Of course, other mesh generation methods can also be used in the embodiments of the present application, and any implementation scheme that can generate a base mesh belongs to the protection scope of the present application.

[0135] Optionally, in an implementation, the specific implementation of obtaining the texture information corresponding to the reconstructed base mesh according to the reconstructed base mesh comprises:

[0136] The encoding end parameterizes the reconstructed base mesh to obtain a parameterized mesh, partition information and partition arrangement information of the reconstructed base mesh, and the parameterized mesh includes the texture coordinates of the vertices.

[0137] The encoding end determines at least one of the texture coordinates of the vertices, the partition information and the partition arrangement information of the reconstructed base mesh as the texture information corresponding to the reconstructed base mesh.

[0138] It should be noted that the parameterized mesh can be understood as a new mesh that increases the texture coordinates of the vertices and is not completely consistent with the vertex positions of the reconstructed base mesh. Through the parameterization process, the texture information corresponding to the vertices of the reconstructed base mesh can be obtained.

[0139] It should be noted that the general parameterization is a process of establishing a mapping between two topological surfaces with distortion minimization as the target. In particular, if the parameterization establishes a mapping between a three-dimensional mesh and a two-dimensional plane, and the texture coordinates of the output three-dimensional mesh are used as the generated mapping relationship, it is called mesh parameterization. For example, UV Atlas or Ortho Atlas can be used to parameterize the mesh to obtain the texture coordinates.

[0140] The different parameterization implementation methods are described as follows.

[0141] I. UV Atlas

[0142] The basic idea of UV Atlas is to generate the initial partition of the input mesh by surface spectral analysis of the input mesh (i.e. the reconstruction base mesh), then iteratively partition the mesh with the minimum error criterion, if the error is less than the user set value, stop the iteration, output the partition at this time as the final partition, otherwise continue the iteration until the error is less than the set value, then arrange the generated partitions to a two-dimensional plane, thereby obtaining the texture coordinates of the input mesh.

[0143] The specific process mainly includes two processes: UV Atlas partition generation and UV Atlas partition arrangement.

[0144] 1. UV Atlas partition generation

[0145] UV Atlas generates the final partition by performing surface spectral analysis on the three-dimensional mesh, and specifically uses the isochart algorithm to generate the partition, wherein the process of the isochart algorithm is as follows:

[0146] Step S11, calculate the surface spectral analysis to provide an initial parameterization;

[0147] Step S12, perform the iteration process of stretch optimization;

[0148] Step S13, if the result of stretch optimization is less than the input value, end the iteration process;

[0149] Step S14, if the spectral analysis fails to produce a parameterization result, perform spectral clustering to divide the surface into charts;

[0150] Step S15, optimize the chart boundary by using the graph cut algorithm;

[0151] Step S16, perform recursive segmentation until the stretch is less than the input value.

[0152] 2. UV Atlas partition arrangement

[0153] UV Atlas uses the Tetris algorithm to arrange the partitions, such as Figure 8As shown, Tetris arrangement uses Horizon to represent the boundary of the current arranged chart, when the next chart arrangement is performed, Tetris calculates the top horizon and bottom horizon of the chart, and then finds the position with the smallest lost space to arrange, and updates the top horizon after arrangement, Tetris arrangement diagram is as shown in Figure 8 .

[0154] II. Ortho Atlas

[0155] The basic idea of Ortho Atlas is to partition the input mesh (i.e. the reconstruction base mesh) by calculating the properties (normal vector, etc.) of the triangular faces of the input mesh, combining the adjacency relationship of the triangular faces, generating connected regions, then using the method of orthogonal projection to orthogonally project each triangle in the connected region, and finally arranging the projected connected region to the two-dimensional plane to obtain the texture coordinates of the input mesh. The basic process of Ortho Atlas is as shown in Figure 9 .

[0156] The specific process mainly includes step S21 and step S22.

[0157] Step S21, Ortho Atlas partition generation

[0158] The specific process of Ortho Atlas partition generation is as shown in Figure 10 , and the Ortho Atlas partition generation mainly includes steps S211 to S214.

[0159] Step S211, generate initial partition.

[0160] Ortho Atlas first calculates the properties (area, normal vector, etc.) of each triangular face of the input mesh, and then determines the projection direction of each triangle according to the normal vector direction. Specifically, the process of the initial partition is to calculate the normal vector and the corresponding area of each face on the input mesh. The process of calculating the triangular face normal vector is as follows: define a triangular face T, whose three vertices are A, B, C, (x i ,y i ,z i ) are the geometric coordinates of the i-th vertex, then the normal vector of the current face T is AB×AC=((y B -y A )*(z C -z A )-(y C -y A )*(zB -z A ),(z B -z A )*(x C -x A )-(z C -z A )*(x B -x A ),(x B -x A )*(y C -y A )-(x C -x A )*(y B -y A )); where AB and AC represent the vectors corresponding to the two sides of triangle T, and × represents the vector cross product. After the normal vector is calculated by the above formula, the normal vector is normalized to obtain the unit normal vector of the current triangle. At the same time, the area of ​​the current triangle can be calculated by calculating the normal vector The modulus length is obtained, that is,

[0161] After obtaining the normal vector and area of ​​each triangle, the optimal projection plane for each triangle is determined based on a pre-defined projection plane. This process calculates the dot product of the current triangle's normal vector with the normal vectors of all projection planes and takes the projection plane corresponding to the maximum value as the projection plane for the current triangle. Triangles belonging to the same projection plane and connected in 3D space are then grouped into the same connected region. This process splits the input mesh into multiple connected regions.

[0162] Step S212: Merge adjacent connected regions.

[0163] After the initial partitioning, we begin to try to merge adjacent connected regions. Connected region adjacency is defined as two connected regions sharing at least one edge. When merging, we need to use a cost function to determine whether the two partitions can be merged. This function is defined by the perimeter and stretch degree of the connected regions, mainly including:

[0164] First, sort the connected regions in a specific order. The sorting criteria can be based on the number of triangles contained and the proximity between the average normal vector of the connected region and the corresponding projection direction, that is, the size of the dot product of the two vectors. The cost function of each connected region is COST(M i )=L 2 (M i )+λPERIMETER(M i );

[0165] After that, each time take the first connected region after sorting, get all connected regions adjacent to the candidate list, and then for each candidate region, first calculate the cost function after the two regions are merged: COST (M i +M j ) = L 2 (M i +M j ) + λPERIMETER (M i +M j ), for all (COST (M i +M j ) <COST (M i ) +COST (M j )) region, select the minimum cost function, merge the current region with the candidate region, update the merged connected region of the best projection plane, and then update the sorting of the connected region. Iteration until all connected regions are traversed.

[0166] Here the cost function is defined as COST (M i ) = L 2 (M i ) + λPERIMETER (M i ), where PERIMETER (M i ) represents the perimeter of the current connected region, L 2 (M i ) represents the stretching degree after projection of the current region, which is defined as follows:

[0167] Suppose a triangle T with two-dimensional texture coordinates p1, p2, p3, where p i = (s i , t i ), the corresponding three-dimensional coordinates are q1, q2, q3, and the calculation process of affine mapping S (p) = S (s, t) = q is shown in formula one:

[0168] Formula one,

[0169] S (p) = (<p, p2, p3> q1 + <p, p3, p1> q2 + <p, p1, p2> q3) / <p1, p2, p3>

[0170] Where <a, b, c> represents the area of triangle abc. Since the mapping is affine, its partial derivative is constant on (s, t), and its calculation process is shown in formula two and formula three:

[0171] Formula two,

[0172]

[0173] Equation Three,

[0174]

[0175] where A = <p1, p2, p3> = ((s2-s1)(t3-t1)-(s3-s1)(t2-t1)) / 2.

[0176] The larger and smaller singular values of the Jacobian matrix [S S ,S t ] are then computed, as shown in Equation Four and Equation Five:

[0177] Equation Four,

[0178]

[0179] Equation Five,

[0180]

[0181] where a = S s · S s , b = S s · S t , and c = S t · S t . The singular values γ max , γ min represent the maximum and minimum length that a unit length vector can obtain when mapped from the two-dimensional texture domain to the three-dimensional surface, i.e., the maximum and minimum local stretch. The stretch measure on a triangle T is defined as shown in Equation Six:

[0182] Equation Six,

[0183]

[0184] The stretch measure over the entire three-dimensional mesh M = {T i} is defined as shown in Equation Seven:

[0185] Equation Seven,

[0186]

[0187] where A'(T i ) is the surface area of the triangle T i in three-dimensional space.

[0188] The cost function is then defined as a weighted sum of L 2 (M i ) and PERIMETER(M i) is defined by the lambda parameter, which trades off the two variables. After the merging step, the number of partitions of a connected region is reduced compared to the initial partitioning.

[0189] Step S213, check occlusion of faces.

[0190] After the merging step, the projection of each connected region is started. During the projection, the current projected triangle face should not occlude the already projected region. This step first traverses all the faces of the current connected region and projects each face according to the projection face of the current connected region. That is, the two-dimensional coordinates of the three vertices of the current face after projection are calculated. Then it is necessary to determine whether the projected face overlaps the already projected region. Overlap mainly falls into three categories, which are: one or more vertices of the current projected face are in the already projected region; the vertices of the already projected region are in the region where the current projected face is located; one or more edges of the current projected face intersect with the edges of the already projected region.

[0191] By judging the above three cases, if the current projected face overlaps the already projected region, the projection of the current face is skipped and the next face is traversed. After a round of traversal, it is determined whether there are unprojected faces in the current connected region, and then these unprojected faces are projected as new connected regions and it is determined whether they overlap, and this iteration continues until the final projected region does not overlap. This step will split the connected region into non-overlapping connected regions.

[0192] Step S214, check connectivity.

[0193] After detecting and splitting the overlapping region, a connected region may be split into multiple connected regions. In these split connected regions, there may be some mutually unconnected faces or regions. This step is to extract these mutually unconnected regions in a connected region and project them as independent connected regions.

[0194] Step S215, merge small connected regions.

[0195] This step is to merge small connected regions, which can output the final partition.

[0196] Step S22, Ortho Atlas partition arrangement.

[0197] Ortho Atlas can use a variety of arrangement methods, for example:

[0198] Scale and rotation packing:

[0199] The arrangement method first sorts the patches generated by projection in descending order of area, and arranges them in this order, i.e., the largest area patch is arranged first, and the smallest area patch is arranged last. In the process of arranging each patch, scaling and rotation are allowed to make the most efficient use of the entire texture map.

[0200] Projection Packing:

[0201] Projection Packing is to arrange patches belonging to the same projection plane together. The specific method is divided into two steps: the first step is to arrange the patches in each projection direction in descending order of area in a region, and the second step is to arrange the projection patches already arranged in each projection direction in descending order of area. The advantage of this method over other arrangement methods is that it can arrange regions with similar positions in three-dimensional space to similar positions in the texture domain, which can better utilize spatial correlation in video encoding.

[0202] It should be noted that the above only exemplifies two partition arrangement methods, and other parameterized arrangement methods can also be used in the embodiments of the present application. Any implementation scheme that can perform partition arrangement belongs to the protection scope of the present application.

[0203] It should be noted that the above only exemplifies two parameterized implementation methods, and other parameterized methods can also be used in the embodiments of the present application. Any implementation scheme that can obtain texture information based on the parameterization process belongs to the protection scope of the present application.

[0204] It should be noted that the parameterized mesh obtained after parameterization is deviated from the reconstructed base mesh before parameterization. At this time, the parameterized mesh needs to be adjusted to make the vertices of the obtained mesh consistent with the vertex order of the reconstructed base mesh. Optionally, in one implementation method, the method further comprises:

[0205] The encoding end obtains parameterization auxiliary information when parameterizing the reconstructed base mesh, and the parameterization auxiliary information includes the correspondence between the faces in the reconstructed base mesh and the faces in the parameterized mesh obtained by parameterization.

[0206] The encoding end adjusts the parameterized mesh according to the parameterization auxiliary information to obtain a reconstructed base mesh with texture coordinates.

[0207] It should be noted that the parameterization changes the order of the vertices of the reconstructed base mesh, and therefore a parameterization auxiliary information needs to be output to record the change in the order of the vertices of the reconstructed base mesh before and after the parameterization, so as to adjust the order of the vertices of the parameterized mesh. Embodiments of the present application record the correspondence between the faces in the reconstructed base mesh and the faces in the parameterized mesh obtained by the parameterization. The correspondence can be represented by a partition index of each face in the parameterization input reconstructed base mesh, which is used to represent the influence of the parameterization on the order of the vertices of the reconstructed base mesh, and the partition index is output after the parameterization ends, and is used to guide the subsequent adjustment of the order of the vertices. That is, the partition index of each face in the parameterization input reconstructed base mesh is recorded and updated at each step where the parameterization partition information changes.

[0208] For the above two parameterization processes, the parameterization auxiliary information is obtained in the following manner:

[0209] For the UV Atlas, the data structure of the partition includes the partition index of each face in the parameterization input mesh in the partition, and only this information needs to be output as the parameterization auxiliary information after the UV Atlas parameterization ends.

[0210] For the Ortho Atlas, this information is not output after the parameterization ends, and therefore, in order to obtain this information, embodiments of the present application propose to record the partition index of the triangle in the parameterization input mesh contained in the partition after each partition change by using an intermediate information ccTriangleList, which is dynamically updated during the current partition change process and is used in the next partition change. The face partition is updated after each partition change, and is output as the parameterization auxiliary information after the parameterization ends.

[0211] Optionally, in an implementation, the adjusting the parameterized mesh according to the parameterization auxiliary information to obtain the reconstructed base mesh with texture coordinates comprises:

[0212] The encoding end removes duplicate points from the parameterized mesh;

[0213] The encoding end adjusts the parameterized mesh from which the duplicate points are removed according to the parameterization auxiliary information to obtain the reconstructed base mesh with texture coordinates.

[0214] It should be noted that the parameterization changes the order of the vertices of the reconstructed base mesh and produces duplicate vertices. In order to make the reconstructed base mesh output by the parameterization correspond one-to-one with the decoded base mesh at the decoding end, embodiments of the present application add a parameterized mesh adjustment process after the parameterization, which adjusts the order of the vertices of the reconstructed base mesh output by the parameterization and removes duplicates. The specific process is as follows:Figure 11 As shown: the parameterized parameterized grid first passes through the duplicate point removal unit to remove the duplicate points generated by the parameterization, and then adjusts the vertex order of the parameterized grid after removing the duplicate points according to the parameterization auxiliary information output by the parameterization, and outputs the adjusted reconstruction base grid.

[0215] Optionally, in an implementation, the specific implementation of encoding the texture information includes at least one of steps S31 and S32.

[0216] Step S31: In the case that the decoding end does not generate the texture coordinates, the encoding end encodes the texture information together with the base grid, and the texture information includes the texture coordinates.

[0217] It should be noted that in this case, the decoding end does not need to generate the texture coordinates, and therefore the encoding end needs to encode the texture coordinates together with the base grid, and the decoding end directly decodes the texture coordinates of the base grid. Optionally, in this case, the code stream further includes auxiliary information code stream, and the auxiliary information code stream includes first indication information, that is, the encoding end only needs to inform the decoding end that the decoding end does not generate the texture coordinates.

[0218] Step S32: In the case that the decoding end generates the texture coordinates, the encoding end encodes the texture information together with the base grid, and the partition arrangement information is separately encoded as auxiliary information code stream; or, the texture information and the partition arrangement information obtained in the parameterization process are encoded as auxiliary information code stream, and the texture information includes the partition information and the partition arrangement information of the reconstruction base grid.

[0219] It should be noted that in this case, the decoding end needs to generate the texture coordinates, and therefore the encoding end does not need to encode the texture coordinates, but only needs to inform the decoding end of the partition arrangement information and the partition information for generating the texture coordinates, and the decoding end generates the texture coordinates based on the information; in one case, the partition information can be encoded together with the base grid, and in this case, the partition arrangement information needs to be separately encoded, that is, the auxiliary information code stream includes the first indication information and the partition arrangement information; in another case, the partition information can be encoded together with the partition arrangement information, that is, the auxiliary information code stream includes the first indication information, the partition arrangement information, and the partition information of the reconstruction base grid.

[0220] The specific implementation of encoding the texture information is illustrated as follows.

[0221] The texture information of the reconstructed base mesh is generated after parameterization, and according to preset parameters of the encoding end, the texture information of the reconstructed base mesh can be written into the generated base mesh in the form of texture coordinates, and can be encoded together with the geometry coordinates of the base mesh, or can be encoded in the form of partition information and partition arrangement information. The partition information can be written into the generated base mesh for encoding, or can be transmitted to the decoding end as auxiliary information together with the partition arrangement information for reconstructing the texture coordinates of the base mesh at the decoding end.

[0222] For the texture coordinate encoding mode, in the parameterization process, the input reconstructed base mesh is divided into several partitions, and then the partitions are arranged in a two-dimensional plane to generate the texture coordinates of each vertex of the partition. After parameterization, the partitions are merged to output the parameterized mesh. After removing the duplicate points of parameterization and adjusting the vertex order, the vertex position coordinates and order can one-to-one correspond to the reconstructed base mesh (i.e. the base network obtained by the decoding end). However, since the texture coordinates of the partition boundary points are redundant during encoding, the texture coordinates corresponding to each vertex of the generated base mesh are taken out and written into the generated base mesh for encoding. The taking-out method is as follows: for the parameterized mesh output by parameterization, all duplicate points generated in the pre-encoding and parameterization process are deleted, and the texture coordinates corresponding to the duplicate points are also deleted. Then the remaining texture coordinates are the texture coordinates that need to be encoded together with the generated base mesh. At this time, the reconstructed base mesh after removing the duplicate points corresponds one-to-one to the vertices of the generated base mesh. After taking out the texture coordinates corresponding to the vertex coordinates of the generated base mesh, according to the correspondence between the reconstructed base mesh and the generated base mesh, the texture coordinates that need to be encoded are written into the generated base mesh, and are encoded together with the base mesh.

[0223] For the encoding mode by partition information and partition arrangement information, the partition information is the index of each partition in the parameterization input reconstruction base grid after parameterization ends, and the partition arrangement information includes: the index of the current partition projection plane, generally 0-5, representing the serial number of the projection plane; the index of the angle of the current partition when arranged on the image, generally 0-3, representing the four rotatable angles; the position of the upper left corner of the partition bounding box on the image when arranged on the image and the width and height of the bounding box; whether the current partition is scaled and the corresponding scaling coefficient. For the partition information, if it is indicated that the partition information is encoded in the base grid, the way and position of the partition information written in the base grid are determined according to the preset parameters of the Ortho Atlas. Specifically, if the Ortho Atlas encodes the partition information through one channel of the texture coordinates of the base grid, the extracted partition information is written into one channel of the texture coordinates of the generated base grid; if the Ortho Atlas encodes the partition information through one attribute of the face of the base grid, the partition information is written into one attribute of the face of the generated base grid; if the Ortho Atlas encodes the partition information through a connected region, the extracted partition information is written into the connected region and encoded together with the connected region. If it is indicated that the partition information is encoded in the auxiliary information, the partition information is encoded in the auxiliary information together with the partition arrangement information.

[0224] For example, as shown in FIG. 1, the overall flow of the encoding of an embodiment of the present application is as follows: Figure 12

[0225] The input mesh is first simplified to generate a simplified mesh, and the simplified mesh is subjected to a base grid generation process to generate a base grid to be encoded. The base grid to be encoded is first pre-encoded to generate a reconstruction base grid corresponding to the decoded base grid at the decoding end. Then, the reconstruction base grid is parameterized to generate corresponding texture coordinates, and parameterization auxiliary information is output for subsequent parameterized mesh adjustment. The base grid adjustment process adjusts the vertex order of the reconstruction base grid according to the indication of the parameterization auxiliary information and removes the duplicate points of the reconstruction base grid generated by parameterization. Subsequently, the reconstruction base grid is subdivided and deformed to generate a subdivision deformed mesh corresponding to the reconstruction base grid and a displacement. Then, the reconstructed displacement is applied to the subdivided reconstruction base grid to generate a reconstruction deformed mesh. Subsequently, a texture map corresponding to the reconstruction base grid is generated through texture map conversion. Finally, the displacement and the texture map are encoded in the manner of displacement encoding and texture map encoding according to the indication of the preset parameters.

[0226] ​For the encoding of the basic grid, it can be indicated according to preset parameters whether it is necessary to restore the texture coordinates of the basic grid at the decoding end. If not, the texture coordinates corresponding to the reconstructed basic grid are taken out and written into the generated basic grid, and encoded together with the geometric coordinates of the basic grid; if the texture coordinates need to be restored at the decoding end, the partition information and partition arrangement information need to be transmitted to the decoding end to guide the reconstruction of the texture coordinates at the decoding end. Therefore, in the embodiment of the present application, a partition information extraction process is set up, and the extracted partition information is written into the basic grid, and the partition arrangement information of each partition is directly encoded as an Atlas code stream, that is, an auxiliary information code stream; alternatively, the partition information and the partition arrangement information are encoded together to obtain an auxiliary information code stream.

[0227] It should be noted that the implementation process of simplified mesh generation, basic mesh generation, basic mesh reconstruction, parameterization, parameterized mesh adjustment and texture information encoding can be found in the above description and will not be repeated here.

[0228] Below Figure 12 Other implementation processes in the implementation flow are described as follows.

[0229] 1. Subdivision deformation process

[0230] In the embodiment of the present application, it is necessary to regenerate the mapping mesh for the parameterized reconstruction base mesh (i.e., the reconstruction base mesh with texture coordinates), and the subdivision deformation framework is as follows: Figure 13 As shown, the reference mesh generated by removing duplicate points from the original input mesh is input into the subdivision deformation module as the reference mesh of the entire subdivision deformation module. The reference mesh is first subdivided to generate a subdivision reference mesh. The parameterized output reconstructed base mesh is deduplicated and the vertex order is adjusted before input into the subdivision deformation module. The subdivided reconstructed base mesh is output after subdivision, and the reconstructed base mesh is mapped to generate its corresponding mapping mesh. The subdivided subdivision mapping mesh, the subdivided reference mesh, the subdivided reconstructed base mesh and the subdivision mapping mesh are used as the input of the initial deformation module for the initial deformation operation to generate the initial deformed mesh. Finally, the deformation iteration generates the subdivided deformed mesh corresponding to the reconstructed base mesh.

[0231] 2. Displacement processing and encoding

[0232] like Figure 14 As shown, the displacement vector is calculated by the difference in vertex position coordinates between the deformed mesh output by the subdivision deformation and the mesh after subdivision of the reconstructed base mesh (the black solid line in the figure below). Figure 1 The two levels of the deformed mesh, LoD1 (blue solid line in the figure) and LoD2 (red solid line in the figure), and the corresponding LoD1 and LoD2 levels of the displacement vector are shown in Figure 3.

[0233] There are multiple ways to encode the displacement. One way is to consider the coordinate system in which the displacement is converted, i.e. to convert the coordinate system of the displacement of each vertex into a coordinate system constructed by the normal vector of the vertex corresponding to the displacement and two components tangent to the normal vector. Then the displacement is transformed, such as wavelet transform. The transformed coefficients are quantized and arranged in the image according to the scanning order, and the video encoding is applied to the image. In addition, the generated or processed displacement can also be directly encoded using entropy encoding.

[0234] The implementation process of video encoding mainly includes coordinate system conversion, wavelet transform, wavelet coefficient quantization, wavelet coefficient arrangement and video encoding process, specifically:

[0235] coordinate system conversion,

[0236] The generated displacement is in the same global coordinate system as the input mesh. One possible optimization method is to convert it into a local coordinate system, and the local coordinate system of each vertex is defined by the normal vector of the vertex on the subdivision mesh. The advantage of this method is that the normal component of the displacement has a more significant impact on the quality of the reconstructed mesh than the two tangent components, and therefore a larger quantization parameter can be set for the tangent components.

[0237] wavelet transform,

[0238] For the quantized displacement, wavelet transform can be applied to it, and one possible wavelet transform, such as linear wavelet transform, is as follows:

[0239] The prediction process is defined as formula eight:

[0240] Formula eight,

[0241]

[0242] where v is the midpoint newly inserted on the edge (v1, v2), Signal(v), Signal(v1) and Signal(v2) are the displacements corresponding to the vertices v, v1 and v2 respectively.

[0243] The update process is defined as formula nine:

[0244] Formula nine,

[0245]

[0246] where v * is the set of all adjacent vertices of vertex v.

[0247] wavelet coefficient quantization,

[0248] There are multiple ways to quantize the wavelet coefficients, and one method is shown in formulas ten and eleven:

[0249] Equation Ten,

[0250] disp[v].d[k] = floor(disp[v].d[k] * scale[k])

[0251] Equation Eleven,

[0252]

[0253] where disp[v] represents the wavelet coefficients of the displacement of the vth vertex, which is the same concept as Signal(v) above, d[k] represents the kth coefficient of its displacement, floor represents rounding down. bitDepthPosition represents the bit depth of the current grid position, and qp[k] represents the quantization parameter of the kth coefficient. As mentioned above, after converting the displacement to the coordinate system, the normal component has a more significant impact on the quality than the tangential component, so a larger quantization parameter can be used for the tangential component.

[0254] At the same time, according to the characteristics of wavelet transform, different quantization parameters can also be used for the newly generated vertices and the original vertices in the subdivision. That is, for the vertices after subdivision, the quantization parameter update is as shown in Equation Twelve:

[0255] Equation Twelve,

[0256] scale[k] = scale[k] * lodScale[k]

[0257] where lodScale[k] represents the coefficient of the quantization parameter of the current subdivision level.

[0258] Wavelet coefficient arrangement,

[0259] For the wavelet coefficients generated after wavelet transform, they are arranged on a two-dimensional image in the following way:

[0260] Traverse the wavelet coefficients in the order from low frequency to high frequency;

[0261] For each coefficient, determine the index of the NxM pixel block (for example, N = M = 16) in which it should be stored in the raster scan order of the block;

[0262] Calculate the position of the corresponding NxM pixel block on the image according to the Morton order.

[0263] It should be noted that the arrangement method here is not limited, and other arrangement schemes such as zigzag order, raster order, etc. can also be used. The encoder can explicitly specify the corresponding arrangement scheme in the code stream.

[0264] Video encoding,

[0265] After the wavelet coefficients are arranged on a two-dimensional image, they can be directly encoded using a video encoder. The proposed scheme is independent of the video coding technology used, and the type of video encoder can be explicitly specified in the bitstream. Meanwhile, if quantization is applied before video coding, the video coding here can use lossless methods, and there is no need to reconstruct the displacement at the encoding end.

[0266] Implementation of entropy coding: For the generated displacement, existing entropy coding algorithms such as CABAC can be directly used for compression. Entropy coding can also be directly applied to the wavelet-transformed coefficients. Corresponding quantization can be performed before entropy coding.

[0267] 3. Base mesh encoding

[0268] There are three different encoding modes for base mesh compression, namely intra mode and inter mode, and skip mode. In the intra mode, the input of the base mesh encoding module is a three-dimensional mesh, including geometric coordinates, connection relationships, and attribute information associated with the vertices. The existing static mesh encoder is used for encoding. In the inter mode, the base mesh first determines whether to use the skip mode through RDO (Rate Distortion Optimization). If the skip mode is used, the reference mesh in the mesh buffer is directly selected as the current frame base mesh. If the skip mode is not used, the motion vector between the geometric positions of the vertices of the reference frame and the current frame is calculated, and the corresponding motion vector is encoded. After encoding, the encoded mesh needs to be reconstructed to provide processing for subsequent modules.

[0269] For the texture coordinate part, in the inter mode or skip mode, the texture coordinates of the vertices of the current frame directly reuse the texture coordinates of the corresponding vertices of the reference frame, so only the related texture information needs to be encoded in the intra mode. In the intra mode, the current mesh codec not only supports directly encoding the texture coordinates of each vertex of the base mesh, but also supports regenerating the texture coordinates of the base mesh at the decoding end. Therefore, in addition to directly encoding the texture coordinates, the related information of the mesh parameterization partition can also be encoded to guide the regeneration of the texture coordinates at the decoding end. The specific encoding process is to encode the partition index number of each face in addition to the geometric information and connection relationships of the mesh in the static mesh encoder. The partition index number of each face can be placed in one of the channels of the texture coordinates of its three vertices and encoded using the static mesh encoder, or the partition index number can be directly encoded as an attribute of the face.

[0270] 4. Texture map conversion and encoding

[0271] The texture map encoding module first performs texture map conversion according to the input original mesh, the input original texture map and the input reconstructed morphed mesh, then outputs a texture map suitable for the reconstructed morphed mesh, and finally performs video encoding on the converted texture map using a video encoder.

[0272] The step of texture map conversion is as follows:

[0273] Step S41, the texture coordinates of each pixel on the texture map to be generated are calculated, such as the texture coordinates corresponding to the pixel A(i,j) being P(u,v).

[0274] Step S42, it is judged whether the texture coordinates are in a triangular face after parameterization of the morphed mesh.

[0275] Step S43, if the texture coordinates do not belong to any triangular face, the pixel is marked as an empty pixel, and then a filling algorithm can be used for filling.

[0276] Step S44, if the texture coordinates belong to a triangular face, then:

[0277] the pixel is marked as filled;

[0278] the barycentric coordinates of the pixel in the current triangular face are calculated according to the texture coordinates;

[0279] the two-dimensional texture coordinates are mapped to three-dimensional geometric coordinates according to the barycentric coordinates and the corresponding triangular face, i.e. to the point on the morphed mesh corresponding to the texture coordinates;

[0280] the point closest to the three-dimensional coordinates is found on the input original mesh;

[0281] the barycentric coordinates of the three-dimensional coordinates are calculated according to the triangular face where the three-dimensional coordinates are located and the barycentric coordinates are mapped to two-dimensional, the texture coordinates P'(u',v') are calculated;

[0282] the value A'(i',j') of the corresponding pixel position is obtained by sampling the input original texture map through the texture coordinates;

[0283] the value is assigned to the corresponding pixel A(i,j) on the texture map to be generated.

[0284] The texture maps Patch generated by texture conversion are discontinuous, which is not conducive to video encoding of the texture map, therefore, the existing filling algorithm (such as Push-Pull algorithm) is used to fill between the patches, which can make the transition between pixels more smooth and improve the efficiency of video encoding.

[0285] In conclusion, the embodiment of the present application can use the reconstructed base mesh to obtain the texture information in parameterization when encoding the texture information at the encoding end, which is consistent with the decoding of the texture information at the decoding end using the decoded base mesh, so as to ensure that the decoding end can obtain the texture coordinates corresponding to the decoded base mesh. Meanwhile, the repeated points are removed first and then the subdivision deformation is performed, which can reduce the redundancy of the mesh and improve the coding efficiency.

[0286] As shown in Figure 15 The embodiment of the present application provides a decoding method, which comprises the following steps:

[0287] Step 1501: The decoding end decodes the received code stream to obtain a base mesh with texture coordinates, wherein the code stream at least comprises a base mesh code stream and an auxiliary information code stream.

[0288] Step 1502: The decoding end removes the repeated points from the base mesh with texture coordinates to obtain a decoded mesh.

[0289] It should be noted that the repeated points are removed from the base mesh with texture coordinates, so as to ensure that the decoding end can correctly use the obtained displacement to decode the accurate mesh, and ensure the consistency of coding and decoding.

[0290] Optionally, the auxiliary information code stream comprises the following one:

[0291] First indication information, wherein the first indication information is used to indicate whether the decoding end generates the texture coordinates.

[0292] The first indication information and the partition arrangement information.

[0293] The first indication information, the partition arrangement information and the partition information of the reconstructed base mesh.

[0294] Optionally, in an implementation manner, the step of decoding the received code stream to obtain the base mesh with texture coordinates comprises the following steps:

[0295] The decoding end decodes the base mesh code stream to obtain the base mesh.

[0296] In a case where the first indication information in the auxiliary information code stream indicates that the decoding end does not generate the texture coordinates, the decoding end determines the decoded base mesh as the base mesh with texture coordinates.

[0297] In a case where the first indication information in the auxiliary information code stream indicates that the decoding end generates the texture coordinates, the decoding end reconstructs the texture coordinates in the base mesh according to the partition information of the reconstructed base mesh and the partition arrangement information in the code stream, and obtains the base mesh with texture coordinates.

[0298] It should be noted that the decoding is consistent with the encoding end, so that the decoding mode corresponds to the encoding end, and the decoding accuracy is ensured.

[0299] For example, as shown in FIG. 1, the decoding process of the embodiment of the present application specifically includes: Figure 16

[0300] The decoding end obtains each part of the code stream and decodes. After decoding the basic grid code stream, a decoded basic grid is generated; after decoding the auxiliary information code stream, it is determined according to the auxiliary information indication whether the decoding end needs to reconstruct the texture coordinates. If the decoding end needs to generate the texture coordinates, the auxiliary information is continuously decoded to obtain the partition arrangement information, and then the texture coordinates corresponding to the basic grid are reconstructed according to the partition information transmitted in the basic grid and the decoded partition arrangement information, to obtain the corresponding basic grid containing the texture coordinates. If the texture coordinates do not need to be reconstructed at the decoding end, the basic grid after decoding already contains the texture coordinates, and then the basic grid is subjected to repeated point removal and subdivision, and the deformed grid reconstructed at the decoding end is obtained by applying the decoded displacement to the subdivided basic grid, that is, the decoding grid finally output by the decoding end. The texture map code stream is decoded by the corresponding video decoder to obtain the texture map finally output by the decoding end.

[0301] The main implementation process in the above-mentioned method will be described as follows. Figure 16

[0302] I. Basic grid decoding

[0303] The basic grid decoding module decodes according to the grid encoding format of the input code stream. For the intra mode, if the auxiliary information indicates that the texture coordinates of the basic grid need to be decoded, the corresponding static grid decoder indicated by the auxiliary information is used for decoding. If the texture coordinates are supported to be regenerated at the decoding end, additional partition information needs to be decoded to indicate the partition index to which the current face belongs. For the inter mode, the corresponding motion vector is decoded, and the corresponding basic grid is reconstructed according to the reference frame in the grid buffer. For the skip mode, the corresponding reference frame in the grid buffer is directly used as the basic grid of the current frame.

[0304] II. Texture information decoding

[0305] 1. Texture coordinate decoding

[0306] If the auxiliary information after decoding indicates that the texture coordinates do not need to be reconstructed at the decoding end, it indicates that the texture information is encoded in the form of texture coordinates together with the geometric information of the basic grid. Therefore, the basic grid after decoding contains the texture coordinates of the basic grid, and the texture coordinates do not need to be reconstructed at the decoding end.

[0307] 2. Partition and partition arrangement information decoding ​​

[0308] If the decoded auxiliary information indicates that the texture coordinate reconstruction is needed at the decoding end, it means that the texture information is encoded in the form of partition information and partition arrangement information, and the reconstruction of the decoded base mesh texture coordinate needs to be performed with the help of the partition information and the partition arrangement information.

[0309] For the partition information, if the auxiliary information indicates that the partition information is encoded in the base mesh, the corresponding partition information can be obtained according to the base mesh decoded by the base mesh decoding module; if the auxiliary information indicates that the partition information is encoded together with the partition arrangement information, the partition information of the base mesh can be obtained by decoding the corresponding auxiliary information.

[0310] For the partition arrangement information, only the partition arrangement information of each partition in the auxiliary information code stream needs to be decoded to obtain the auxiliary information.

[0311] III. Texture coordinate reconstruction

[0312] After the base mesh decoding is completed, if the decoded auxiliary information indicates that the texture coordinate is regenerated at the decoding end, the texture coordinate reconstruction process needs to be performed, and the texture coordinate reconstruction module obtains the texture coordinate corresponding to each vertex according to the partition information obtained by the base mesh decoding module and the arrangement information of each partition in the auxiliary information. The specific process is as follows:

[0313] First, the index of the current partition on the surface is determined, which is set as i. Then, the projection plane index of the current partition is set as proId(i), the partition rotation index is set as orienId(i), the upper left corner position of the partition bounding box is set as posX(i) and posY(i), the width and height of the partition bounding box are set as sizeX(i) and sizeY(i), and the partition scaling factor is set as LoD(i) if it exists. Next, for the three vertices on the current surface, the texture coordinates corresponding to the vertices are determined according to the following formula, and the current vertex is set as v0(x0, y0, z0). The corresponding texture coordinates v t (u0, v0) are as follows:

[0314]

[0315] wherein width and height represent the width and height of the texture map, and the expressions of P, O and R are shown in the following formula:

[0316]

[0317]

[0318]

[0319]

[0320]

[0321]

[0322]

[0323]

[0324]

[0325]

[0326]

[0327]

[0328]

[0329]

[0330] According to the above formula, the corresponding texture coordinates of all vertices on the surface are calculated to obtain the reconstructed base mesh containing the texture coordinates.

[0331] Four, duplicate point removal

[0332] Since the encoding end will remove duplicate points from the reconstructed base mesh when subdividing and deforming, the de-duplicated reconstructed base mesh is generated, and the corresponding deformed mesh and displacement vector are generated. In order to enable the decoding end to correctly apply the displacement vector to the decoded base mesh, a duplicate point removal process is added to the decoded base mesh so that the decoding end can correctly reconstruct the decoded mesh.

[0333] Five, base mesh subdivision

[0334] After obtaining the decoded base mesh, the decoded base mesh needs to be subdivided, and the subdivision method and the number of iterations are determined by the auxiliary information obtained by decoding.

[0335] Six, displacement decoding and reconstruction

[0336] The displacement code stream is decoded by the displacement decoder. If the encoding end compresses the displacement by video encoding, the decoding end decodes it by the corresponding video decoder, and recovers it from the two-dimensional image according to the arrangement scheme in the corresponding order. Then, inverse transformation and inverse quantization are performed to recover the displacement consistent with the encoding end. If entropy encoding is used, it can be directly entropy decoded.

[0337] Seven, texture video decoding

[0338] The texture decoder is responsible for decoding the texture code stream, and the texture is decoded by the video decoder. The texture is converted to the color space that matches the input texture of the encoder. Figure 1 The consistent image format is obtained to obtain the final decoded output texture map.

[0339] 8. Deformed Mesh Reconstruction

[0340] After the base mesh and displacement decoding and reconstruction are completed, the subdivided deformed mesh is reconstructed based on the two parts of the mesh. If the auxiliary information indicates that the displacement encoding process uses the reference frame as a reference, that is, the displacement of the vertices in the area matching the reference frame in the decoded displacement is the residual of the displacement of the current frame vertex relative to the reference frame vertex, then first restore the displacement of the vertices in the current frame matching area through the vertices and residuals of the subdivided deformed mesh reconstructed by the reference frame, and then perform subsequent reconstruction steps. That is, the reconstructed base mesh is subdivided, consistent with the subdivision method of the encoding end. Add the corresponding displacement to each vertex of the subdivided mesh, as shown in Formula 13:

[0341] Formula 13:

[0342] deformedmesh[i].v[k]=subdivmesh[i].v[k]+displacement[k]

[0343] Among them, subdivmesh[i].v[k] is the geometric coordinates of the k-th vertex after the base mesh is subdivided in the current frame (index is i), displacement[k] is the displacement corresponding to the k-th vertex, and deformedmesh[i].v[k] is the geometric coordinates of the k-th vertex after subdivision and deformation in the current frame.

[0344] It should be noted that by providing a decoding method at the decoding end corresponding to the encoding end, the consistency of encoding and decoding can be ensured.

[0345] The encoding method provided in the embodiment of the present application can be executed by an encoding device. In the embodiment of the present application, the encoding device provided in the embodiment of the present application is described by taking the encoding method executed by the encoding device as an example.

[0346] like Figure 17 As shown, the encoding device 1700 of the embodiment of the present application includes:

[0347] A first acquisition module 1701 is used to reconstruct the basic grid and obtain a reconstructed basic grid;

[0348] A second acquisition module 1702 is configured to acquire texture information corresponding to the reconstructed basic mesh according to the reconstructed basic mesh;

[0349] The encoding module 1703 is configured to encode the texture information.

[0350] The sending module 1704 is configured to send the encoded bitstream to a decoding end, wherein the bitstream at least includes a base grid bitstream and an auxiliary information bitstream.

[0351] Optionally, the first obtaining module 1701 is configured to:

[0352] The connection relationship and the geometry information of the base grid are reconstructed, and a reconstructed base grid is obtained.

[0353] Optionally, the apparatus further includes one of the following:

[0354] The third obtaining module is configured to simplify an original grid to obtain a base grid.

[0355] The fourth obtaining module is configured to simplify an original grid to obtain a simplified grid, perform mesh generation based on the simplified grid, and obtain a base grid.

[0356] Optionally, the second obtaining module 1702 includes:

[0357] The first obtaining unit is configured to parameterize the reconstructed base grid to obtain a parameterized grid, partition information and partition arrangement information of the reconstructed base grid, wherein the parameterized grid includes texture coordinates of vertices.

[0358] The first determining unit is configured to determine at least one of the texture coordinates of the vertices, the partition information of the reconstructed base grid and the partition arrangement information as texture information corresponding to the reconstructed base grid.

[0359] Optionally, the apparatus further includes:

[0360] The fifth obtaining module is configured to obtain parameterization auxiliary information when the reconstructed base grid is parameterized, wherein the parameterization auxiliary information includes a correspondence between a face in the reconstructed base grid and a face in a parameterized grid obtained by parameterization.

[0361] The sixth obtaining module is configured to adjust the parameterized grid based on the parameterization auxiliary information to obtain the reconstructed base grid with texture coordinates.

[0362] Optionally, the sixth obtaining module includes:

[0363] The processing unit is configured to remove duplicate points from the parameterized grid.

[0364] The second obtaining unit is configured to adjust the parameterized grid from which the duplicate points are removed based on the parameterization auxiliary information to obtain the reconstructed base grid with texture coordinates.

[0365] Optionally, the encoding module 1703 comprises:

[0366] a first encoding unit, configured to write the texture information into the base mesh for encoding together with the base mesh without generating the texture coordinates at the decoding end, the texture information comprising the texture coordinates;

[0367] a second encoding unit, configured to write the texture information into the base mesh for encoding together with the base mesh in the case of generating the texture coordinates at the decoding end, the partition arrangement information being encoded into the auxiliary information bitstream separately; or the texture information and the partition arrangement information obtained in the parameterization process being encoded into the auxiliary information bitstream.

[0368] Optionally, the auxiliary information bitstream comprises one of the following:

[0369] the first indication information;

[0370] the first indication information and the partition arrangement information;

[0371] the first indication information, the partition arrangement information and the partition information of the reconstructed base mesh.

[0372] It should be noted that the apparatus embodiment corresponds to the method described above, and all implementation manners in the method embodiment are applicable to the apparatus embodiment, and the same technical effects can be achieved.

[0373] The encoding apparatus in the embodiment of the application can be an electronic device, for example, an electronic device with an operating system, or a component in the electronic device, for example, an integrated circuit or a chip. The electronic device can be a terminal or other device except the terminal. Illustratively, the terminal can include, but is not limited to, the types of the terminal 11 listed above, and the other device can be a server, a network attached storage (NAS) or the like, which is not limited in the embodiment of the application.

[0374] The encoding apparatus provided in the embodiment of the application can implement the processes of the method embodiment Figure 4 and achieve the same technical effects. To avoid repetition, details are not described herein.

[0375] The embodiment of the application further provides an encoding end, comprising a processor and a communication interface, the processor being configured to reconstruct a base mesh to obtain a reconstructed base mesh, obtain texture information corresponding to the reconstructed base mesh according to the reconstructed base mesh, encode the texture information, and send the encoded bitstream to a decoding end, the bitstream comprising at least a base mesh bitstream and an auxiliary information bitstream.

[0376] Optionally, the processor is configured to:

[0377] The connection relationship and geometric information of the basic grid are reconstructed to obtain a reconstructed basic grid.

[0378] Optionally, the processor is further configured to implement one of the following:

[0379] Simplify the original mesh to obtain the base mesh;

[0380] The original mesh is simplified to obtain a simplified mesh, and mesh generation is performed based on the simplified mesh to obtain a base mesh.

[0381] Optionally, the processor is configured to:

[0382] Parameterizing the reconstructed basic mesh, obtaining the parameterized mesh and partition information and partition arrangement information of the reconstructed basic mesh, wherein the parameterized mesh includes texture coordinates of vertices;

[0383] At least one of the texture coordinates of the vertex, the partition information of the reconstructed basic mesh, and the partition arrangement information is determined as texture information corresponding to the reconstructed basic mesh.

[0384] Optionally, the processor is further configured to:

[0385] When reconstructing the basic mesh parameterization, obtaining parameterization auxiliary information, the parameterization auxiliary information including: the correspondence between the faces in the reconstructed basic mesh and the faces in the parameterized mesh obtained by parameterization;

[0386] The parameterized mesh is adjusted according to the parameterized auxiliary information to obtain a reconstructed basic mesh with texture coordinates.

[0387] Optionally, the processor is configured to:

[0388] performing duplicate point removal on the parameterized grid;

[0389] The parameterized mesh with duplicate points removed is adjusted according to the parameterized auxiliary information to obtain a reconstructed basic mesh with texture coordinates.

[0390] Optionally, the processor is configured to:

[0391] In the case where the decoding end does not generate texture coordinates, the texture information is written into the base mesh and encoded together with the base mesh, the texture information including the texture coordinates; or

[0392] In the case that the texture coordinates are generated at the decoding end, the texture information is written into the base mesh and encoded together with the base mesh, and the partition arrangement information is separately encoded as the auxiliary information code stream; or the texture information and the partition arrangement information obtained in the parameterization process are encoded as the auxiliary information code stream, and the texture information includes the partition information and the partition arrangement information of the reconstructed base mesh.

[0393] Optionally, the auxiliary information code stream includes one of the following:

[0394] The first indication information includes one of the following:

[0395] The first indication information and the partition arrangement information;

[0396] The first indication information, the partition arrangement information, and the partition information of the reconstructed base mesh.

[0397] The encoding end embodiment corresponds to the above-mentioned method embodiment, and each implementation process and implementation manner of the above-mentioned method embodiment can be applied to the electronic device embodiment and can achieve the same technical effects. Specifically, Figure 18 A hardware structure diagram of an encoding end for implementing an embodiment of the present application.

[0398] The encoding end 1800 includes, but is not limited to, at least part of components such as a radio frequency unit 1801, a network module 1802, an audio output unit 1803, an input unit 1804, a sensor 1805, a display unit 1806, a user input unit 1807, an interface unit 1808, a memory 1809, and a processor 1810.

[0399] Those skilled in the art can understand that the encoding end 1800 can also include a power supply (such as a battery) for supplying power to each component, and the power supply can be logically connected to the processor 1810 through a power management system, so as to realize functions such as management of charging, discharging, and power consumption management through the power management system. Figure 18 The electronic device structure shown in the figure does not constitute a limitation on the electronic device, and the electronic device can include more or fewer components than the figure, or combine certain components, or different component arrangements, which are not described here.

[0400] It should be understood that in the embodiments of the present application, the input unit 1804 can include a graphics processing unit (GPU) 18041 and a microphone 18042. The graphics processor 18041 processes image data of a still picture or a video obtained by an image capture device (such as a camera) in a video capture mode or an image capture mode. The display unit 1806 can include a display panel 18061, which can be configured in the form of a liquid crystal display, an organic light-emitting diode, etc. The user input unit 1807 includes at least one of a touch panel 18071 and other input devices 18072. The touch panel 18071 is also called a touch screen. The touch panel 18071 can include two parts of a touch detection device and a touch controller. The other input devices 18072 can include, but are not limited to, a physical keyboard, function keys (such as volume control keys, on-off keys, etc.), trackballs, mice, joysticks, and the like, which will not be described here.

[0401] In the embodiments of the present application, after the radio frequency unit 1801 receives the downlink data from the access network device, it can be transmitted to the processor 1810 for processing. In addition, the radio frequency unit 1801 can send uplink data to the network side device. Generally, the radio frequency unit 1801 includes, but is not limited to, an antenna, an amplifier, a transceiver, a coupler, a low noise amplifier, a duplexer, etc.

[0402] The memory 1809 can be used to store software programs or instructions and various data. The memory 1809 can mainly include a first storage area storing programs or instructions and a second storage area storing data, wherein the first storage area can store an operating system, application programs or instructions required by at least one function (such as a sound playing function, an image playing function, etc.), and the like. In addition, the memory 1809 can include a volatile memory or a non-volatile memory, or the memory 1809 can include both volatile and non-volatile memories. The non-volatile memory can be a Read-Only Memory (ROM), a Programmable ROM (PROM), an Erasable PROM (EPROM), an Electrically EPROM (EEPROM), or a flash memory. The volatile memory can be a Random Access Memory (RAM), a Static RAM (SRAM), a Dynamic RAM (DRAM), a Synchronous DRAM (SDRAM), a Double Data Rate SDRAM (DDR SDRAM), an Enhanced SDRAM (ESDRAM), a Synch link DRAM (SLDRAM), and a Direct Rambus RAM (DRRAM). The memory 1809 in the embodiments of the present application includes but is not limited to these and any other suitable types of memories.

[0403] The processor 1810 can include one or more processing units; optionally, the processor 1810 integrates an application processor and a modem processor, wherein the application processor mainly processes operations related to an operating system, a user interface, and an application program, and the modem processor mainly processes wireless communication signals, such as a baseband processor. It can be understood that the above-mentioned modem processor can also not be integrated into the processor 1810.

[0404] The processor 1810 is configured to:

[0405] The base grid is reconstructed to obtain a reconstructed base grid; texture information corresponding to the reconstructed base grid is obtained according to the reconstructed base grid; the texture information is encoded; and the encoded code stream is sent to a decoding end, wherein the code stream at least includes a base grid code stream and an auxiliary information code stream.

[0406] Optionally, the processor 1810 is configured to:

[0407] reconstruct the connection relationship and the geometric information of the base mesh to obtain a reconstructed base mesh.

[0408] Optionally, the processor 1810 is further configured to implement one of the following:

[0409] simplifying the original mesh to obtain a base mesh;

[0410] simplifying the original mesh to obtain a simplified mesh, performing mesh generation based on the simplified mesh to obtain the base mesh.

[0411] Optionally, the processor 1810 is configured to:

[0412] parameterizing the reconstructed base mesh to obtain a parameterized mesh, partition information and partition arrangement information of the reconstructed base mesh, wherein the parameterized mesh comprises texture coordinates of vertices;

[0413] determining at least one of the texture coordinates of the vertices, the partition information and the partition arrangement information of the reconstructed base mesh as texture information corresponding to the reconstructed base mesh.

[0414] Optionally, the processor 1810 is further configured to:

[0415] when parameterizing the reconstructed base mesh, obtaining parameterization auxiliary information, wherein the parameterization auxiliary information comprises a correspondence between a face in the reconstructed base mesh and a face in the parameterized mesh obtained by parameterization;

[0416] adjusting the parameterized mesh according to the parameterization auxiliary information to obtain the reconstructed base mesh with the texture coordinates.

[0417] Optionally, the processor 1810 is configured to:

[0418] performing duplicate point removal on the parameterized mesh;

[0419] adjusting the parameterized mesh with the duplicate points removed according to the parameterization auxiliary information to obtain the reconstructed base mesh with the texture coordinates.

[0420] Optionally, the processor 1810 is configured to:

[0421] in a case where the texture coordinates are not generated at the decoding end, writing the texture information into the base mesh to be encoded together with the base mesh, wherein the texture information comprises the texture coordinates; or

[0422] In the case that the texture coordinates are generated at the decoding end, the texture information is written into the base mesh and encoded together with the base mesh, and the partition arrangement information is separately encoded into the auxiliary information code stream; or the texture information and the partition arrangement information obtained in the parameterization process are encoded into the auxiliary information code stream, and the texture information includes the partition information and the partition arrangement information of the reconstructed base mesh.

[0423] Optionally, the auxiliary information code stream includes one of the following:

[0424] The first indication information includes one of the following:

[0425] The first indication information and the partition arrangement information;

[0426] The first indication information, the partition arrangement information, and the partition information of the reconstructed base mesh.

[0427] Preferably, the embodiment of the application further provides an encoding end, including a processor, a memory, a program or instructions stored in the memory and executable on the processor, which implement each process of the encoding method embodiment and achieve the same technical effects when executed by the processor. To avoid repetition, details are not repeated here.

[0428] The embodiment of the application further provides a readable storage medium, which stores a program or instructions executable by a processor, which implement each process of the encoding method embodiment and achieve the same technical effects when executed by the processor. To avoid repetition, details are not repeated here.

[0429] The computer readable storage medium is, for example, a Read-Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk or an optical disk.

[0430] As shown in FIG. 19, the decoding device 1900 of the embodiment of the application includes: Figure 19 A decoding module 1901 is configured to decode a received code stream to obtain a base mesh with texture coordinates, the code stream including at least a base mesh code stream and an auxiliary information code stream.

[0431] A processing module 1902 is configured to remove repeated points from the base mesh with texture coordinates to obtain a decoded mesh.

[0432] Optionally, the auxiliary information code stream includes one of the following:

[0433] The first indication information is used to indicate whether the decoding end generates texture coordinates.

[0434] The first indication information is used to indicate whether the decoding end generates texture coordinates.

[0435] the first indication information and the partition arrangement information;

[0436] the first indication information, the partition arrangement information and the partition information of the reconstructed base mesh.

[0437] Optionally, the decoding module 1901 comprises:

[0438] a first determining unit, configured to decode the base mesh code stream to obtain a base mesh;

[0439] a second determining unit, configured to, in a case where the first indication information in the auxiliary information code stream indicates that the decoding end does not generate the texture coordinates, determine the decoded base mesh as a base mesh with texture coordinates;

[0440] a third determining unit, configured to, in a case where the first indication information in the auxiliary information code stream indicates that the decoding end generates the texture coordinates, perform reconstruction of the texture coordinates in the base mesh according to the partition information of the reconstructed base mesh and the partition arrangement information in the code stream, to obtain a base mesh with texture coordinates.

[0441] It should be noted that the apparatus embodiment corresponds to the method described above, and all implementation manners in the method embodiment are applicable to the apparatus embodiment, and the same technical effects can be achieved.

[0442] The decoding apparatus in the embodiment of the present applicationapplicationbe an electronic device, for example, an electronic device with an operating system, or a component in an electronic device, for example, an integrated circuit or a chip. The electronic deviceapplicationbe a terminal or other devices except the terminal. Exemplarily, the terminalapplicationinclude but not limited to the types of the terminal 11 listed above, and the other devicesapplicationbe a server, a network attached storage (NAS) or the like, which are not limited in the embodiment of the present application.

[0443] The decoding apparatus provided in the embodiment of the present applicationapplicationimplement the processes of the method embodimentapplicationachieve the same technical effects, and thus the details are not repeated here. Figure 15

[0444] The embodiment of the present application further provides a decoding end, comprising a processor and a communication interface, wherein the processor is configured to decode a received code stream to obtain a base mesh with texture coordinates, the code stream comprising at least a base mesh code stream and an auxiliary information code stream; and perform repeated point removal on the base mesh with texture coordinates to obtain a decoded mesh.

[0445] Optionally, the auxiliary information code stream comprises one of the following:

[0446] ​The first indication information is used for indicating whether the decoding end generates the texture coordinates or not.

[0447] The first indication information and the partition arrangement information.

[0448] The first indication information, the partition arrangement information and the partition information of the reconstructed base mesh.

[0449] Optionally, the processor is configured to:

[0450] decode the base mesh code stream to obtain the base mesh;

[0451] in a case where the first indication information in the auxiliary information code stream indicates that the decoding end does not generate the texture coordinates, determining the decoded base mesh as the base mesh with the texture coordinates;

[0452] in a case where the first indication information in the auxiliary information code stream indicates that the decoding end generates the texture coordinates, performing reconstruction of the texture coordinates in the base mesh according to the partition information of the reconstructed base mesh and the partition arrangement information in the code stream, to obtain the base mesh with the texture coordinates.

[0453] The decoding end embodiment corresponds to the method embodiment, and each implementation process and implementation manner of the method embodiment can be applied to the electronic device embodiment and achieve the same technical effects.

[0454] Optionally, the embodiment of the application further provides a decoding end, and a structure of the decoding end can be referred to as shown in Figure 18 , and details are not repeated here.

[0455] The processor is configured to:

[0456] decode the received code stream to obtain the base mesh with the texture coordinates, the code stream at least including a base mesh code stream and an auxiliary information code stream, and remove repeated points from the base mesh with the texture coordinates to obtain a decoded mesh.

[0457] Optionally, the auxiliary information code stream includes one of the following:

[0458] The first indication information is used for indicating whether the decoding end generates the texture coordinates or not.

[0459] The first indication information and the partition arrangement information.

[0460] The first indication information, the partition arrangement information and the partition information of the reconstructed base mesh.

[0461] Optionally, the processor is configured to:

[0462] decode the base mesh code stream to obtain the base mesh;

[0463] In a case that the first indication information in the auxiliary information code stream indicates that the decoding end does not generate the texture coordinates, the decoded base mesh is determined as the base mesh with the texture coordinates.

[0464] In a case that the first indication information in the auxiliary information code stream indicates that the decoding end generates the texture coordinates, the texture coordinates are reconstructed in the base mesh according to the partition information and the partition arrangement information of the reconstructed base mesh in the code stream, and the base mesh with the texture coordinates is obtained.

[0465] Preferably, the embodiment of the application further provides a decoding end, comprising a processor, a memory, a program or instructions stored in the memory and executable in the processor, which, when executed by the processor, implement each process of the above-mentioned decoding method embodiment and achieve the same technical effects. To avoid repetition, no further description is given here.

[0466] The embodiment of the application further provides a readable storage medium, in which a program or instructions are stored, which, when executed by a processor, implement each process of the above-mentioned decoding method embodiment and achieve the same technical effects. To avoid repetition, no further description is given here.

[0467] The computer readable storage medium is, for example, a Read-Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk or an optical disk, etc.

[0468] Optionally, as shown in Figure 20 The embodiment of the application further provides an electronic device 2000, comprising a processor 2001 and a memory 2002, in which a program or instructions executable in the processor 2001 are stored. In a case that the electronic device is an encoding end, the program or instructions, when executed by the processor 2001, implement each step of the above-mentioned encoding method embodiment and achieve the same technical effects. In a case that the electronic device is a decoding end, the program or instructions, when executed by the processor 2001, implement each step of the above-mentioned decoding method embodiment and achieve the same technical effects. To avoid repetition, no further description is given here.

[0469] The embodiment of the application further provides a chip, comprising a processor and a communication interface, wherein the communication interface is coupled with the processor, and the processor is used to run a program or instructions to implement each process of the above-mentioned encoding method or decoding method embodiment and achieve the same technical effects. To avoid repetition, no further description is given here.

[0470] It should be understood that the chip mentioned in the embodiments of the present application can also be called a system-level chip, a system chip, a chip system or a system-on-chip chip, etc.

[0471] An embodiment of the present application further provides a computer program / program product, which is stored in a storage medium. The computer program / program product is executed by at least one processor to implement the various processes of the above-mentioned encoding method or decoding method embodiment, and can achieve the same technical effect. To avoid repetition, it will not be repeated here.

[0472] An embodiment of the present application further provides a coding and decoding system, including: an encoding end and a decoding end, wherein the encoding end can be used to execute the steps of the above-mentioned encoding method, and the decoding end can be used to execute the steps of the above-mentioned decoding method.

[0473] It should be noted that, in this article, the terms "comprise", "include" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device comprising a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the presence of other identical elements in the process, method, article or device comprising the element. In addition, it should be noted that the scope of the methods and devices in the embodiments of the present application is not limited to performing functions in the order shown or discussed, and may also include performing functions in a substantially simultaneous manner or in the opposite order according to the functions involved. For example, the described method may be performed in an order different from that described, and various steps may also be added, omitted, or combined. In addition, the features described with reference to certain examples may be combined in other examples.

[0474] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus the necessary general hardware platform, and of course can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a computer software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), and includes a number of instructions for enabling a terminal (which can be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in each embodiment of the present application.

[0475] The embodiments of the present application are described above with reference to the accompanying drawings, but the present application is not limited to the specific embodiments described above, and the specific embodiments described above are merely illustrative, but not restrictive, and a person of ordinary skill in the art can make many forms under the inspiration of the present application without departing from the purpose of the present application and the scope protected by the claims.

Claims

1. An encoding method characterized by comprising: The method comprises the following steps: The encoding end reconstructs the base mesh to obtain a reconstructed base mesh; The encoding end obtains texture information corresponding to the reconstructed base mesh according to the reconstructed base mesh; The encoding end encodes the texture information; The encoding end sends the encoded code stream to the decoding end, wherein the code stream at least comprises a base mesh code stream and an auxiliary information code stream.

2. The method of claim 1, wherein, The method further comprises the following step: The encoding end simplifies the original mesh to obtain the base mesh; The encoding end simplifies the original mesh to obtain a simplified mesh, generates a mesh according to the simplified mesh, and obtains the base mesh.

3. The method according to claim 1 or 2, characterized in that, The step of obtaining the texture information corresponding to the reconstructed base mesh according to the reconstructed base mesh comprises the following steps: The encoding end parameterizes the reconstructed base mesh to obtain a parameterized mesh, partition information of the reconstructed base mesh, and partition arrangement information, wherein the parameterized mesh comprises texture coordinates of vertices; The encoding end determines at least one of the texture coordinates of the vertices, the partition information of the reconstructed base mesh, and the partition arrangement information as the texture information corresponding to the reconstructed base mesh.

4. The method according to any one of claims 1 to 3, characterized in that, The method further comprises the following steps: The encoding end obtains parameterization auxiliary information when parameterizing the reconstructed base mesh, wherein the parameterization auxiliary information comprises a correspondence between a face in the reconstructed base mesh and a face in the parameterized mesh obtained by parameterization; The encoding end adjusts the parameterized mesh according to the parameterization auxiliary information to obtain the reconstructed base mesh with texture coordinates.

5. The method of claim 4, wherein, The step of adjusting the parameterized mesh according to the parameterization auxiliary information to obtain the reconstructed base mesh with texture coordinates comprises the following steps: The encoding end removes duplicate points from the parameterized mesh; The encoding end adjusts the parameterized mesh with removed duplicate points according to the parameterization auxiliary information to obtain the reconstructed base mesh with texture coordinates.

6. The method according to any one of claims 1 to 5, characterized in that, The step of encoding the texture information comprises the following steps: In the case that the decoding end does not generate texture coordinates, the encoding end encodes the texture information into the base mesh together with the base mesh, wherein the texture information comprises texture coordinates; or In the case that the decoding end generates texture coordinates, the encoding end encodes the texture information into the base mesh together with the base mesh, and separately encodes the partition arrangement information into the auxiliary information code stream; or encodes the texture information and the partition arrangement information obtained in the parameterization process into the auxiliary information code stream, wherein the texture information comprises the partition information and the partition arrangement information of the reconstructed base mesh.

7. The method of claim 6, wherein, The auxiliary information code stream comprises the following information: First indication information, wherein the first indication information is used to indicate whether the decoding end generates texture coordinates; The first indication information and the partition arrangement information; The first indication information, the partition arrangement information, and the partition information of the reconstructed base mesh.

8. A decoding method, comprising: The method comprises the following steps: The decoding end decodes the received code stream to obtain a base mesh with texture coordinates, wherein the code stream at least comprises a base mesh code stream and an auxiliary information code stream; The decoding end removes duplicate points from the base mesh with texture coordinates to obtain a decoded mesh.

9. The method of claim 8, wherein, The auxiliary information code stream comprises the following information: The first indication information is used for indicating whether a texture coordinate is generated at a decoding end. The first indication information and the partition arrangement information. The first indication information, the partition arrangement information and the partition information of the reconstructed base mesh.

10. The method of claim 9, wherein, The decoding of the received code stream and the acquisition of the base mesh with the texture coordinate comprises: The decoding end decodes the base mesh code stream and acquires the base mesh. In a case where the first indication information in the auxiliary information code stream indicates that the decoding end does not generate the texture coordinate, the decoding end determines the decoded base mesh as the base mesh with the texture coordinate. In a case where the first indication information in the auxiliary information code stream indicates that the decoding end generates the texture coordinate, the decoding end reconstructs the texture coordinate in the base mesh according to the partition information of the reconstructed base mesh and the partition arrangement information in the code stream, and acquires the base mesh with the texture coordinate.

11. An encoding apparatus, comprising: Comprise: The first acquisition module is used for reconstructing the base mesh and acquiring the reconstructed base mesh. The second acquisition module is used for acquiring the texture information corresponding to the reconstructed base mesh according to the reconstructed base mesh. The encoding module is used for encoding the texture information. The sending module is used for sending the encoded code stream to the decoding end, and the code stream at least comprises a base mesh code stream and an auxiliary information code stream.

12. The apparatus of claim 11, wherein, Further comprise one of the following: The third acquisition module is used for simplifying the original mesh and acquiring the base mesh. The fourth acquisition module is used for simplifying the original mesh and acquiring the simplified mesh, and acquiring the base mesh according to the mesh generation of the simplified mesh.

13. The apparatus of claim 11 or 12, wherein, The second acquisition module comprises: The first acquisition unit is used for parameterizing the reconstructed base mesh, acquiring the parameterized mesh, the partition information of the reconstructed base mesh and the partition arrangement information, and the parameterized mesh comprises the texture coordinate of the vertex. The first determination unit is used for determining at least one of the texture coordinate of the vertex, the partition information of the reconstructed base mesh and the partition arrangement information as the texture information corresponding to the reconstructed base mesh.

14. The apparatus of any one of claims 11-13, wherein, Further comprise: The fifth acquisition module is used for acquiring the parameterization auxiliary information when the reconstructed base mesh is parameterized, and the parameterization auxiliary information comprises the corresponding relationship between the face in the reconstructed base mesh and the face in the parameterized mesh obtained by parameterization. The sixth acquisition module is used for adjusting the parameterized mesh according to the parameterization auxiliary information and acquiring the reconstructed base mesh with the texture coordinate.

15. The apparatus of claim 14, wherein, The sixth acquisition module comprises: The processing unit is used for removing repeated points from the parameterized mesh. The second acquisition unit is used for adjusting the parameterized mesh with the removed repeated points according to the parameterization auxiliary information and acquiring the reconstructed base mesh with the texture coordinate.

16. The apparatus of any one of claims 11-15, wherein, The encoding module comprises: The first encoding unit is used for writing the texture information into the base mesh and encoding the base mesh with the texture information in a case where the decoding end does not generate the texture coordinate, and the texture information comprises the texture coordinate; or, A second encoding unit is configured to, when the texture coordinates are generated at the decoding end, write the texture information into a base grid and encode it together with the base grid, and encode the partition arrangement information separately as an auxiliary information code stream; or, alternatively, encode the texture information together with the partition arrangement information obtained during the parameterization process as an auxiliary information code stream, wherein the texture information includes: partition information and partition arrangement information of the reconstructed base grid.

17. The apparatus of claim 16, wherein, The auxiliary information code stream includes the following: first indication information, where the first indication information is used to instruct the decoding end whether to generate texture coordinates; first instruction information and partition arrangement information; First indication information, partition arrangement information, and partition information of the reconstructed basic grid.

18. An encoding end, characterized by The method comprises a processor and a memory, wherein the memory stores a program or instruction that can be run on the processor, and when the program or instruction is executed by the processor, the steps of the encoding method according to any one of claims 1 to 7 are implemented.

19. A decoding apparatus, comprising: include: A decoding module, configured to decode a received code stream to obtain a basic mesh having texture coordinates, wherein the code stream includes at least a basic mesh code stream and an auxiliary information code stream; The processing module is used to remove duplicate points from the basic grid with texture coordinates to obtain a decoded grid.

20. The apparatus of claim 19, wherein, The auxiliary information code stream includes the following: first indication information, where the first indication information is used to instruct the decoding end whether to generate texture coordinates; first instruction information and partition arrangement information; First indication information, partition arrangement information, and partition information of the reconstructed basic grid.

21. The apparatus of claim 20, wherein, The decoding module includes: An acquisition unit, configured to decode the basic grid code stream to acquire a basic grid; a second determining unit, configured to determine the decoded basic mesh as a basic mesh with texture coordinates when the first indication information in the auxiliary information code stream indicates that the decoding end does not generate texture coordinates; The third determination unit is configured to reconstruct the texture coordinates in the base grid according to the partition information and partition arrangement information of the reconstructed base grid in the code stream, thereby obtaining the base grid with texture coordinates when the first indication information in the auxiliary information code stream indicates that the decoding end generates texture coordinates.

22. A decoding end, comprising: The method comprises a processor and a memory, wherein the memory stores a program or instruction that can be run on the processor, and when the program or instruction is executed by the processor, the steps of the decoding method according to any one of claims 8 to 10 are implemented.

23. A readable storage medium characterized by, The readable storage medium stores a program or instruction, and when the program or instruction is executed by the processor, the steps of the encoding method according to any one of claims 1 to 7 or the steps of the decoding method according to any one of claims 8 to 10 are implemented.

24. A chip, characterized by The chip includes a processor and a communication interface, the communication interface is coupled to the processor, and the processor is used to run a program or instruction to implement the steps of the encoding method according to any one of claims 1 to 7 or the steps of the decoding method according to any one of claims 8 to 10.

25. A computer program product, characterised in that, The method comprises computer instructions, which, when executed by a processor, implement the steps of the method according to any one of claims 1 to 10.

Images