Grid coding method, grid decoding method and related equipment

Abstract

The invention discloses a grid coding method, a grid decoding method and related equipment, and belongs to the technical field of computers.The grid coding method comprises the steps that a basic grid corresponding to a three-dimensional grid is obtained, subdivision processing is carried out based on the basic grid, and a subdivided grid is obtained; deformation operation is carried out on the basis of the subdivided grid, a deformed grid is obtained, displacement information corresponding to the subdivided grid is determined, and the displacement information is used for representing the displacement of the vertex of the subdivided grid relative to the deformed grid; selecting a target image block parameter of a video frame corresponding to the three-dimensional grid from a plurality of preset image block parameters, and arranging image blocks divided by the displacement information according to the target image block parameter into the video frame; and performing coding processing on the arranged video frames to obtain a first coding result.

Description

Technical Field

[0001] The present application belongs to the field of computer technology, and specifically relates to a grid encoding method, a grid decoding method and related equipment. Background Art

[0002] With the rapid development of multimedia technology, 3D models have become a new generation of digital media after audio, images, and videos. 3D mesh and point cloud are two commonly used ways to represent 3D models. Compared with traditional multimedia such as images and videos, 3D mesh models have stronger interactivity and realism, and are widely used.

[0003] In the related art, when the displacement information of the subdivided grids obtained based on the three-dimensional grid is arranged in the video at the encoding end, the displacement information is arranged in the video using fixed image block parameters, which has poor flexibility. Summary of the invention

[0004] The embodiments of the present application provide a grid encoding method, a grid decoding method and related equipment, which can solve the problem of poor flexibility in video arrangement of displacement information using fixed image block parameters.

[0005] In a first aspect, a grid coding method is provided, comprising:

[0006] Obtaining a basic grid corresponding to the three-dimensional grid, and performing subdivision processing based on the basic grid to obtain a subdivided grid;

[0007] Performing a deformation operation based on the subdivided mesh to obtain a deformed mesh, and determining displacement information corresponding to the subdivided mesh, wherein the displacement information is used to characterize the displacement of vertices of the subdivided mesh relative to the deformed mesh;

[0008] Selecting a target image block parameter of a video frame corresponding to the three-dimensional grid from a plurality of preset image block parameters, and arranging the image blocks divided by the displacement information according to the target image block parameter into the video frame;

[0009] The arranged video frames are encoded to obtain a first encoding result.

[0010] In a second aspect, a grid decoding method is provided, comprising:

[0011] Decoding a code stream corresponding to the three-dimensional grid to obtain a first decoding result, a second decoding result, and a third decoding result, wherein the first decoding result is used to characterize a video frame corresponding to the three-dimensional grid, the second decoding result is used to characterize a target image block parameter of the video frame corresponding to the three-dimensional grid, and the third decoding result is used to characterize a basic grid;

[0012] Performing subdivision processing based on the third decoding result to obtain a subdivided grid;

[0013] Performing displacement decoding on the first decoding result based on the second decoding result to obtain displacement information corresponding to the subdivided mesh, wherein the displacement information is used to represent the displacement of vertices of the subdivided mesh relative to the reconstructed mesh;

[0014] Reconstruction processing is performed based on the displacement information and the subdivided grid to obtain the reconstructed grid.

[0015] In a third aspect, a grid coding device is provided, comprising:

[0016] An acquisition module is used to acquire a basic grid corresponding to the three-dimensional grid, and perform subdivision processing based on the basic grid to obtain a subdivided grid;

[0017] A determination module, configured to perform a deformation operation based on the subdivided mesh to obtain a deformed mesh, and determine displacement information corresponding to the subdivided mesh, wherein the displacement information is used to characterize the displacement of vertices of the subdivided mesh relative to the deformed mesh;

[0018] An arrangement module, configured to select a target image block parameter of a video frame corresponding to the three-dimensional grid from a plurality of preset image block parameters, and arrange the image blocks divided by the displacement information according to the target image block parameter into the video frame;

[0019] The encoding module is used to encode the arranged video frames to obtain a first encoding result.

[0020] In a fourth aspect, a grid decoding device is provided, comprising:

[0021] A first decoding module is used to decode a code stream corresponding to a three-dimensional grid to obtain a first decoding result, a second decoding result, and a third decoding result, wherein the first decoding result is used to characterize a video frame corresponding to the three-dimensional grid, the second decoding result is used to characterize a target image block parameter of the video frame corresponding to the three-dimensional grid, and the third decoding result is used to characterize a basic grid;

[0022] A processing module, configured to perform subdivision processing based on the third decoding result to obtain a subdivided grid;

[0023] a second decoding module, configured to perform displacement decoding on the first decoding result based on the second decoding result to obtain displacement information corresponding to the subdivided mesh, wherein the displacement information is used to characterize the displacement of the vertices of the subdivided mesh relative to the reconstructed mesh;

[0024] A reconstruction module is used to perform reconstruction processing based on the displacement information and the subdivided grid to obtain the reconstructed grid.

[0025] In a fifth aspect, a terminal is provided, which includes a processor, a memory, and a program or instruction stored in the memory and executable on the processor, wherein the program or instruction, when executed by the processor, implements the steps of the method described in the first aspect; or, the program or instruction, when executed by the processor, implements the steps of the method described in the second aspect.

[0026] In a sixth aspect, a terminal is provided, comprising a processor and a communication interface, wherein the processor is used to: obtain a basic grid corresponding to a three-dimensional grid, perform subdivision processing based on the basic grid to obtain a subdivided grid; perform a deformation operation based on the subdivided grid to obtain a deformed grid, and determine displacement information corresponding to the subdivided grid, wherein the displacement information is used to characterize the displacement of vertices of the subdivided grid relative to the deformed grid; select a target image block parameter of a video frame corresponding to the three-dimensional grid from a plurality of preset image block parameters, and arrange the image blocks divided by the target image block parameter into the video frame; and perform encoding processing on the arranged video frame to obtain a first encoding result.

[0027] In a seventh aspect, a terminal is provided, comprising a processor and a communication interface, wherein the processor is used to: decode a code stream corresponding to a three-dimensional grid to obtain a first decoding result, a second decoding result and a third decoding result, wherein the first decoding result is used to characterize a video frame corresponding to the three-dimensional grid, the second decoding result is used to characterize a target image block parameter of the video frame corresponding to the three-dimensional grid, and the third decoding result is used to characterize a basic grid; perform subdivision processing based on the third decoding result to obtain a subdivided grid; perform displacement decoding on the first decoding result based on the second decoding result to obtain displacement information corresponding to the subdivided grid, wherein the displacement information is used to characterize the displacement of vertices of the subdivided grid relative to a reconstructed grid; perform reconstruction processing based on the displacement information and the subdivided grid to obtain the reconstructed mesh.

[0028] In an eighth aspect, a readable storage medium is provided, on which a program or instruction is stored. When the program or instruction is executed by a processor, the steps of the grid encoding method as described in the first aspect are implemented, or when the program or instruction is executed by a processor, the steps of the grid decoding method as described in the second aspect are implemented.

[0029] In the ninth aspect, a chip is provided, comprising a processor and a communication interface, wherein the communication interface is coupled to the processor, and the processor is used to run a program or instructions to implement the steps of the method described in the first aspect, or to implement the steps of the method described in the second aspect.

[0030] In the tenth aspect, a computer program / program product is provided, wherein the computer program / program product is stored in a non-volatile storage medium, and the program / program product is executed by at least one processor to implement the steps of the method described in the first aspect, or to implement the steps of the method described in the second aspect.

[0031] In an embodiment of the present application, a basic grid corresponding to a three-dimensional grid is obtained, and a subdivision process is performed based on the basic grid to obtain a subdivided grid; a deformation operation is performed based on the subdivided grid to obtain a deformed grid, and displacement information corresponding to the subdivided grid is determined, and the displacement information is used to characterize the displacement of the vertices of the subdivided grid relative to the deformed grid; a target image block parameter of a video frame corresponding to the three-dimensional grid is selected from a plurality of preset image block parameters, and the image blocks divided according to the target image block parameter are arranged in the video frame; and the arranged video frame is encoded to obtain a first encoding result. In this way, by selecting the target image block parameter of the video frame corresponding to the three-dimensional grid from a plurality of preset image block parameters, the flexibility of video arrangement of the displacement information can be improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] Figure 1 It is a schematic diagram of V3C encoding in the related art;

[0033] Figure 2 It is a schematic diagram of V3C decoding in the related art;

[0034] Figure 3 is a schematic diagram of V-DMC coding in the related art;

[0035] Figure 4 is a schematic diagram of V-DMC decoding in the related art;

[0036] Figure 5 is a flow chart of a grid coding method provided in an embodiment of the present application;

[0037] Figure 6 It is a displacement coding schematic diagram provided in an embodiment of the present application;

[0038] Figure 7 This is a schematic diagram of displacement decoding provided by an embodiment of the present application;

[0039] Figure 8 is a flow chart of a grid decoding method provided in an embodiment of the present application;

[0040] Fig. 9 This is a schematic diagram of a grid coding process provided by an embodiment of the present application;

[0041] Fig.10 It is a simplified schematic diagram of a grid provided in an embodiment of the present application;

[0042] Fig.11 is an example diagram of displacement calculation provided in an embodiment of the present application;

[0043] Fig.12 It is a basic grid compression schematic diagram provided in an embodiment of the present application;

[0044] Fig.13 This is a schematic diagram of displacement information arrangement provided by an embodiment of the present application;

[0045] Fig.14 This is a schematic diagram of attribute graph conversion provided by an embodiment of the present application;

[0046] Fig.15 This is a schematic diagram of a grid decoding process provided by an embodiment of the present application;

[0047] Fig.16 is a structural schematic diagram of a grid coding device provided in an embodiment of the present application;

[0048] Fig.17 It is a structural schematic diagram of a grid decoding device provided in an embodiment of the present application;

[0049] Fig.18 It is a structural diagram of a communication device provided in an embodiment of the present application;

[0050] Fig.19 It is a structural diagram of a terminal provided in an embodiment of the present application. DETAILED DESCRIPTION

[0051] The following will be combined with the drawings in the embodiments of the present application to clearly describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field belong to the scope of protection of this application.

[0052] The terms "first", "second", etc. of the present application are used to distinguish similar objects, and are not used to describe a specific order or sequence. It should be understood that the terms used in this way are interchangeable where appropriate, so that the embodiments of the present application can be implemented in an order other than those illustrated or described herein, and the objects distinguished by "first" and "second" are generally of one type, and the number of objects is not limited, for example, the first object can be one or more. In addition, "or" in the present application represents at least one of the connected objects. For example, "A or B" covers three schemes, namely, Scheme 1: including A but not including B; Scheme 2: including B but not including A; Scheme 3: including both A and B. The character " / " generally indicates that the objects associated with each other are in an "or" relationship.

[0053] The term "indication" in this application can be a direct indication (or explicit indication) or an indirect indication (or implicit indication). A direct indication can be understood as the sender explicitly informing the receiver of specific information, operations to be performed, or request results in the sent indication; an indirect indication can be understood as the receiver determining the corresponding information according to the indication sent by the sender, or making a judgment and determining the operation to be performed or the request result according to the judgment result.

[0054] The encoding and decoding end corresponding to the encoding and decoding method in the embodiment of the present application may be a terminal, which may also be referred to as a terminal device or a user terminal (User Equipment, UE). The terminal may be a mobile phone, a tablet computer (Tablet Personal Computer), a laptop computer (Laptop Computer) or a notebook computer, a personal digital assistant (Personal Digital Assistant, PDA), a handheld computer, a netbook, an ultra-mobile personal computer (ultra-mobile personal computer, UMPC), a mobile Internet device (Mobile Internet Device, MID), an augmented reality (augmented reality, AR) / virtual reality (virtual reality, VR) device, a robot, a wearable device (Wearable Device) or a vehicle-mounted device (Vehicle User Equipment, VUE), a pedestrian terminal (Pedestrian User Equipment, PUE) and other terminal side devices, and the wearable device includes: a smart watch, a bracelet, a headset, glasses, etc. It should be noted that the specific type of the terminal is not limited in the embodiment of the present application.

[0055] For ease of understanding, some contents involved in the embodiments of the present application are described below:

[0056] 1. 3D Grid

[0057] In recent years, with the rapid development of multimedia technology, related research results have been rapidly industrialized and have become an indispensable part of people's lives. Three-dimensional models have become a new generation of digital media after audio, images, and videos. Three-dimensional grids are a commonly used way to represent three-dimensional models. Compared with traditional multimedia such as images and videos, three-dimensional grid models have stronger interactivity and realism, making them more and more widely used in various fields such as commerce, manufacturing, construction, education, medicine, entertainment, art, and military.

[0058] Although there are many ways to represent 3D meshes, triangular meshes are still the most common representation method. A 3D mesh can be considered to be composed of three basic elements: vertices, edges, and faces. Vertices are the most basic elements in a mesh, and they define a position in a 3D space. Edges are line segments connecting two vertices in a mesh. Faces can be considered as polygons formed by closed paths of edges. For a triangular mesh, each face is a triangle.

[0059] The information contained in the mesh is usually divided into three categories: geometric information, connection information, and attribute information. Geometric information refers to the position of each vertex of the mesh in three-dimensional space. Connection information describes the association between the elements in the mesh, that is, the connection relationship between vertices. Attribute information is optional, and it can associate attributes with corresponding mesh elements (such as vertex color, normal vector, etc. can be associated with mesh vertices). Mesh parameterization can also be used to map the mesh from three-dimensional space to a two-dimensional plane area. This mapping relationship is usually described by a set of parameter coordinates, called UV coordinates or texture coordinates, which are associated with mesh vertices. Among them, U refers to the horizontal direction and V refers to the vertical direction. This two-dimensional mapping can be used to represent high-resolution attribute information, such as texture, normal vector, etc.

[0060] In almost all application fields that use 3D meshes (such as computational simulation, entertainment, medical imaging, digital cultural relics, computer design, e-commerce, etc.), as people's demand for higher visual effects of 3D mesh models increases, the models are becoming more and more complex, and the accuracy of the models is also increasing. Therefore, the amount of data required to represent the 3D mesh is also increasing accordingly. The above problems have led to the increasing complexity of the processing, visualization, transmission and storage of 3D meshes. 3D mesh compression can be regarded as a way to solve the above problems. It reduces the size of model data and is conducive to the processing, storage and transmission of 3D meshes. Therefore, it is necessary to propose an efficient and general 3D mesh compression algorithm.

[0061] Recently, the Moving Picture Expert Group (MPEG) of the International Organization for Standardization in the field of audio and video coding compression has begun to develop a compression standard for three-dimensional meshes, Video-based dynamic mesh coding (V-DMC or VDMC). This standard is specified based on the existing Visual Volumetric Video-based Coding (V3C) standard. The V3C standard provides a general method for compressing three-dimensional models, which can be presented in the form of point clouds, meshes or panoramic videos. Making the compression method of three-dimensional mesh models compatible with this standard is conducive to the promotion and applicability of this method. Therefore, it is of great significance to optimize the three-dimensional mesh encoding and decoding method in VDMC and combine the optimization method with the V3C standard. One possible optimization method is to optimize the displacement coding method. In the existing framework, the displacement is obtained by calculating the distance between the reconstructed mesh and the original mesh vertex, and its purpose is to improve the quality of the mesh. In the existing framework, the displacement is encoded using a video encoder, and providing multiple options for the encoding method of the displacement is conducive to improving the encoding performance.

[0062] 2. V3C Standard

[0063] The V3C standard provides a method for encoding and decoding various three-dimensional media through video or image coding technology. Specifically, it converts the three-dimensional media content from a three-dimensional representation to multiple two-dimensional representations (called V3C components) through projection and other methods before encoding, and then encodes the two-dimensional representation using existing video or image coding technology. V3C components mainly include occupancy components, geometric components, and attribute components. The occupancy component can indicate which areas in the two-dimensional representation are associated with the data in the three-dimensional representation; the geometric component represents information related to the position of the three-dimensional data in space, and the attribute component can provide attribute information corresponding to the vertex, such as material, texture, etc. In addition, the components also contain information on how to reconstruct the three-dimensional model through these components, which is called atlas information. An example diagram of the V3C standard is shown below. Figure 1 and Figure 2 shown.

[0064] The atlas information is used to associate all components, and additional information for reconstructing from 2D back to 3D is also included in the atlas components. The atlas consists of multiple basic units, called patches. Each patch represents an area in the available 2D components and contains the information needed to project the area back to 3D space.

[0065] 3. V-DMC

[0066] V-DMC is a standard developed by MPEG for compressing 3D meshes. Its main idea is to compress 3D meshes by using the existing V3C standard. Since 3D meshes have connection information that needs to be encoded, its specific encoding process is slightly different from V3C. It is necessary to expand the syntax and semantics of the V3C standard decoding end and the decoding operation to support the decoding and reconstruction of 3D meshes. Figure 3 and Figure 4 It is the current coding and decoding framework of V-DMC.

[0067] The overall framework of the encoding end is as follows Figure 3 As shown in the figure, for the input mesh, it is first simplified by the simplification module, and then the new texture coordinates are generated for the mesh through mesh parameterization. Then the parameterized mesh is subdivided and deformed, that is, new vertices are inserted on the mesh according to a specific subdivision method and the distance from the vertex of the subdivided mesh to the nearest neighbor of the input mesh is calculated, which is called displacement information. Subsequently, the mesh before subdivision, called the base mesh, is sent to the base mesh encoding module and compressed using the existing mesh encoder. In the inter-frame mode, the base mesh encoding module can generate a motion vector for each vertex of the base mesh according to the reference frame, and then only compress the motion vector.

[0068] The base mesh is reconstructed after encoding, and then the order of displacement is adjusted according to the vertex order of the reconstructed base mesh. Subsequently, the vertex displacement information after the order is adjusted is subjected to wavelet transformation, the transformed coefficients are quantized, and then the quantized coefficients are arranged into a two-dimensional image according to a specific scanning order, and the two-dimensional image is encoded using a video encoder. Then, the reconstructed displacement information is applied to the subdivided base mesh to obtain a reconstructed subdivided deformed mesh, and the mesh, the original input mesh and its corresponding texture map are input into the corresponding texture map conversion module to obtain the texture map corresponding to the reconstructed mesh, and the texture map is also encoded using a video encoder. The parameters used in the encoding process, such as the type of video encoder used, the type of mesh encoder, the transformation parameters, the quantization parameters, etc., are transmitted to the decoding end through auxiliary information.

[0069] The overall framework of the decoding end is as follows Figure 4As shown, for the received code stream, the decoding end first demultiplexes each part of the code stream to obtain the basic grid code stream, the displacement video code stream, the texture map video code stream and the auxiliary information code stream respectively. For the basic grid code stream, the grid decoder indicated by the auxiliary information is used to decode the basic grid. The displacement video code stream and the texture map video code stream are decoded by the video decoder. For the displacement part, after the video is decoded, the displacement needs to be taken out from the image through the displacement decoding module, and the steps such as inverse quantization and inverse transformation are performed, and then it is applied to the subdivided basic grid to obtain the deformed grid reconstructed by the decoding end. After decoding, the texture map is the texture map corresponding to the reconstructed deformed grid. The subsequent application or rendering module processes the reconstructed deformed grid and the decoded texture map as input.

[0070] The grid encoding method, grid decoding method and related equipment provided by the embodiments of the present application are described in detail below through some embodiments and their application scenarios in combination with the accompanying drawings.

[0071] See also Figure 5 , Figure 5 is a flowchart of a grid coding method provided in an embodiment of the present application, which can be applied to a coding end device, such as Figure 5 As shown, the grid coding method includes the following steps:

[0072] Step 101: Obtain a basic grid corresponding to a three-dimensional grid, and perform subdivision processing based on the basic grid to obtain a subdivided grid.

[0073] Among them, the three-dimensional grid can be mesh simplified to obtain the basic grid corresponding to the three-dimensional grid; or, the three-dimensional grid can be mesh simplified and mesh parameterized to obtain the basic grid corresponding to the three-dimensional grid; or, the basic grid corresponding to the three-dimensional grid can be obtained by other methods; and so on, this embodiment does not limit this.

[0074] In addition, the basic mesh may be subdivided to obtain a subdivided mesh. The subdivided process may be to add vertices to the edges constituting the basic mesh, and the basic mesh after adding the vertices is a subdivided mesh.

[0075] In one implementation, the grid may be represented in the form of a grid sequence, and obtaining a basic grid corresponding to the three-dimensional grid and performing subdivision processing based on the basic grid to obtain a subdivided grid may include:

[0076] A basic grid sequence corresponding to the three-dimensional grid sequence is obtained, and subdivision processing is performed based on the basic grid sequence to obtain a subdivided grid sequence.

[0077] Step 102: Perform a deformation operation based on the subdivided mesh to obtain a deformed mesh, and determine displacement information corresponding to the subdivided mesh, where the displacement information is used to characterize the displacement of vertices of the subdivided mesh relative to the deformed mesh.

[0078] The displacement information may be used to characterize the displacement of the vertices of the subdivided mesh relative to the vertices of the deformed mesh. The displacement information may also be described as displacement data. For example, for each vertex of the subdivided mesh, a corresponding vertex may be found on the deformed mesh, and the displacement information corresponding to the subdivided mesh includes displacement information for characterizing the displacement of the vertex of each subdivided mesh relative to the corresponding vertex on the deformed mesh. In addition, the deformation operation may be to deform the subdivided mesh into a mesh having the same shape as the three-dimensional mesh, and the mesh obtained after deformation is the deformed mesh.

[0079] In one implementation, the mesh may be represented in the form of a mesh sequence, and the deformation operation is performed based on the subdivided mesh to obtain the deformed mesh, and the displacement information corresponding to the subdivided mesh is determined, where the displacement information is used to represent the displacement of the vertices of the subdivided mesh relative to the deformed mesh, and may include:

[0080] A deformation operation is performed based on the subdivided mesh sequence to obtain a deformed mesh sequence; and displacement information corresponding to the subdivided mesh sequence is determined, where the displacement information is used to characterize the displacement of vertices of the subdivided mesh sequence relative to the deformed mesh sequence.

[0081] Step 103: Select a target image block parameter of the video frame corresponding to the three-dimensional grid from a plurality of preset image block parameters, and pack the image blocks divided by the target image block parameter into the video frame.

[0082] The number of the three-dimensional grids may be one or more. The video frames corresponding to the three-dimensional grids may include one or more video frames. The image block parameters may include an image block size, which may be referred to as a block size.

[0083] In one embodiment, the selecting of the target image block parameters of the video frame corresponding to the three-dimensional grid from a plurality of preset image block parameters, and arranging the image blocks divided by the target image block parameters into the video frame may include: for each three-dimensional grid in the plurality of the three-dimensional grids, selecting the image block parameters of the video frame corresponding to each three-dimensional grid from a plurality of preset image block parameters, and arranging the displacement information corresponding to each three-dimensional grid, according to the image blocks divided by the image block parameters of the video frame corresponding to each three-dimensional grid, into the video frame corresponding to each three-dimensional grid. Thus, for different video frames in the video frames corresponding to the three-dimensional grid, it is supported to select different image block parameters to arrange the displacement information in the video.

[0084] In one embodiment, the step of selecting a target image block parameter of a video frame corresponding to the three-dimensional grid from a plurality of preset image block parameters and arranging the image blocks divided by the target image block parameter into the video frame may include: selecting a target image block parameter from a plurality of preset image block parameters and arranging the image blocks divided by the selected target image block parameter into the video frame for the displacement information corresponding to all three-dimensional grids. Thus, for all video frames corresponding to the three-dimensional grid, it is supported to select an image block parameter to arrange all displacement information in the video.

[0085] In one implementation, the grid may be represented in the form of a grid sequence, and selecting a target image block parameter of a video frame corresponding to the three-dimensional grid from a plurality of preset image block parameters, and arranging the image blocks divided by the target image block parameter according to the displacement information into the video frame may include:

[0086] A target image block parameter of a video frame corresponding to a three-dimensional grid sequence is selected from a plurality of preset image block parameters, and the image blocks divided according to the target image block parameter are arranged in the video frame.

[0087] It should be noted that the selecting of target image block parameters of the video frame corresponding to the three-dimensional grid from multiple preset image block parameters may include selecting the target image block parameters of the video frame corresponding to the three-dimensional grid from multiple preset image block parameters according to a preset selection rule. For example, a large number of experiments may be conducted to test the encoding effects under different image block parameters, summarize the correspondence between image block parameters with better encoding effects and the features of the three-dimensional grid or subdivided grid or deformed grid, establish the correspondence between the features of the three-dimensional grid or subdivided grid or deformed grid and the image block parameters, and thus select the corresponding image block parameters for video arrangement based on the features of different three-dimensional grids or subdivided grids or deformed grids; or the selecting of target image block parameters of the video frame corresponding to the three-dimensional grid may include randomly selecting the target image block parameters of the video frame corresponding to the three-dimensional grid from multiple preset image block parameters; and so on. This embodiment is not limited to this.

[0088] Step 104: Encode the arranged video frames to obtain a first encoding result.

[0089] The video frames corresponding to the three-dimensional grid may include multiple video frames, and the encoding process on the arranged video frames may be to encode the video constituted by the arranged video frames to obtain a first encoding result.

[0090] It should be noted that there are many types of video encoders. For the grid, the most suitable block sizes for different video codecs are different, and different video codecs allow different types of block size divisions. For example, H264 allows 16x16 and 4x4 blocks, and H265 allows 64x64, 32x32 blocks, etc. This embodiment provides a block size selection function, which not only allows the same video codec to use different block size divisions, but also allows different video codecs to use different block size divisions. For example, for different video codecs, different preset image block parameters can be set, so that for different video codecs, it supports selecting image block parameters from different preset image block parameters for video arrangement.

[0091] In the related art, a fixed block size is used to arrange the video for displacement information, which is not suitable for a variety of displacement information distributions. The embodiment of the present application proposes a new video arrangement method with dynamic grid displacement, which allows multiple block size options to be provided when arranging the video for displacement information, and indicates the block size selected by the encoding end to the decoding end by encoding a mark. The decoding end can extract the displacement information from the decoded displacement video by using the obtained block size mark and the same arrangement order as the encoding end; and when using different video codecs to encode and decode the displacement information, different codecs are supported to use different block sizes.

[0092] In an embodiment of the present application, a basic grid corresponding to a three-dimensional grid is obtained, and a subdivision process is performed based on the basic grid to obtain a subdivided grid; a deformation operation is performed based on the subdivided grid to obtain a deformed grid, and displacement information corresponding to the subdivided grid is determined, and the displacement information is used to characterize the displacement of the vertices of the subdivided grid relative to the deformed grid; a target image block parameter of a video frame corresponding to the three-dimensional grid is selected from a plurality of preset image block parameters, and the image blocks divided according to the target image block parameter are arranged in the video frame; and the arranged video frame is encoded to obtain a first encoding result. In this way, by selecting the target image block parameter of the video frame corresponding to the three-dimensional grid from a plurality of preset image block parameters, the flexibility of video arrangement of the displacement information can be improved.

[0093] Optionally, the image block parameters include image block size.

[0094] The preset image block parameters may include the image block size. The target image block parameters may include the image block size.

[0095] In this implementation, the image block parameters include the image block size, so that when the displacement information is arranged in a video, the image block size can be flexibly selected to arrange the displacement information in a video frame.

[0096] Optionally, the three-dimensional grid includes a first grid and a second grid, a first video frame corresponding to the first grid and a second video frame corresponding to the second grid are different video frames, the target image block parameters include image block parameters of the first video frame and image block parameters of the second video frame, and selecting the target image block parameters of the video frame corresponding to the three-dimensional grid from a plurality of preset image block parameters, and arranging the image blocks divided by the displacement information according to the target image block parameters into the video frame includes:

[0097] Selecting an image block parameter of the first video frame from a plurality of preset image block parameters, and arranging the displacement information corresponding to the first grid into the first video frame according to the image blocks divided by the image block parameter of the first video frame;

[0098] An image block parameter of the second video frame is selected from the plurality of preset image block parameters, and the displacement information corresponding to the second grid is arranged in the second video frame according to the image blocks divided according to the image block parameters of the second video frame.

[0099] The displacement information corresponding to the first grid refers to the displacement information corresponding to the subdivided grid corresponding to the first grid; the displacement information corresponding to the second grid refers to the displacement information corresponding to the subdivided grid corresponding to the second grid. The image block parameters of the first video frame selected from a plurality of preset image block parameters may be the same as or different from the image block parameters of the second video frame selected from a plurality of preset image block parameters. The selection method of selecting the image block parameters of the first video frame may be the same as or different from the selection method of selecting the image block parameters of the second video frame.

[0100] In this implementation, the image block parameters of the first video frame are selected from a plurality of preset image block parameters, and the displacement information corresponding to the first grid is arranged in the first video frame according to the image blocks divided by the image block parameters of the first video frame; the image block parameters of the second video frame are selected from the plurality of preset image block parameters, and the displacement information corresponding to the second grid is arranged in the second video frame according to the image blocks divided by the image block parameters of the second video frame. Thus, for different video frames in the video frames corresponding to the three-dimensional grid, it is supported to select different image block parameters to arrange the displacement information in the video, which can further improve the flexibility of arranging the displacement information in the video.

[0101] Optionally, the code stream corresponding to the three-dimensional grid includes the first encoding result and the second encoding result, and the second encoding result is used to indicate the target image block parameters of the video frame corresponding to the three-dimensional grid.

[0102] The second encoding result may include an encoding result of the first parameter corresponding to each of the video frames, and the second encoding result is used to indicate that the image block parameters of each of the video frames are image block parameters represented by the corresponding first parameters; or, the second encoding result may include an encoding result of the second parameter, and the second encoding result is used to indicate that the image block parameters of each of the video frames are image block parameters represented by the second parameters.

[0103] In this implementation, the bitstream corresponding to the three-dimensional grid includes the first encoding result and the second encoding result, and the second encoding result is used to indicate the target image block parameters of the video frame corresponding to the three-dimensional grid. In this way, the image block parameters of the video frame corresponding to the three-dimensional grid are indicated by the second encoding result, so that the decoding end can decode the displacement information in the bitstream through the second encoding result carried in the bitstream.

[0104] Optionally, the second encoding result includes an encoding result of the first flag bit;

[0105] When the first flag bit is a first preset value, the second encoding result also includes an encoding result of a first parameter corresponding to each of the video frames, and the second encoding result is used to indicate that the image block parameters of each of the video frames are image block parameters represented by the corresponding first parameters.

[0106] The first parameter can be used to characterize the image block parameter. When the first flag is a first preset value, each of the video frames has a corresponding first parameter, and the second encoding result also includes an encoding result of the first parameter corresponding to each of the video frames.

[0107] For example, the first flag may be: afps_vdmc_ext_packing_block_size_enable_flag. The first preset value may be 1. The first parameter may be: afps_vdmc_ext_packing_block_size.

[0108] In this implementation, when the first flag is a first preset value, the second encoding result also includes an encoding result of the first parameter corresponding to each of the video frames, and the second encoding result is used to indicate that the image block parameter of each of the video frames is the image block parameter represented by the corresponding first parameter. In this way, the image block parameter of each of the video frames is jointly indicated by the first flag and the first parameter, so that different video frames can use their own image block parameters to arrange the displacement information in a video.

[0109] Optionally, the second encoding result also includes an encoding result of a second parameter;

[0110] When the first flag bit is a second preset value, the second encoding result is used to indicate that the image block parameters of each of the video frames are image block parameters represented by the second parameters.

[0111] The second parameter can be used to represent the image block parameter. When the first flag is a second preset value, all video frames can use the image block parameter represented by the second parameter to perform video arrangement on the displacement information.

[0112] For example, the first flag may be: afps_vdmc_ext_packing_block_size_enable_flag. The second preset value may be 0. The second parameter may be: asps_vdmc_ext_packing_block_size.

[0113] In this implementation, when the first flag is the second preset value, the second encoding result is used to indicate that the image block parameters of each of the video frames are image block parameters represented by the second parameter. In this way, the image block parameters of all video frames are jointly indicated by the first flag and the global second parameter, supporting the determination of the image block parameters of all video frames by only parsing the first flag and the second parameter.

[0114] As a specific embodiment, Figure 6 and Figure 7As shown, the embodiment of the present application proposes a new video arrangement method for dynamic grid displacement, which allows providing multiple block size options for video arrangement of displacement information, and indicates the block size selected by the encoding end to the decoding end by encoding a mark. The decoding end can extract the displacement information from the decoded displacement video by using the obtained block size mark and the same arrangement order as the encoding end.

[0115] Displacement is an important part of the dynamic mesh encoding and decoding process, storing the distance information from the vertex of the subdivided mesh obtained after the base mesh is subdivided to the nearest neighbor point of the original input mesh along the normal direction. In the current V-DMC encoding and decoding framework, the three components of the displacement information are arranged vertex by vertex into the three YUV channels of the video pixel, and the displacement information is encoded and decoded with the help of the video codec.

[0116] In this embodiment, the displacement information is first input into the pre-processing module at the encoding end, and the pre-processing module is used to perform all operations except the arrangement of the displacement information before video encoding, such as transformation, quantization, etc.; then the displacement information after pre-processing is arranged into the channel of the video pixel according to the preset rules. In the process of arranging the displacement information to the video, the video is divided into blocks as basic units, and the displacement information is arranged into the video frame block by block. This embodiment provides multiple block size options, and the encoding end can flexibly select a block size suitable for the current coding grid sequence for arrangement, and transmit the selected block size to the decoding end through a block size tag. This embodiment does not limit the specific arrangement order of the displacement information, and the arrangement order adopted by the encoding and decoding end is consistent. For example, the Morton order can be adopted in the video block for arrangement, and the raster scanning order can be adopted between the video blocks. After the displacement information is arranged into the video frame block by block, the arranged video is sent to the video encoder for encoding to obtain the displacement code stream. This embodiment does not limit the video format of the video, which can be YUV444, YUV400 or YUV420, etc. At the decoding end, the video is decoded by a video decoder, and the displacement information can be extracted from the video through block size marking according to the same arrangement order as the encoding end. The displacement information extracted from the video is input into the post-processing module to obtain the final displacement information. The operation of the post-processing module is the reverse implementation of the operation of the pre-processing module at the encoding end, such as inverse quantization and inverse transformation.

[0117] The embodiments of the present application can improve the flexibility of video arrangement of displacement information.

[0118] See also Figure 8 , Figure 8 is a flowchart of a grid decoding method provided by an embodiment of the present application, which can be applied to a decoding end device, such as Figure 8 As shown, the grid decoding method includes the following steps:

[0119] Step 201: Decode a code stream corresponding to a three-dimensional grid to obtain a first decoding result, a second decoding result, and a third decoding result, wherein the first decoding result is used to characterize a video frame corresponding to the three-dimensional grid, the second decoding result is used to characterize a target image block parameter of the video frame corresponding to the three-dimensional grid, and the third decoding result is used to characterize a basic grid;

[0120] Step 202: Perform subdivision processing based on the third decoding result to obtain a subdivision grid;

[0121] Step 203: performing displacement decoding on the first decoding result based on the second decoding result to obtain displacement information corresponding to the subdivided mesh, wherein the displacement information is used to represent the displacement of the vertices of the subdivided mesh relative to the reconstructed mesh;

[0122] Step 204: Reconstruct the mesh based on the displacement information and the subdivided mesh to obtain the reconstructed mesh.

[0123] Optionally, the image block parameters include image block size.

[0124] Optionally, target image block parameters of the video frame corresponding to the three-dimensional grid are obtained by selecting from a plurality of preset image block parameters.

[0125] Optionally, the three-dimensional grid includes a first grid and a second grid, a first video frame corresponding to the first grid and a second video frame corresponding to the second grid are different video frames; and the target image block parameters include image block parameters of the first video frame and image block parameters of the second video frame.

[0126] Optionally, the image block parameters of the first video frame are selected from a plurality of preset image block parameters, and the image block parameters of the second video frame are selected from the plurality of preset image block parameters.

[0127] Optionally, the second decoding result includes a first flag bit;

[0128] When the first flag bit is a first preset value, the second decoding result also includes a first parameter corresponding to each of the video frames, and the second decoding result is used to indicate that the image block parameters of each of the video frames are image block parameters represented by the corresponding first parameters.

[0129] Optionally, the second decoding result also includes a second parameter;

[0130] When the first flag bit is a second preset value, the second decoding result is used to indicate that the image block parameters of each of the video frames are image block parameters represented by the second parameters.

[0131] It should be noted that this embodiment is Figure 5 The corresponding decoding side implementation in the embodiment shown in the figure can be found in the specific implementation. Figure 5 In order to avoid repeated description, the relevant description of the embodiment shown will not be repeated in this embodiment, and the same beneficial effects can still be achieved.

[0132] The mesh encoding and decoding method is described below through two specific embodiments:

[0133] Embodiment 1:

[0134] Take the application of the grid coding method to the V-DMC coding framework as an example. Fig. 9 As shown, the grid coding method performed by the coding end includes the following process:

[0135] (1) Mesh simplification

[0136] Mesh simplification is to simplify the current input mesh to a basic mesh with relatively few points and faces, and keep the shape of the original mesh as much as possible. The key of mesh simplification is the simplification operation and the corresponding error measurement. A feasible mesh simplification operation is Fig.10 As shown in the figure, the vertices at both ends of the edge are merged into one vertex and the connection between the two vertices is deleted. This process is repeated in the entire mesh according to certain rules to reduce the number of faces and vertices of the mesh to the target value.

[0137] During the simplification process, you can select an error metric to optimize the simplified result. For example, you can select the sum of the coefficients of the equations of all adjacent faces of a vertex as the error metric of the vertex, and the error metric of the corresponding edge is the sum of the error metrics of the two vertices on the edge. In other words, the error generated by merging an edge is the sum of the distances of all adjacent planes from the merged vertex to the original two vertices of the edge.

[0138] After determining the simplification operation and the corresponding error metric, start iterating to simplify the mesh. First, calculate the vertex error of the initial mesh to get the error of each edge. Then arrange each edge from small to large error, and select the edge with the smallest error to merge each time. At the same time, calculate the position of the merged vertex and update the error of all edges related to the merged vertex. That is, update the order of edge arrangement to ensure that each iteration is based on the global error metric. Simplify the mesh face to the number required to meet lossy coding through iteration.

[0139] (2) Mesh parameterization

[0140] Texture coordinates may be regenerated based on the reconstructed base mesh of the current frame. This step requires generating texture coordinates for each attribute map of the input mesh. If multiple attribute maps have similar characteristics, they can share the same texture coordinates. There are many ways to generate texture coordinates, including mesh parameterization. There are many algorithms for parameterizing meshes, such as the Isocharts algorithm, which uses spectral analysis to achieve stretch-driven 3D mesh parameterization, UV-unwrapping, slicing, and packing the 3D mesh into a 2D texture domain.

[0141] (3) Subdivision deformation

[0142] The subdivision and deformation modules are optional modules that can improve the quality of the mesh reconstructed at the decoding end. This module can be omitted when the quality of the basic mesh can meet the application requirements.

[0143] The basic idea of ​​subdivision and deformation module is as follows Fig.11 As shown in , the same concept is applied to the input 3D mesh to generate displacement vector information. Fig.11 In the module, the input 2D curve (represented by a 2D polyline), called the "original" curve, is first downsampled to generate a basic curve / polyline, called the "simplified" curve. The subdivision scheme is then applied to the simplified polyline to generate the "subdivided" curve. The subdivided polyline is then deformed to obtain a better approximation of the original curve. That is, a geometric displacement vector is calculated for each vertex of the subdivided mesh so that the shape of the subdivided curve is as close to the shape of the original curve as possible. These geometric displacement vectors are the geometric displacement vector information (vertex displacement) output by this module. The same deformation process is also applied to the attribute information corresponding to the vertex to obtain the corresponding attribute displacement vector.

[0144] (4) Compressing the base mesh

[0145] The basic mesh compression module compresses the basic mesh information output by the subdivision deformation module. There are two main modes of basic mesh compression, namely intra-frame mode and inter-frame mode. Fig.12 As shown in the figure. In intra mode, the input of the basic mesh compression module is a 3D mesh, which contains geometric coordinates, connection relationships, and attribute information associated with vertices. In inter mode, the input of the basic mesh compression module is motion vectors and their identifiers, as well as possible intra-frame sub-meshes. After encoding, the encoded mesh needs to be reconstructed so that it can be provided to subsequent modules for processing.

[0146] Among them, the grid coding in the intra-frame mode can use the existing grid encoder to encode the input basic grid, such as the Draco encoder, etc. The grid encoder type can be encoded through auxiliary information and passed to the decoding end;

[0147] In inter-frame mode, motion vectors and identifiers need to be encoded. The motion vector identifier is an array consisting of numbers greater than or equal to 0. The array size is the same as the number of vertices of the base grid of the reference frame corresponding to the temporal motion vector identifier. The array can be directly encoded using an existing entropy coding algorithm, such as Context Adaptive Binary Arithmetic Coding (CABAC).

[0148] (5) Reconstruct the base mesh

[0149] After the base grid is encoded, the base grid is reconstructed. For a grid encoded in intra-frame mode, the encoded grid is decoded to obtain a reconstructed grid.

[0150] For the basic grid information encoded in the inter-frame mode, the inter-frame sub-grid is reconstructed according to the motion vector and the identifier. If there is an intra-frame sub-grid in the inter-frame mode, the reconstructed inter-frame sub-grid is merged with the intra-frame sub-grid to obtain the reconstructed basic grid.

[0151] (6) Encoding vertex displacement

[0152] To encode the vertex displacement, first adjust the order of vertex displacement according to the reconstructed basic mesh; then perform wavelet transform and quantization on the displacement to obtain the quantized wavelet transform coefficients; then arrange the quantized wavelet transform coefficients into a two-dimensional image, and finally form a YUV video, which is encoded with the help of a video encoder to obtain the displacement code stream. Encoding the vertex displacement includes the following process:

[0153] a. Adjust the displacement order

[0154] The order of vertex displacements is adjusted according to the reconstructed base mesh. The reconstructed base mesh is first subdivided, and then the order of displacements is adjusted to be the same as the subdivided base mesh vertex order. Optionally, the coordinate system of the vertex displacements can be converted from a Cartesian coordinate system to a local coordinate system.

[0155] b. Wavelet transform

[0156] A transformation can be applied to the displacement vector to reduce the correlation between its data. An optional transformation is a linear wavelet transform, and its prediction process is defined as follows:

[0157]

[0158] Where v is the newly inserted midpoint on the edge (v1, v2), Signal(v), Signal(v1) and Signal(v2) are the displacement vectors corresponding to vertices v, v1 and v2 respectively. The displacement vector of vertex v is predicted and then updated. The update process is defined as follows:

[0159]

[0160] Among them, v * is the set of all adjacent vertices of vertex v. The transformed displacement vector is called wavelet coefficient.

[0161] c. Coefficient quantization

[0162] The transformed displacement vector, i.e., the wavelet coefficient, can be quantified in a variety of ways. One quantization formula is as follows:

[0163] disp[v].d[k]=floor(disp[v].d[k]*scale[k])

[0164]

[0165] Where disp[v] represents the transformed value of the displacement vector of the vth vertex, d[k] represents the kth value of the displacement vector, and floor represents rounding down. scale[k] is an intermediate parameter. bitDepthPosition represents the bit depth of the geometric position of the current mesh vertex, and qp[k] represents the quantization parameter of the kth coefficient. As mentioned above, after the displacement vector is converted to the coordinate system, its normal component has a more significant effect on the quality than the tangential component, so a larger quantization parameter can be used for the tangential component.

[0166] At the same time, according to the characteristics of wavelet transform, different quantization parameters can be used for the newly generated vertices and the original vertices. That is, for the subdivided vertices, the quantization parameter update formula is as follows:

[0167] scale[k]=scale[k]*lodScale[k]

[0168] Among them, lodScale[k] represents the coefficient of the quantization parameter of the current subdivision level.

[0169] d. Two-dimensional arrangement

[0170] After transformation and quantization, the displacement data can be arranged on a two-dimensional image as follows:

[0171] Traverse the wavelet coefficients in order from high frequency to low frequency;

[0172] Select a most appropriate block size for the currently encoded grid sequence and indicate the selected block size through a block size flag;

[0173] For each block of the video frame, the wavelet coefficients are arranged in reverse Morton order, such as Fig.13 As shown;

[0174] For blocks, the reverse raster scanning order is used to select blocks for the arrangement of wavelet coefficients, such as Fig.13 shown.

[0175] It should be noted that the arrangement of video blocks is not limited here, and other arrangement schemes can also be used, such as zigzag order, raster order, etc. The encoder can explicitly specify the corresponding arrangement scheme in the bitstream. In order to ensure that the resolution of each frame of the generated YUV video is the same and the number of pixels can be arranged in a rectangle, the video can be filled.

[0176] e. Video compression

[0177] After arranging the wavelet coefficients on a two-dimensional image, they can be directly encoded using a video encoder to obtain a displacement code stream.

[0178] (7) Reconstruction of deformed mesh

[0179] When encoding the displacement vector, it is necessary to obtain the reconstructed displacement vector value. First, the block size selected in the arrangement process is obtained according to the block size mark, and then the quantized wavelet transform coefficients can be extracted from the reconstructed two-dimensional image according to the arrangement order; the quantized wavelet transform coefficients are dequantized and inversely transformed to obtain the displacement vector consistent with the decoding end. After obtaining the reconstructed geometric displacement vector, the subdivided reconstructed basic grid is obtained according to the corresponding geometric displacement vector to obtain the reconstructed subdivided deformed grid, which is passed to the attribute graph conversion module for attribute graph conversion.

[0180] (8) Attribute Graph Conversion

[0181] The attribute map is transformed according to the input original mesh, the input original attribute map and the reconstructed deformed mesh, such as Fig.14 shown.

[0182] The steps of property graph conversion are as follows:

[0183] a. Calculate the texture coordinates of each pixel on the attribute map to be generated, such as the texture coordinates corresponding to pixel A(i,j) is P(u,v);

[0184] b. Determine whether the texture coordinate is within a certain triangle face after the parameterization of the subdivided deformed mesh;

[0185] c. If the texture coordinate does not belong to any triangle, the pixel is marked as an empty pixel, which can be filled with a filling algorithm later;

[0186] d. If the texture coordinate belongs to a triangle, then:

[0187] Mark the pixel as filled;

[0188] Calculate the center of gravity coordinates of the texture in the current triangle surface according to the texture coordinates;

[0189] According to the barycentric coordinates and the corresponding triangular face, the two-dimensional texture coordinates are mapped to three-dimensional geometric coordinates, that is, mapped to the points on the subdivided deformation grid corresponding to the texture coordinates, such as Fig.14 As shown in M(x,y,z);

[0190] Find the point closest to the three-dimensional coordinate on the input original grid, such as Fig.14 Medium ′ (x, y, z);

[0191] The three-dimensional coordinates are calculated according to the triangular face where they are located, and their center coordinates are mapped to two dimensions to calculate their texture coordinates, i.e. P ′ (u ′ ,v ′ );

[0192] The texture coordinates are used to sample the original attribute map input to obtain the value A at the corresponding pixel position. ′ (i ′ ,j ′ );

[0193] Assign this value to the corresponding pixel A(i,j) on the attribute map to be generated.

[0194] (9) Attribute Graph Compression

[0195] After obtaining the converted attribute map, the empty pixels therein can be filled using existing filling algorithms (such as the Push-Pull algorithm). Then, existing video encoders such as H.264 / AVC, H.265 / HEVC, and H.266 / VVC can be used to encode the empty pixels to obtain the bitstream of the output attribute map. In addition, color space conversion and chroma subsampling can be selectively applied to enable video encoding to obtain better rate-distortion performance, such as color space conversion from RGB 444 to YUV420. In the case of multiple attribute maps, the texture coordinates corresponding to each attribute map are identified by auxiliary information during encoding.

[0196] (10) Auxiliary Information Coding

[0197] During the encoding process, there are some alternative schemes in each module, such as the type of grid encoder, the type of video encoder, the grid subdivision scheme, the spatial displacement vector transformation scheme, etc., and different schemes are allowed to be used in the encoding framework. Therefore, the selected scheme can be passed to the decoding end to guide the correct decoding.

[0198] Auxiliary information can be used to indicate the type of static grid encoder, the type of video encoder, grid subdivision scheme, grid iteration number, geometric displacement vector transformation scheme, coefficient arrangement scheme, block size mark, or color conversion scheme, etc. For the grid part encoded using the inter-frame coding mode, it can also include the corresponding reference frame list, etc.

[0199] After all modules are encoded, the basic grid bitstream, texture coordinate bitstream, displacement vector video bitstream, attribute graph video bitstream and auxiliary information bitstream are mixed to obtain the encoded bitstream finally output by the encoding end.

[0200] like Fig.15 As shown, the grid decoding method performed by the decoding end includes the following process:

[0201] (1) Auxiliary Information Decoding

[0202] The auxiliary information is used to indicate the type of static grid encoder, the type of video encoder, grid subdivision scheme, grid iteration number, geometric displacement vector transformation scheme, coefficient arrangement scheme, displacement truncation mark, or color conversion scheme, etc. For the grid part encoded using the inter-frame coding mode, it also includes the corresponding reference frame list, etc. The auxiliary information can be used to guide the decoding end to perform correct decoding, and the information is represented by the corresponding syntax parameters after decoding.

[0203] (2) Basic grid decoding

[0204] The decoding of the base grid can be divided into intra-frame mode and inter-frame mode. The base grid decoding is only responsible for decoding the information representing the base grid. For the intra-frame mode, the output of the base grid decoding is a three-dimensional grid, which contains geometric information and connection relationships, as well as possible attribute information and texture coordinate information. For the inter-frame mode, the base grid decoding is responsible for decoding the motion vector and identifier as well as the possible intra-frame sub-grid, and the output information is the motion vector, identifier and intra-frame sub-grid.

[0205] The decoding of the basic grid information in the intra-frame mode is also the decoding of the basic grid. The basic grid can be decoded using the corresponding grid decoder through the static grid decoder type indicated by the auxiliary information;

[0206] In inter-frame mode, the motion vector, identifier and possible intra-frame sub-grid are decoded. For the intra-frame sub-grid, the grid decoder indicated by the auxiliary information can be used for decoding. The motion vector identifier is obtained by entropy decoding, and the motion vector is obtained by deprediction and dequantization after entropy decoding.

[0207] (3) Segmentation

[0208] The subdivision operation is the same as that on the encoding side, and auxiliary information is used to indicate the number of subdivision iterations of different areas of the current basic grid.

[0209] (4) Displacement vector video decoding

[0210] The displacement vector video code stream is decoded according to the video decoder type indicated in the auxiliary information, including decoding of geometric displacement vector video and possible displacement vector video with different attributes.

[0211] (5) Displacement vector decoding

[0212] The block size selected by the encoder is obtained according to the block size tag carried in the bitstream, and the quantized wavelet transform coefficients can be extracted from the decoded displacement vector video according to the arrangement order of the encoder. Then, the quantized wavelet transform coefficients are dequantized, inversely transformed, and other operations are performed on them to restore the corresponding displacement vector.

[0213] (6) Reconstruct the deformed mesh

[0214] After the base mesh and displacement vector decoding and reconstruction are completed, the deformed mesh is reconstructed based on these two parts, and the corresponding displacement vector is added to each vertex of the subdivided mesh. The calculation formula is as follows:

[0215] deformedmesh[i].v[k]=subdivmesh[i].v[k]+displacement[k]

[0216] Among them, subdivmesh[i].v[k] is the geometric coordinates of the kth vertex after the base mesh is subdivided in the current frame (index is i), displacement[k] is the spatial displacement vector corresponding to the kth vertex, and deformedmesh[i].v[k] is the geometric coordinates of the kth vertex after subdivision and deformation in the current frame.

[0217] In addition, the attribute displacement vector is applied to the corresponding attribute value in the same way. Taking texture coordinates as an example, the calculation formula is as follows:

[0218] deformedmesh[i].vt[k]=subdivmesh[i].vt[k]+attdisplacement[k]

[0219] Among them, subdivmesh[i].vt[k] represents the k-th texture coordinate after the base mesh is subdivided in the current frame, attdisplacement[k] represents the k-th value of the attribute displacement vector, and deformedmesh[i].vt[k] represents the value of the k-th texture coordinate after subdivision and deformation in the current frame.

[0220] (7) Attribute Graph Decoding

[0221] The attribute graph decoder is responsible for decoding the attribute graph bitstream. The attribute graph is decoded using the video decoder indicated in the auxiliary information. An optional color space conversion is performed on it to obtain the attribute graph that matches the input attribute graph of the encoder. Figure 1 The image format is consistent, and the attribute map of the final decoded output is obtained. For multiple attribute map bitstreams, after decoding, each attribute map is matched with the corresponding texture coordinates according to the identifier.

[0222] After each module is processed, the deformed mesh and the corresponding attribute map reconstructed by the decoder are finally obtained. Subsequent applications use the reconstructed deformed mesh and attribute map as input for processing.

[0223] Embodiment 2:

[0224] In this embodiment, the syntax structure designed for the V-DMC codec framework is used as an example to illustrate the syntax structure of the code stream corresponding to the three-dimensional grid:

[0225] Table 1: Atlas sequence parameter set vdmc extension RBSP syntax

[0226]

[0227]

[0228] Wherein, Atlas sequence parameter set vdmc extension RBSP syntax refers to Atlas sequence parameter set vdmc extension network raw byte sequence payload (RBSP) syntax. Descriptor refers to a descriptor symbol.

[0229] Wherein, asps_vdmc_ext_subdivision_method indicates the subdivision method of the current mesh sequence. Table 2 describes the corresponding relationship between the subdivision method and asps_vdmc_ext_subdivision_method.

[0230] Table 2

[0231] asps_vdmc_ext_subdivision_method Name of subdivision method 0 NONE 1 MIDPOINT

[0232] Among them, Name of subdivision method refers to the name of the subdivision method.

[0233] Where asps_vdmc_ext_subdivision_iteration_count indicates the number of mesh subdivision iterations. If it does not exist, the asps_vdmc_ext_subdivision_iteration_count value is considered to be 0.

[0234] Wherein, asps_vdmc_ext_displacement_coordinate_system indicates the coordinate system identifier of the current grid sequence. Table 3 describes the correspondence between the supported coordinate systems and asps_vdmc_ext_displacement_coordinate_system.

[0235] Table 3

[0236] asps_vdmc_ext_displacement_coordinate_system Name of displacement coordinate system 0 CANNONICAL 1 LOCAL

[0237] Among them, Name of displacement coordinate system refers to the name of the displacement coordinate system.

[0238] Wherein, asps_vdmc_ext_transform_method indicates the wavelet transform identifier applied to the displacement. Table 4 describes the correspondence between the supported wavelet transform methods and asps_vdmc_ext_transform_method.

[0239] Table 4

[0240]

[0241]

[0242] Among them, Name of transform method refers to the name of the transformation method.

[0243] asps_vdmc_ext_num_attribute_video indicates the number of attributes sent through the video sub-bitstream.

[0244] asps_vdmc_ext_attribute_type_id[i] indicates the attribute type of the attribute video data unit with index i.

[0245] asps_vdmc_ext_attribute_frame_width[i] indicates the atlas frame width of the attribute video data unit with index i.

[0246] asps_vdmc_ext_attribute_frame_height[i] indicates the atlas frame height of the attribute video data unit with index i.

[0247] asps_vdmc_ext_attribute_transform_method[i] applies to the transform identifier of the attribute with index i in the attribute video data unit.

[0248] Table 5 describes the correspondence between the supported transformation methods and asps_vdmc_ext_attribute_transform_method.

[0249] Table 5

[0250] asps_vdmc_ext_attribute_transform_method Name of transform method 0 NONE 1 LINEAR_LIFTING

[0251] Among them, Name of transform method refers to the name of the transformation method.

[0252] asps_vdmc_ext_packing_method equal to 0 indicates that the displacement component samples are arranged in ascending order, and asps_vdmc_ext_packing_method equal to 1 indicates that the displacement component samples are arranged in descending order.

[0253] asps_vdmc_ext_1d_displacement_flag equal to 1 indicates that only the normal component of the displacement is present in the displacement vector video. The other two components are inferred to be 0. asps_vdmc_ext_1D_displacement_flag equal to 0 indicates that all 3 components of the displacement are present in the displacement vector video.

[0254] asps_vdmc_ext_packing_block_size indicates the identifier of the block size selected when the current grid sequence is shifted. Table 6 describes the correspondence between the available block sizes and asps_vdmc_ext_packing_block_size.

[0255] Table 6

[0256] asps_vdmc_ext_packing_block_size block size 0 16×16 1 32×32 2 64×64 3 SEI or user defined

[0257] SEI refers to Supplemental Enhancement Information. SEI is the part of VDMC used to store user-defined information. SEI or user defined refers to SEI or user-defined.

[0258] Table 7: Atlas frame parameter set vdmc extension RBSP syntax

[0259]

[0260] Among them, Atlas frame parameter set vdmc extension RBSP syntax refers to Atlas frame parameter set vdmc extended RBSP syntax.

[0261] Among them, afps_vdmc_ext_overriden_flag is equal to 1, indicating that the following parameters exist in the atlas frame parameter set extension:

[0262] afps_vdmc_ext_subdivision_enable_flag;

[0263] afps_vdmc_ext_displacement_coordinate_system_enable_flag;

[0264] afps_vdmc_ext_transform_method_enable_flag;

[0265] afps_vdmc_ext_transform_parameters_enable_flag;

[0266] afps_vdmc_ext_packing_block_size_enable_flag.

[0267] Among them, afps_vdmc_ext_subdivision_enable_flag is equal to 1, which indicates that the following parameters exist in the atlas frame parameter set extension: afps_vdmc_ext_subdivision_method and afps_vdmc_ext_subdivision_iteration_count. When afps_vdmc_ext_subdivision_enable_flag does not exist, its value is inferred to be 0.

[0268] Among them, afps_vdmc_ext_displacement_coordinate_system_enable_flag is equal to 1, indicating that afps_vdmc_ext_displacement_coordinate_system exists in the atlas frame parameter set extension.

[0269] When afps_vdmc_ext_displacement_coordinate_system_enable_flag is not present, its value is inferred to be 0.

[0270] Among them, afps_vdmc_ext_transform_method_enable_flag is equal to 1 to indicate that afps_vdmc_ext_transform_method exists in the atlas frame parameter set extension. When afps_vdmc_ext_transform_method_enable_flag does not exist, its value is inferred to be equal to 0.

[0271] Among them, afps_vdmc_ext_transform_parameters_enable_flag is equal to 1, indicating that vdmc_lifting_transform_parameters exist in the atlas frame parameter set extension.

[0272] When afps_vdmc_ext_transform_parameters_enable_flag is not present, its value is inferred to be equal to 0.

[0273] Among them, afps_vdmc_ext_packing_block_size_enable_flag is equal to 1 to indicate the presence of afps_vdmc_ext_packing_block_size in the atlas frame parameter set extension. When afps_vdmc_ext_packing_block_size_enable_flag does not exist, its value is inferred to be equal to 0.

[0274] Among them, afps_vdmc_ext_subdivision_method indicates the subdivision method of the current frame.

[0275] When afps_vdmc_ext_subdivision_method does not exist, afps_vdmc_ext_subdivision_method is equal to asps_vdmc_ext_subdivision_method.

[0276] Among them, afps_vdmc_ext_subdivision_iteration_count indicates the number of subdivision iterations.

[0277] When afps_vdmc_ext_subdivision_method is equal to 0 and afps_vdmc_ext_subdivision_iteration_count is not present, afps_vdmc_ext_subdivision_iteration_count is equal to 0.

[0278] When afps_vdmc_ext_subdivision_method is not 0 and afps_vdmc_ext_subdivision_iteration_count does not exist, afps_vdmc_ext_subdivision_iteration_count is equal to asps_vdmc_ext_subdivision_iteration_count.

[0279] Wherein, afps_vdmc_ext_displacement_coordinate_system indicates the coordinate system identifier of the current frame grid displacement. When afps_vdmc_ext_displacement_coordinate_system does not exist, afps_vdmc_ext_displacement_coordinate_system is equal to asps_vdmc_ext_displacement_coordinate_system.

[0280] Wherein, afps_vdmc_ext_transform_method indicates an identifier applied to displacement transform. When afps_vdmc_ext_transform_method does not exist, afps_vdmc_ext_transform_method is equal to asps_vdmc_ext_transform_method.

[0281] Wherein, afps_vdmc_ext_packing_block_size indicates the identifier of the block size used by the current frame. When afps_vdmc_ext_packing_block_size does not exist, afps_vdmc_ext_packing_block_size is equal to asps_vdmc_ext_packing_block_size.

[0282] The embodiments of the present application can improve the flexibility of video arrangement of displacement information.

[0283] It should be noted that the grid coding method provided in the embodiment of the present application can be executed by a grid coding device, or a control module in the grid coding device for executing the grid coding method. In the embodiment of the present application, the grid coding device provided in the embodiment of the present application is described by taking the grid coding method executed by the grid coding device as an example.

[0284] See also Fig.16 , Fig.16 is a structural diagram of a grid coding device provided in an embodiment of the present application, such as Fig.16 As shown, the grid coding device 300 includes:

[0285] An acquisition module 301 is used to acquire a basic grid corresponding to a three-dimensional grid, and perform subdivision processing based on the basic grid to obtain a subdivided grid;

[0286] A determination module 302 is used to perform a deformation operation based on the subdivided mesh to obtain a deformed mesh, and determine displacement information corresponding to the subdivided mesh, wherein the displacement information is used to represent the displacement of the vertices of the subdivided mesh relative to the deformed mesh;

[0287] An arrangement module 303 is used to select a target image block parameter of a video frame corresponding to the three-dimensional grid from a plurality of preset image block parameters, and arrange the image blocks into which the displacement information is divided according to the target image block parameter into the video frame;

[0288] The encoding module 304 is used to encode the arranged video frames to obtain a first encoding result.

[0289] Optionally, the image block parameters include image block size.

[0290] Optionally, the three-dimensional grid includes a first grid and a second grid, a first video frame corresponding to the first grid and a second video frame corresponding to the second grid are different video frames, the target image block parameters include image block parameters of the first video frame and image block parameters of the second video frame, and the arrangement module is specifically used to:

[0291] Selecting an image block parameter of the first video frame from a plurality of preset image block parameters, and arranging the displacement information corresponding to the first grid into the first video frame according to the image blocks divided by the image block parameter of the first video frame;

[0292] An image block parameter of the second video frame is selected from the plurality of preset image block parameters, and the displacement information corresponding to the second grid is arranged in the second video frame according to the image blocks divided according to the image block parameters of the second video frame.

[0293] Optionally, the code stream corresponding to the three-dimensional grid includes the first encoding result and the second encoding result, and the second encoding result is used to indicate the target image block parameters of the video frame corresponding to the three-dimensional grid.

[0294] Optionally, the second encoding result includes an encoding result of the first flag bit;

[0295] When the first flag bit is a first preset value, the second encoding result also includes an encoding result of a first parameter corresponding to each of the video frames, and the second encoding result is used to indicate that the image block parameters of each of the video frames are image block parameters represented by the corresponding first parameters.

[0296] Optionally, the second encoding result also includes an encoding result of a second parameter;

[0297] When the first flag bit is a second preset value, the second encoding result is used to indicate that the image block parameters of each of the video frames are image block parameters represented by the second parameters.

[0298] The grid coding device in the embodiment of the present application can be a device, a device or electronic device with an operating system, or a component, an integrated circuit, or a chip in a terminal. The device or electronic device can be a mobile terminal or a non-mobile terminal. Exemplarily, the mobile terminal can include but is not limited to the types of terminals listed above, and the non-mobile terminal can be a server, a network attached storage (Network Attached Storage, NAS), a personal computer (personalcomputer, PC), a television (television, TV), a teller machine or a self-service machine, etc., which is not specifically limited in the embodiment of the present application.

[0299] The grid coding device provided in the embodiment of the present application can achieve Figure 5 The various processes implemented by the method embodiment and achieving the same technical effect are not described here to avoid repetition.

[0300] It should be noted that the grid decoding method provided in the embodiment of the present application can be executed by a grid decoding device, or a control module in the grid decoding device for executing the grid decoding method. In the embodiment of the present application, the grid decoding device provided in the embodiment of the present application is described by taking the grid decoding method executed by the grid decoding device as an example.

[0301] See also Fig.17 , Fig.17 is a structural diagram of a grid decoding device provided in an embodiment of the present application, such as Fig.17 As shown, the grid decoding device 400 includes:

[0302] A first decoding module 401 is used to decode the code stream corresponding to the three-dimensional grid to obtain a first decoding result, a second decoding result and a third decoding result, wherein the first decoding result is used to represent the video frame corresponding to the three-dimensional grid, the second decoding result is used to represent the target image block parameters of the video frame corresponding to the three-dimensional grid, and the third decoding result is used to represent the basic grid;

[0303] A processing module 402 is used to perform subdivision processing based on the third decoding result to obtain a subdivided grid;

[0304] A second decoding module 403 is used to perform displacement decoding on the first decoding result based on the second decoding result to obtain displacement information corresponding to the subdivided mesh, where the displacement information is used to represent the displacement of the vertices of the subdivided mesh relative to the reconstructed mesh;

[0305] The reconstruction module 404 is used to perform reconstruction processing based on the displacement information and the subdivided grid to obtain the reconstructed grid.

[0306] Optionally, the image block parameters include image block size.

[0307] Optionally, target image block parameters of the video frame corresponding to the three-dimensional grid are obtained by selecting from a plurality of preset image block parameters.

[0308] Optionally, the three-dimensional grid includes a first grid and a second grid, a first video frame corresponding to the first grid and a second video frame corresponding to the second grid are different video frames; and the target image block parameters include image block parameters of the first video frame and image block parameters of the second video frame.

[0309] Optionally, the image block parameters of the first video frame are selected from a plurality of preset image block parameters, and the image block parameters of the second video frame are selected from the plurality of preset image block parameters.

[0310] Optionally, the second decoding result includes a first flag bit;

[0311] When the first flag bit is a first preset value, the second decoding result also includes a first parameter corresponding to each of the video frames, and the second decoding result is used to indicate that the image block parameters of each of the video frames are image block parameters represented by the corresponding first parameters.

[0312] Optionally, the second decoding result also includes a second parameter;

[0313] When the first flag bit is a second preset value, the second decoding result is used to indicate that the image block parameters of each of the video frames are image block parameters represented by the second parameters.

[0314] The grid decoding device in the embodiment of the present application can be a device, a device or electronic device with an operating system, or a component, an integrated circuit, or a chip in a terminal. The device or electronic device can be a mobile terminal or a non-mobile terminal. Exemplarily, the mobile terminal can include but is not limited to the types of terminals listed above, and the non-mobile terminal can be a server, a network attached storage (Network Attached Storage, NAS), a personal computer (personalcomputer, PC), a television (television, TV), a teller machine or a self-service machine, etc., which is not specifically limited in the embodiment of the present application.

[0315] The grid decoding device provided in the embodiment of the present application can achieve Figure 8 The various processes implemented by the method embodiment and achieving the same technical effect are not described here to avoid repetition.

[0316] Alternatively, if Fig.18 As shown, the embodiment of the present application further provides a communication device 500, including a processor 501 and a memory 502, and the memory 502 stores a program or instruction that can be run on the processor 501. For example, when the communication device 500 is an encoding end device, the program or instruction is executed by the processor 501 to implement the various steps of the above-mentioned grid encoding method embodiment, and can achieve the same technical effect. When the communication device 500 is a decoding end device, the program or instruction is executed by the processor 501 to implement the various steps of the above-mentioned grid decoding method embodiment, and can achieve the same technical effect. To avoid repetition, it will not be repeated here.

[0317] An embodiment of the present application also provides a terminal, including a processor and a communication interface, wherein the processor is used to: obtain a basic grid corresponding to a three-dimensional grid, perform subdivision processing based on the basic grid to obtain a subdivided grid; perform a deformation operation based on the subdivided grid to obtain a deformed grid, and determine displacement information corresponding to the subdivided grid, wherein the displacement information is used to characterize the displacement of vertices of the subdivided grid relative to the deformed grid; select a target image block parameter of a video frame corresponding to the three-dimensional grid from a plurality of preset image block parameters, and arrange the image blocks divided by the target image block parameter into the video frame; and perform encoding processing on the arranged video frame to obtain a first encoding result. Or, the processor is used to: decode the code stream corresponding to the three-dimensional mesh to obtain a first decoding result, a second decoding result and a third decoding result, wherein the first decoding result is used to characterize the video frame corresponding to the three-dimensional mesh, the second decoding result is used to characterize the target image block parameters of the video frame corresponding to the three-dimensional mesh, and the third decoding result is used to characterize the basic mesh; perform subdivision processing based on the third decoding result to obtain a subdivided mesh; perform displacement decoding on the first decoding result based on the second decoding result to obtain displacement information corresponding to the subdivided mesh, and the displacement information is used to characterize the displacement of the vertices of the subdivided mesh relative to the reconstructed mesh; perform reconstruction processing based on the displacement information and the subdivided mesh to obtain the reconstructed mesh. This terminal embodiment corresponds to the above-mentioned mesh encoding method or mesh decoding method embodiment, and each implementation process and implementation method of the above-mentioned mesh encoding method or mesh decoding method embodiment can be applied to this terminal embodiment and can achieve the same technical effect. Specifically, Fig.19 A schematic diagram of the hardware structure of a terminal for implementing an embodiment of the present application.

[0318] The terminal 600 includes but is not limited to: a radio frequency unit 601, a network module 602, an audio output unit 603, an input unit 604, a sensor 605, a display unit 606, a user input unit 607, an interface unit 608, a memory 609 and at least some of the components of a processor 610.

[0319] Those skilled in the art will appreciate that the terminal 600 may also include a power source (such as a battery) for supplying power to various components, and the power source may be logically connected to the processor 610 through a power management system, thereby implementing functions such as managing charging, discharging, and power consumption management through the power management system. Fig.19 The terminal structure shown in the figure does not constitute a limitation on the terminal. The terminal may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently, which will not be described in detail here.

[0320] It should be understood that in the embodiment of the present application, the input unit 604 may include a graphics processing unit (GPU) 6041 and a microphone 6042, and the GPU 6041 processes the image data of the static picture or video obtained by the image capture device (such as a camera) in the video capture mode or the image capture mode. The display unit 606 may include a display panel 6061, and the display panel 6061 may be configured in the form of a liquid crystal display, an organic light emitting diode, etc. The user input unit 607 includes a touch panel 6071 and at least one of other input devices 6072. The touch panel 6071 is also called a touch screen. The touch panel 6071 may include two parts: a touch detection device and a touch controller. Other input devices 6072 may include, but are not limited to, a physical keyboard, function keys (such as volume control keys, switch keys, etc.), a trackball, a mouse, and a joystick, which will not be repeated here.

[0321] In the embodiment of the present application, after receiving downlink data from the network side device, the RF unit 601 can transmit the data to the processor 610 for processing; in addition, the RF unit 601 can send uplink data to the network side device. Generally, the RF unit 601 includes but is not limited to an antenna, an amplifier, a transceiver, a coupler, a low noise amplifier, a duplexer, etc.

[0322] The memory 609 can be used to store software programs or instructions and various data. The memory 609 may mainly include a first storage area for storing programs or instructions and a second storage area for storing data, wherein the first storage area may store an operating system, an application program or instruction required for at least one function (such as a sound playback function, an image playback function, etc.), etc. In addition, the memory 609 may include a volatile memory or a non-volatile memory, or the memory 609 may include both volatile and non-volatile memories. Among them, the non-volatile memory may be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory may be a random access memory (RAM), a static random access memory (SRAM), a dynamic random access memory (DRAM), a synchronous dynamic random access memory (SDRAM), a double data rate synchronous dynamic random access memory (DDRSDRAM), an enhanced synchronous dynamic random access memory (ESDRAM), a synchronous link dynamic random access memory (SLDRAM) and a direct memory bus random access memory (DRRAM). The memory 609 in the embodiment of the present application includes but is not limited to these and any other suitable types of memory.

[0323] The processor 610 may include one or more processing units; optionally, the processor 610 integrates an application processor and a modem processor, wherein the application processor mainly processes operations related to an operating system, a user interface, and application programs, and the modem processor mainly processes wireless communication signals, such as a baseband processor. It is understandable that the modem processor may not be integrated into the processor 610.

[0324] Wherein, when the terminal is a coding end device:

[0325] The processor 610 is configured to:

[0326] Obtaining a basic grid corresponding to the three-dimensional grid, and performing subdivision processing based on the basic grid to obtain a subdivided grid;

[0327] Performing a deformation operation based on the subdivided mesh to obtain a deformed mesh, and determining displacement information corresponding to the subdivided mesh, wherein the displacement information is used to characterize the displacement of vertices of the subdivided mesh relative to the deformed mesh;

[0328] Selecting a target image block parameter of a video frame corresponding to the three-dimensional grid from a plurality of preset image block parameters, and arranging the image blocks divided by the displacement information according to the target image block parameter into the video frame;

[0329] The arranged video frames are encoded to obtain a first encoding result.

[0330] Optionally, the image block parameters include image block size.

[0331] Optionally, the three-dimensional grid includes a first grid and a second grid, a first video frame corresponding to the first grid and a second video frame corresponding to the second grid are different video frames, the target image block parameters include image block parameters of the first video frame and image block parameters of the second video frame, and the processor 610 is specifically configured to:

[0332] Selecting an image block parameter of the first video frame from a plurality of preset image block parameters, and arranging the displacement information corresponding to the first grid into the first video frame according to the image blocks divided by the image block parameter of the first video frame;

[0333] An image block parameter of the second video frame is selected from the plurality of preset image block parameters, and the displacement information corresponding to the second grid is arranged in the second video frame according to the image blocks divided according to the image block parameters of the second video frame.

[0334] Optionally, the code stream corresponding to the three-dimensional grid includes the first encoding result and the second encoding result, and the second encoding result is used to indicate the target image block parameters of the video frame corresponding to the three-dimensional grid.

[0335] Optionally, the second encoding result includes an encoding result of the first flag bit;

[0336] When the first flag bit is a first preset value, the second encoding result also includes an encoding result of a first parameter corresponding to each of the video frames, and the second encoding result is used to indicate that the image block parameters of each of the video frames are image block parameters represented by the corresponding first parameters.

[0337] Optionally, the second encoding result also includes an encoding result of a second parameter;

[0338] When the first flag bit is a second preset value, the second encoding result is used to indicate that the image block parameters of each of the video frames are image block parameters represented by the second parameters.

[0339] Wherein, when the terminal is a decoding end device:

[0340] The processor 610 is configured to:

[0341] Decoding a code stream corresponding to the three-dimensional grid to obtain a first decoding result, a second decoding result, and a third decoding result, wherein the first decoding result is used to characterize a video frame corresponding to the three-dimensional grid, the second decoding result is used to characterize a target image block parameter of the video frame corresponding to the three-dimensional grid, and the third decoding result is used to characterize a basic grid;

[0342] Performing subdivision processing based on the third decoding result to obtain a subdivided grid;

[0343] Performing displacement decoding on the first decoding result based on the second decoding result to obtain displacement information corresponding to the subdivided mesh, wherein the displacement information is used to represent the displacement of vertices of the subdivided mesh relative to the reconstructed mesh;

[0344] Reconstruction processing is performed based on the displacement information and the subdivided grid to obtain the reconstructed grid.

[0345] Optionally, the image block parameters include image block size.

[0346] Optionally, target image block parameters of the video frame corresponding to the three-dimensional grid are obtained by selecting from a plurality of preset image block parameters.

[0347] Optionally, the three-dimensional grid includes a first grid and a second grid, a first video frame corresponding to the first grid and a second video frame corresponding to the second grid are different video frames; and the target image block parameters include image block parameters of the first video frame and image block parameters of the second video frame.

[0348] Optionally, the image block parameters of the first video frame are selected from a plurality of preset image block parameters, and the image block parameters of the second video frame are selected from the plurality of preset image block parameters.

[0349] Optionally, the second decoding result includes a first flag bit;

[0350] When the first flag bit is a first preset value, the second decoding result also includes a first parameter corresponding to each of the video frames, and the second decoding result is used to indicate that the image block parameters of each of the video frames are image block parameters represented by the corresponding first parameters.

[0351] Optionally, the second decoding result also includes a second parameter;

[0352] When the first flag bit is a second preset value, the second decoding result is used to indicate that the image block parameters of each of the video frames are image block parameters represented by the second parameters.

[0353] Specifically, the terminal of the embodiment of the present application also includes: instructions or programs stored in the memory 609 and executable on the processor 610, and the processor 610 calls the instructions or programs in the memory 609 to execute Fig.16 or Fig.17 The methods executed by the modules shown achieve the same technical effects, and therefore will not be described here in detail to avoid repetition.

[0354] An embodiment of the present application also provides a readable storage medium, on which a program or instruction is stored. When the program or instruction is executed by a processor, the various processes of the above-mentioned grid encoding method embodiment are implemented, or when the program or instruction is executed by a processor, the various processes of the above-mentioned grid decoding method embodiment are implemented, and the same technical effect can be achieved. To avoid repetition, it will not be repeated here.

[0355] The processor is the processor in the terminal described in the above embodiment. The readable storage medium includes a computer readable storage medium, such as a computer read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0356] An embodiment of the present application further provides a chip, which includes a processor and a communication interface, wherein the communication interface is coupled to the processor, and the processor is used to run programs or instructions to implement the various processes of the above-mentioned grid encoding method embodiment, or to implement the various processes of the above-mentioned grid decoding method embodiment, and can achieve the same technical effect. To avoid repetition, it will not be repeated here.

[0357] It should be understood that the chip mentioned in the embodiments of the present application can also be called a system-level chip, a system chip, a chip system or a system-on-chip chip, etc.

[0358] It should be noted that, in this article, the terms "comprise", "include" or any other variant thereof are intended to cover non-exclusive inclusion, so that the process, method, article or device including a series of elements includes not only those elements, but also includes other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise one..." do not exclude the presence of other identical elements in the process, method, article or device including the element. In addition, it should be pointed out that the scope of the methods and devices in the embodiments of the present application is not limited to performing functions in the order shown or discussed, and may also include performing functions in a substantially simultaneous manner or in reverse order according to the functions involved, for example, the described method may be performed in an order different from that described, and various steps may also be added, omitted, or combined. In addition, the features described with reference to certain examples may be combined in other examples.

[0359] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a computer software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), and includes a number of instructions for a terminal (which can be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in each embodiment of the present application.

[0360] The embodiments of the present application are described above in conjunction with the accompanying drawings, but the present application is not limited to the above-mentioned specific implementation methods. The above-mentioned specific implementation methods are merely illustrative and not restrictive. Under the guidance of the present application, ordinary technicians in this field can also make many forms without departing from the purpose of the present application and the scope of protection of the claims, all of which are within the protection of the present application.

Claims

1. A grid coding method, characterized in that: include: Obtaining a basic grid corresponding to the three-dimensional grid, and performing subdivision processing based on the basic grid to obtain a subdivided grid; Performing a deformation operation based on the subdivided mesh to obtain a deformed mesh, and determining displacement information corresponding to the subdivided mesh, wherein the displacement information is used to characterize the displacement of vertices of the subdivided mesh relative to the deformed mesh; Selecting a target image block parameter of a video frame corresponding to the three-dimensional grid from a plurality of preset image block parameters, and arranging the image blocks divided by the displacement information according to the target image block parameter into the video frame; The arranged video frames are encoded to obtain a first encoding result.

2. The method according to claim 1, characterized in that The image block parameters include image block size.

3. The method according to claim 1 or 2, characterized in that: The three-dimensional grid includes a first grid and a second grid, a first video frame corresponding to the first grid and a second video frame corresponding to the second grid are different video frames, the target image block parameters include image block parameters of the first video frame and image block parameters of the second video frame, and the target image block parameters of the video frame corresponding to the three-dimensional grid are selected from a plurality of preset image block parameters, and the image blocks divided according to the target image block parameters are arranged in the video frame, including: Selecting an image block parameter of the first video frame from a plurality of preset image block parameters, and arranging the displacement information corresponding to the first grid into the first video frame according to the image blocks divided by the image block parameter of the first video frame; An image block parameter of the second video frame is selected from the plurality of preset image block parameters, and the displacement information corresponding to the second grid is arranged in the second video frame according to the image blocks divided according to the image block parameters of the second video frame.

4. The method according to any one of claims 1 to 3, characterized in that The code stream corresponding to the three-dimensional grid includes the first encoding result and the second encoding result, and the second encoding result is used to indicate the target image block parameters of the video frame corresponding to the three-dimensional grid.

5. The method according to claim 4, characterized in that The second encoding result includes the encoding result of the first flag bit; When the first flag bit is a first preset value, the second encoding result also includes an encoding result of a first parameter corresponding to each of the video frames, and the second encoding result is used to indicate that the image block parameters of each of the video frames are image block parameters represented by the corresponding first parameters.

6. The method according to claim 5, characterized in that The second encoding result also includes an encoding result of a second parameter; When the first flag bit is a second preset value, the second encoding result is used to indicate that the image block parameters of each of the video frames are image block parameters represented by the second parameters.

7. A grid decoding method, characterized in that: include: Decoding a code stream corresponding to the three-dimensional grid to obtain a first decoding result, a second decoding result, and a third decoding result, wherein the first decoding result is used to characterize a video frame corresponding to the three-dimensional grid, the second decoding result is used to characterize a target image block parameter of the video frame corresponding to the three-dimensional grid, and the third decoding result is used to characterize a basic grid; Performing subdivision processing based on the third decoding result to obtain a subdivided grid; Performing displacement decoding on the first decoding result based on the second decoding result to obtain displacement information corresponding to the subdivided mesh, wherein the displacement information is used to represent the displacement of vertices of the subdivided mesh relative to the reconstructed mesh; Reconstruction processing is performed based on the displacement information and the subdivided grid to obtain the reconstructed grid.

8. The method according to claim 7, characterized in that The image block parameters include image block size.

9. The method according to claim 7 or 8, characterized in that: The target image block parameters of the video frame corresponding to the three-dimensional grid are selected and obtained from a plurality of preset image block parameters.

10. The method according to any one of claims 7 to 9, characterized in that: The three-dimensional grid includes a first grid and a second grid, a first video frame corresponding to the first grid and a second video frame corresponding to the second grid are different video frames; the target image block parameters include image block parameters of the first video frame and image block parameters of the second video frame.

11. The method according to any one of claims 7 to 10, characterized in that: The second decoding result includes a first flag bit; When the first flag bit is a first preset value, the second decoding result also includes a first parameter corresponding to each of the video frames, and the second decoding result is used to indicate that the image block parameters of each of the video frames are image block parameters represented by the corresponding first parameters.

12. The method according to claim 11, characterized in that The second decoding result also includes a second parameter; When the first flag bit is a second preset value, the second decoding result is used to indicate that the image block parameters of each of the video frames are image block parameters represented by the second parameters.

13. A grid coding device, characterized in that: The device comprises: An acquisition module is used to acquire a basic grid corresponding to the three-dimensional grid, and perform subdivision processing based on the basic grid to obtain a subdivided grid; A determination module, configured to perform a deformation operation based on the subdivided mesh to obtain a deformed mesh, and determine displacement information corresponding to the subdivided mesh, wherein the displacement information is used to characterize the displacement of vertices of the subdivided mesh relative to the deformed mesh; An arrangement module, configured to select a target image block parameter of a video frame corresponding to the three-dimensional grid from a plurality of preset image block parameters, and arrange the image blocks divided by the displacement information according to the target image block parameter into the video frame; The encoding module is used to encode the arranged video frames to obtain a first encoding result.

14. A grid decoding device, characterized in that: The device comprises: A first decoding module is used to decode a code stream corresponding to a three-dimensional grid to obtain a first decoding result, a second decoding result, and a third decoding result, wherein the first decoding result is used to characterize a video frame corresponding to the three-dimensional grid, the second decoding result is used to characterize a target image block parameter of the video frame corresponding to the three-dimensional grid, and the third decoding result is used to characterize a basic grid; A processing module, configured to perform subdivision processing based on the third decoding result to obtain a subdivided grid; a second decoding module, configured to perform displacement decoding on the first decoding result based on the second decoding result to obtain displacement information corresponding to the subdivided mesh, wherein the displacement information is used to characterize the displacement of the vertices of the subdivided mesh relative to the reconstructed mesh; A reconstruction module is used to perform reconstruction processing based on the displacement information and the subdivided grid to obtain the reconstructed grid.

15. A terminal, characterized in that: It includes a processor, a memory, and a program or instruction stored in the memory and executable on the processor, wherein the program or instruction, when executed by the processor, implements the steps of the grid encoding method as described in any one of claims 1 to 6; or, the program or instruction, when executed by the processor, implements the steps of the grid decoding method as described in any one of claims 7 to 12.

16. A readable storage medium, characterized in that: The readable storage medium stores programs or instructions, and when the program or instructions are executed by the processor, the steps of the grid encoding method as described in any one of claims 1 to 6 are implemented, or when the program or instructions are executed by the processor, the steps of the grid decoding method as described in any one of claims 7 to 12 are implemented.

Images