Three-dimensional geometry model sequence compression method and device based on binary arithmetic coding, equipment and storage medium

Abstract

The application discloses a three-dimensional geometric model sequence compression method and device based on binary arithmetic coding, equipment and a storage medium, and the method comprises the following steps: segmenting a three-dimensional geometric model sequence based on the three-dimensional model data type of the three-dimensional geometric model sequence, obtaining a plurality of substructures, symbolizing the mesh patches of the three-dimensional geometric model with patch connection attributes, classifying the three-dimensional geometric model with vertex attributes based on interframe correlation, predicting the residual of the three-dimensional geometric model with vertex attributes based on the interframe correlation attributes, binarizing the quantized residual and / or the symbolized mesh patches, obtaining a binary bit string, and encoding the binary bit string, so that the three-dimensional geometric model sequence compression is realized, the multi-dimensional attributes of the dynamic three-dimensional geometric model sequence are effectively processed, the compression ratio and the compression efficiency are greatly improved, and the timeliness requirement of the three-dimensional geometric model sequence is met.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of computer graphics processing, and in particular to a three-dimensional geometric model sequence compression method and device based on binary arithmetic coding, equipment and storage medium. BACKGROUND

[0002] Computer graphics processing is an important field in computer science, and three-dimensional model processing is an essential part of computer graphics. In recent years, with the rapid development of computer processing power and computer vision algorithms, a new modeling method has emerged, real scene modeling. Real scene modeling usually uses color cameras, infrared cameras, laser radars and other sensing devices to take multiple-angle photos or scan of real three-dimensional objects in reality, so as to obtain three-dimensional geometric and color appearance characteristics at each angle, and then combine the characteristics at each angle by three-dimensional fusion algorithm to obtain a complete model. This model can be identical to the model obtained by traditional modeling methods in terms of data structure. The advantage of real scene modeling is that it can model real objects with high efficiency (can be completed in a few seconds). Based on the reconstruction of static objects, dynamic three-dimensional real scene modeling has been developed. Dynamic three-dimensional real scene modeling is a sequence of multiple three-dimensional models arranged in time sequence. However, dynamic three-dimensional real scene modeling has the characteristic of large data volume, resulting in the need for large capacity storage of three-dimensional geometric model sequence. In order to solve this problem, data compression of dynamic three-dimensional real scene model is needed. However, the data composition of three-dimensional model is very complex, usually involving attributes of different properties. The current three-dimensional model compression method cannot effectively handle the multi-dimensional attributes in three-dimensional model, resulting in low compression ratio and dynamic compression efficiency. SUMMARY

[0003] The main purpose of the present application is to provide a three-dimensional geometric model sequence compression method and device based on binary arithmetic coding, equipment and storage medium, which aims to solve the technical problem that the prior art cannot effectively handle the multi-dimensional attributes in three-dimensional model, resulting in low compression ratio and compression efficiency.

[0004] To achieve the above purpose, the present application provides a three-dimensional geometric model sequence compression method based on binary arithmetic coding, which comprises the following steps:

[0005] In response to the input of three-dimensional geometric model sequence, the data type and attribute type of the three-dimensional model sequence are identified, the data type of the three-dimensional model includes point cloud sequence and mesh sequence, the attribute type of the point cloud sequence includes vertex attribute, and the attribute type of the mesh sequence includes vertex attribute and face connection attribute;

[0006] segment the three-dimensional geometric model sequence based on the three-dimensional model data type to obtain a plurality of substructures, the substructure including a point cloud cluster and a submesh;

[0007] symbolize the mesh facets of the three-dimensional geometric model with the facet connection attribute based on an EdgeBreaker facet symbolization algorithm to obtain symbolized facets;

[0008] interframe correlation classification of the three-dimensional geometric model with vertex attributes to determine the interframe correlation attribute between frames of the three-dimensional geometric model with vertex attributes;

[0009] residual prediction of the three-dimensional geometric model with vertex attributes based on the interframe correlation attribute to determine quantized residuals;

[0010] binaryization of the quantized residuals and / or the symbolized facets to obtain a binary bit string;

[0011] bit string encoding of the binary bit string based on adaptive binary arithmetic coding to generate compressed data of vertex attributes and / or compressed data of facet connection attributes, thereby realizing compression of the three-dimensional geometric model sequence.

[0012] Optionally, the segmenting the three-dimensional geometric model sequence based on the three-dimensional model data type to obtain a plurality of substructures includes:

[0013] determining a segmentation strategy based on the three-dimensional model data type;

[0014] segmenting the three-dimensional geometric model sequence based on the segmentation strategy to obtain a plurality of substructures;

[0015] the segmentation strategy includes a spatial segmentation strategy, a propagation segmentation strategy, and a semantic segmentation strategy;

[0016] the spatial segmentation strategy includes:

[0017] determining a three-dimensional spatial boundary of a three-dimensional geometric model of a frame to be segmented in the three-dimensional geometric model sequence, and segmenting a three-dimensional space in which the three-dimensional geometric model is located into a plurality of subspaces based on the three-dimensional spatial boundary;

[0018] point cloud segmentation or mesh segmentation of the three-dimensional geometric model in each subspace based on point cloud attribute information or mesh attribute information of the three-dimensional geometric model, to obtain a plurality of point cloud clusters or a plurality of submeshes, the mesh attribute information including vertex position information of mesh facets and center of gravity position of mesh facets;

[0019] the propagation segmentation strategy includes:

[0020] determining a target number of facets of a submesh required in a propagation process;

[0021] The mesh faces in the three-dimensional geometric model of the frame to be segmented are iterated based on a breadth-first principle, and the three-dimensional geometric model is mesh segmented based on an iteration result and the target number of face pieces, to obtain a plurality of sub-meshes;

[0022] The semantic segmentation strategy comprises:

[0023] The three-dimensional geometric model of the frame to be segmented is rendered to generate a two-dimensional color image, and the two-dimensional color image is subjected to semantic recognition to generate a semantic map;

[0024] The three-dimensional geometric model is subjected to three-dimensional semantic segmentation based on the semantic map to obtain a plurality of point cloud clusters or a plurality of sub-meshes.

[0025] Optionally, the residual prediction of the three-dimensional geometric model with vertex attributes based on the inter-frame association attribute comprises:

[0026] Determining a change rate of the vertex attribute of the three-dimensional geometric model in the three-dimensional geometric model with vertex attributes based on the inter-frame association attribute;

[0027] Predicting the attribute value of the current vertex to be compressed according to the change rate to obtain a predicted attribute value;

[0028] Determining a prediction residual based on the predicted attribute value and the current attribute value of the current vertex to be compressed;

[0029] Determining a target quantization bit width according to a value range of the prediction residual and a quantization requirement, and quantizing the prediction residual based on the target quantization bit width to obtain a quantized residual, wherein the prediction residual is quantized according to the following formula:

[0030]

[0031] wherein, represents the quantized residual, and respectively represent a lower limit value and an upper limit value in the value range of the prediction residual, represents a floating-point value selected in the value range, represents the target quantization bit width, represents an adjustment factor of non-uniform quantization, represents the prediction residual to be quantized.

[0032] Optionally, the inter-frame association attribute comprises a first association attribute, a second association attribute and a third association attribute, the first association attribute is an attribute without inter-frame association, the second association attribute is an attribute with two-frame association, and the third association attribute is an attribute with more than two-frame association.

[0033] The attribute value of the current vertex to be compressed is predicted according to the change rate to obtain a predicted attribute value, including:

[0034] Obtaining buffered data of the current data frame and the historical data frame in a data buffer, the data buffer being a buffer area with a buffer upper limit exceeding three frames of data;

[0035] Constructing a prediction function based on the buffered data of the current data frame and the historical data frame, the prediction function including:

[0036]

[0037]

[0038]

[0039] wherein, the prediction function of the third associated attribute, the prediction function of the second associated attribute, the prediction function of the second associated attribute, , and respectively represent the first three frames of data of the current data frame, is the data that has been encoded and decoded in the current data frame;

[0040] The attribute value of the current vertex to be compressed is predicted based on the prediction function to obtain a predicted attribute value.

[0041] Optionally, the mesh patches of the three-dimensional geometric model with patch connection attributes are symbolized based on the EdgeBreaker patch symbolization algorithm to obtain symbolized patches, including:

[0042] Performing patch traversal on each frame of the three-dimensional geometric model with patch connection attributes to detect mesh holes;

[0043] In the patch traversal process, the position of the mesh patch without adjacent patches on any side is marked as a mesh hole position;

[0044] Adding a virtual vertex and a virtual patch at the mesh hole position to fill the mesh hole position;

[0045] Obtaining a symbol library based on the EdgeBreaker patch symbolization algorithm, the symbol library including a plurality of encoding symbols;

[0046] Generating a virtual encoding symbol according to the mesh hole position, and updating the symbol library based on the virtual encoding symbol;

[0047] Based on the updated symbol library, the mesh patches of the three-dimensional geometric model in the three-dimensional geometric model with patch connection attributes are symbolized, and a symbolized patch is obtained.

[0048] Optionally, after the patch traversal of each frame three-dimensional geometric model in the three-dimensional geometric model with patch connection attributes, the method further comprises:

[0049] Constructing a sorting buffer of vertex attributes and a sorting counter;

[0050] In the patch traversal process, in response to passing through a new mesh vertex, recording a vertex order value of the mesh vertex through the sorting counter, and generating a mapping relationship between the vertex attributes of the mesh vertex and the vertex order value, and writing the mapping relationship into the sorting buffer;

[0051] In response to the end of the patch traversal, reordering the vertex attributes of each mesh vertex based on the mapping relationship in the sorting buffer.

[0052] Optionally, the quantized residual and / or the symbolized patch are binarized to obtain a binary bit string, comprising:

[0053] The quantized residual is binarized to obtain a binary bit string;

[0054] And / or, based on the symbolized patch, determining the symbol occurrence frequency of each encoding symbol and the symbol co-occurrence frequency of each encoding symbol and other symbols;

[0055] Dividing the symbol co-occurrence frequency of each encoding symbol by the symbol occurrence frequency to obtain the prior probability of each encoding symbol;

[0056] Based on the prior probability, the encoding symbols are sorted, and based on the sorting result, the symbolized patch is binarily mapped to obtain a binary bit string.

[0057] In addition, to achieve the above-mentioned purpose, the application further provides a three-dimensional geometric model sequence compression device based on binary arithmetic coding, which comprises:

[0058] The three-dimensional geometric model sequence compression device based on binary arithmetic coding comprises:

[0059] A data recognition module is configured to recognize the three-dimensional model data type and attribute type of the three-dimensional geometric model sequence in response to the input of the three-dimensional geometric model sequence, wherein the three-dimensional model data type comprises a point cloud sequence and a mesh sequence, the attribute type of the point cloud sequence comprises vertex attributes, and the attribute type of the mesh sequence comprises vertex attributes and patch connection attributes;

[0060] a model segmentation module configured to segment the sequence of three-dimensional geometric models based on the three-dimensional model data type, to obtain a plurality of substructures, the substructures including point cloud clusters and sub-meshes;

[0061] a patch symbolization module configured to symbolize the mesh patches of the three-dimensional geometric model having patch connection attributes based on an EdgeBreaker patch symbolization algorithm, to obtain symbolized patches;

[0062] an inter-frame association analysis module configured to classify the three-dimensional geometric model having vertex attributes in terms of inter-frame association, to determine inter-frame association attributes between frames of the three-dimensional geometric model having vertex attributes;

[0063] a residual prediction module configured to predict residuals of the three-dimensional geometric model having vertex attributes based on the inter-frame association attributes, to determine quantized residuals;

[0064] a binarization module configured to binarize the quantized residuals and / or the symbolized patches, to obtain binary bit strings;

[0065] a binary compression module configured to encode the binary bit strings based on adaptive binary arithmetic coding, to generate compressed data of vertex attributes and / or compressed data of patch connection attributes, to realize compression of the sequence of three-dimensional geometric models.

[0066] In addition, to achieve the above object, the present application further provides a device for compressing a sequence of three-dimensional geometric models based on binary arithmetic coding, which comprises a memory, a processor and a computer program stored in the memory and executable on the processor, the computer program being configured to implement the steps of the method for compressing a sequence of three-dimensional geometric models based on binary arithmetic coding as described above.

[0067] In addition, to achieve the above object, the present application further provides a computer readable storage medium having a computer program stored thereon, the computer program being executable by a processor to implement the steps of the method for compressing a sequence of three-dimensional geometric models based on binary arithmetic coding as described above.

[0068] In addition, to achieve the above object, the present application further provides a computer program product comprising a computer program, the computer program being executable by a processor to implement the steps of the method for compressing a sequence of three-dimensional geometric models based on binary arithmetic coding as described above.

[0069] The application identifies the three-dimensional model data type and attribute type of the three-dimensional geometric model sequence in response to the three-dimensional geometric model sequence input, the three-dimensional model data type includes a point cloud sequence and a mesh sequence, the attribute type of the point cloud sequence includes a vertex attribute, the attribute type of the mesh sequence includes a vertex attribute and a face connection attribute, the three-dimensional geometric model sequence is segmented based on the three-dimensional model data type, a plurality of substructures are obtained, the substructure includes a point cloud group and a sub-mesh, the mesh face of the three-dimensional geometric model with the face connection attribute is symbolized based on an EdgeBreaker face symbolization algorithm, a symbolized face is obtained, the three-dimensional geometric model with the vertex attribute is classified in terms of inter-frame correlation, the inter-frame correlation attribute between frames in the three-dimensional geometric model with the vertex attribute is determined, the three-dimensional geometric model with the vertex attribute is predicted based on the inter-frame correlation attribute, a quantized residual error is determined, the quantized residual error and / or the symbolized face is binarized, a binary bit string is obtained, the binary bit string is bit string encoded based on adaptive binary arithmetic coding, compressed data of the vertex attribute and / or compressed data of the face connection attribute are generated, and the three-dimensional geometric model sequence compression is realized. BRIEF DESCRIPTION OF DRAWINGS

[0070] In order to more clearly illustrate the technical solutions of the embodiments of the present application or the prior art, the drawings needed in the embodiments or prior art description will be briefly introduced as follows. Obviously, for those skilled in the art, other drawings can also be obtained based on these drawings without any creative labor.

[0071] Figure 1 It is a structure schematic diagram of the three-dimensional geometric model sequence compression device based on binary arithmetic coding of the hardware running environment related to the embodiment scheme of the present application.

[0072] Figure 2 It is a flowchart of the three-dimensional geometric model sequence compression method based on binary arithmetic coding of the embodiment of the present application.

[0073] Figure 3A three-dimensional geometric model sequence compression flowchart in an embodiment of the three-dimensional geometric model sequence compression method based on binary arithmetic coding of the present application;

[0074] Figure 4 A schematic diagram of virtual surface filling holes in an embodiment of the three-dimensional geometric model sequence compression method based on binary arithmetic coding of the present application;

[0075] Figure 5 A flowchart of encoding compression in an embodiment of the three-dimensional geometric model sequence compression method based on binary arithmetic coding of the present application;

[0076] Figure 6 A structural block diagram of an embodiment of the three-dimensional geometric model sequence compression device based on binary arithmetic coding of the present application.

[0077] The implementation, functional features and advantages of the present application will be further described with reference to the embodiments and the accompanying drawings. DETAILED DESCRIPTION

[0078] It should be understood that the specific embodiments described herein are intended to be illustrative only and not limiting of the present application.

[0079] Reference Figure 1 , Figure 1 A three-dimensional geometric model sequence compression device structure schematic diagram based on binary arithmetic coding of the hardware running environment involved in the embodiment scheme of the present application.

[0080] As Figure 1 shown, the three-dimensional geometric model sequence compression device based on binary arithmetic coding can include a processor 1001, such as a central processing unit (CPU), a communication bus 1002, a user interface 1003, a network interface 1004, and a memory 1005. The communication bus 1002 is used to realize the connection and communication between the components. The user interface 1003 can include a display screen, an input unit such as a keyboard, and can also include a standard wired interface, a wireless interface. The network interface 1004 can optionally include a standard wired interface, a wireless interface (such as a wireless fidelity (WI-FI) interface). The memory 1005 can be a high-speed random access memory (RAM), and can also be a stable non-volatile memory (NVM), such as a disk memory. The memory 1005 can also be a storage device independent of the aforementioned processor 1001.

[0081] Those skilled in the art can understand that Figure 1 The structure shown in the figure does not constitute a limitation on the binary arithmetic coding-based three-dimensional geometric model sequence compression device, and can include more or fewer components than the figure, or combine certain components, or different component arrangements.

[0082] As Figure 1 As shown, the memory 1005 as a computer readable storage medium can include an operating system, a network communication module, a user interface module, and a binary arithmetic coding-based three-dimensional geometric model sequence compression program.

[0083] In Figure 1 The network interface 1004 in the binary arithmetic coding-based three-dimensional geometric model sequence compression device shown in the figure is mainly used for data communication with a network server; the user interface 1003 is mainly used for data interaction with a user; the processor 1001 and the memory 1005 in the binary arithmetic coding-based three-dimensional geometric model sequence compression device of the application can be arranged in the binary arithmetic coding-based three-dimensional geometric model sequence compression device, and the binary arithmetic coding-based three-dimensional geometric model sequence compression device calls the binary arithmetic coding-based three-dimensional geometric model sequence compression program stored in the memory 1005 through the processor 1001, and executes the binary arithmetic coding-based three-dimensional geometric model sequence compression method provided by the embodiment of the application.

[0084] The embodiment of the application provides a binary arithmetic coding-based three-dimensional geometric model sequence compression method. Figure 2 , Figure 2 The embodiment of the application provides a binary arithmetic coding-based three-dimensional geometric model sequence compression method.

[0085] In the embodiment, the binary arithmetic coding-based three-dimensional geometric model sequence compression method comprises the following steps:

[0086] Step S10: In response to a three-dimensional geometric model sequence input, identifying the three-dimensional model data type and the attribute type of the three-dimensional geometric model sequence.

[0087] It should be noted that the embodiment is applied to compress a three-dimensional geometric model sequence, and solves the problems of large three-dimensional geometric sequence data, difficult network transmission, and great storage resource occupation by efficiently compressing the three-dimensional geometric model sequence.

[0088] The main method in the prior art for three-dimensional geometry sequence compression in the form of point cloud is to project the point cloud to multiple two-dimensional planes to form multiple two-dimensional video sequences of different views, and then use mature two-dimensional video coding for compression. This method is only applicable to point cloud, and the entire process has a huge amount of calculation and obvious quality loss. Therefore, this method is usually only used in static three-dimensional geometry model compression.

[0089] The focus of the prior art for three-dimensional geometry sequence compression in the form of mesh formed by triangular faces is generally placed on how to symbolize the triangular connection information, and little attention is paid to how to efficiently encode the symbol information. The mainstream method mainly uses Huffman coding or arithmetic coding to encode and decode the symbol sequence. Although these methods can achieve the purpose of reducing data volume, the effect is very limited (i.e. the compression ratio before and after compression is not high), and they do not consider or have the ability of real-time encoding and decoding. The compression of vertex attributes in the three-dimensional geometry sequence in the form of mesh is considered even less. Generally, only the simple difference between the front and back frames is made for the vertex position attributes, and then the difference is encoded using Huffman coding or arithmetic coding. This compression method has the same problem as the compression of face information, i.e. it cannot achieve high efficiency and real-time performance.

[0090] It should be understood that the execution subject of the embodiment can be a computing service device with data processing, network communication and program running functions, such as a tablet computer, a personal computer, a mobile phone, etc., or a terminal electronic device capable of realizing the above functions. The following takes a three-dimensional geometry model sequence compression device (referred to as a compression device) based on binary arithmetic coding as an example to explain the embodiment and the following embodiments.

[0091] It should be noted that the three-dimensional geometry model sequence compression device (referred to as a compression device) is used as an example to explain the embodiment and the following embodiments. Figure 3 , Figure 3 The figure is a three-dimensional geometry model sequence compression flowchart in an embodiment of the present application. In this embodiment, the format and attributes of the model sequence are identified, then the three-dimensional geometry model in the form of point cloud / grid is segmented, then the segmented model is binary coded, the redundancy of various attributes within and between frames is deeply mined without obvious quality loss, thereby greatly improving the compression efficiency of the three-dimensional geometry sequence, and finally all the to-be-coded information is binary coded to greatly reduce the amount of calculation in the entropy coding process, thereby meeting the requirement of real-time performance.

[0092] It should be noted that the three-dimensional model data type includes point cloud sequence and mesh sequence, the attribute type of the point cloud sequence includes vertex attribute, and the attribute type of the mesh sequence includes vertex attribute and face connection attribute.

[0093] It should be noted that the mesh sequence and the point cloud sequence are two common formats for representing a sequence of three-dimensional geometric models. The mesh format is a sequence of models composed of three-dimensional models with mesh attributes, and the point cloud format is a sequence of models composed of three-dimensional models with point cloud attributes. The mesh format contains vertex attributes and face connection attributes, while the point cloud format only contains vertex attributes.

[0094] Step S20: segmenting the sequence of three-dimensional geometric models based on the three-dimensional model data type to obtain a plurality of sub-structures.

[0095] It should be noted that the sub-structures include point cloud clusters and sub-meshes.

[0096] In some embodiments, the compression device segments the sequence of three-dimensional geometry, divides the three-dimensional geometric models in point cloud format into a plurality of sub-point cloud clusters, or divides the three-dimensional geometric models in mesh format into a plurality of sub-meshes, and independently processes and encodes the sub-three-dimensional structures. The segmentation method can be a spatially fixed partitioning method or a spatially spreading method, or a three-dimensional semantic recognition method based on the semantics of the object, and then segmenting based on the recognized semantics, or both.

[0097] Further, in order to improve the model segmentation efficiency and provide appropriate segmentation strategies for three-dimensional geometric models with different format attributes, in some embodiments, the above step S20 can include:

[0098] Step S201: determining a segmentation strategy based on the three-dimensional model data type;

[0099] Step S202: segmenting the sequence of three-dimensional geometric models according to the segmentation strategy to obtain a plurality of sub-structures.

[0100] It should be noted that the segmentation strategy includes a spatial segmentation strategy, a spreading segmentation strategy, and a semantic segmentation strategy.

[0101] The spatial segmentation strategy includes: determining the three-dimensional spatial boundary of the three-dimensional geometric model of the frame to be segmented in the sequence of three-dimensional geometric models, and segmenting the three-dimensional space where the three-dimensional geometric model is located into a plurality of sub-spaces based on the three-dimensional spatial boundary; based on the point cloud attribute information or the mesh attribute information of the three-dimensional geometric model, performing point cloud segmentation or mesh segmentation on the three-dimensional geometric model in each sub-space to obtain a plurality of point cloud clusters or a plurality of sub-meshes, the mesh attribute information including vertex position information of mesh faces and center of gravity position of mesh faces;

[0102] The spreading segmentation strategy comprises: determining a target number of patches of a required sub-grid in a spreading process; performing spreading traversal on each grid patch in a three-dimensional geometric model of a frame to be segmented based on a breadth-first principle, and performing grid segmentation on the three-dimensional geometric model based on a spreading traversal result and the target number of patches, to obtain a plurality of sub-grids;

[0103] The semantic segmentation strategy comprises: rendering a three-dimensional geometric model of a frame to be segmented to generate a two-dimensional color image, and performing semantic recognition on the two-dimensional color image to generate a semantic graph; performing three-dimensional semantic segmentation on the three-dimensional geometric model based on the semantic graph to obtain a plurality of point cloud clusters or a plurality of sub-grids.

[0104] It can be understood that the compression device performs regional segmentation on three-dimensional data, and then performs predictive coding on the segmented data. If the input three-dimensional format is a point cloud, the segmentation only classifies the point cloud into a plurality of point cloud clusters in form. If the input three-dimensional format is a grid, the grid is physically split into sub-grids, and the splitting process does not cut the patch edge, but copies the patch edge to maintain the structure of the grid patch from being damaged. In this process, first, the patch edge to be split is determined, the numbers of the two end vertices are found from the edge, and a new vertex number is added to point to the same vertex attribute. The method of splitting the patch can be one of spatial segmentation, spreading segmentation, and semantic segmentation according to different requirements and the structure of the model.

[0105] It should be understood that in the spatial method, in view of the characteristics of a three-dimensional model being large, complex, and without subjective composition, the upper and lower boundaries of the model in the X, Y, and Z directions in the three-dimensional space are first determined in the segmentation process, thereby forming a cuboid rectangular space that wraps the three-dimensional model. The space is equally segmented in the X, Y, and Z dimensions to form a plurality of continuous and closely connected subspaces.

[0106] For three-dimensional data in the form of a point cloud, the point cloud in each subspace forms an independent point cloud cluster, thereby realizing segmentation of the point cloud.

[0107] For three-dimensional data in the form of a grid, the barycenter position of the patch is first calculated using the vertex position information of the patch, and the sub-grids formed by the barycenters of the patches in each subspace, thereby realizing segmentation of the grid patch.

[0108] The propagation method is mainly suitable for three-dimensional data in grid form. First, the maximum number of patches required in the propagation process is determined. Propagation starts with an arbitrary patch. The breadth-first principle is used during propagation, that is, all patches connected to the same vertex are traversed first, and untraversed vertices are recorded. When the patches of a vertex are all accessed, the oldest accessed patch is selected to repeat the same operation. A counter is maintained during traversal. When the counter reaches the upper limit of the target sub-grid, all patches traversed in this round form a segmented sub-grid, and the counter is reset. This method is used until all patches are accessed.

[0109] The semantic segmentation method is suitable for objects with natural semantic features. This method uses image recognition means to segment three-dimensional objects with semantic features, such as identifying the head, hands, thighs, and body of a human three-dimensional geometric model, and then segmenting each part on the three-dimensional model.

[0110] The specific process is as follows. First, render the three-dimensional model to generate a two-dimensional color image, and project the patch or point cloud numbers to the forming plane to generate a number map. Apply a semantic recognition algorithm to the rendered two-dimensional image to form a semantic map. Using the semantic map and the number map, classify the vertex numbers (for point clouds) or patch numbers (for grids) according to different semantics to achieve three-dimensional semantic segmentation. The segmented sub-point cloud or sub-grid will be independently predicted and encoded.

[0111] Step S30: Based on the EdgeBreaker patch symbolization algorithm, the grid patches of the three-dimensional geometric model with patch connection attributes are symbolized to obtain symbolized patches.

[0112] In a specific implementation, the compression device directly loads the patch connection attributes in the grid data into the system memory. For a grid with holes, it is first virtually filled, then symbolized, and finally the symbol representing the patch is binarized.

[0113] In some embodiments, if the three-dimensional geometric model sequence to be compressed is a point cloud format sequence, step S30 is skipped and step S40 is directly executed; if the three-dimensional geometric model sequence to be compressed is a grid format sequence, step S30 is directly executed to compress the patch connection attributes, and then the vertex attributes are compressed. This is because the three-dimensional geometric model in point cloud format only has vertex attributes and does not need to be compressed. The three-dimensional geometric model in grid format contains both vertex attributes and patch connection attributes, so the patch connection attributes need to be compressed first, and then the vertex attributes are compressed.

[0114] In some embodiments, when the compression device compresses the mesh format three-dimensional geometric model, the face connection attribute is compressed first, then the vertex attribute is reordered, and after the reordering is completed, the vertex attribute is compressed, thereby avoiding the problem of loss or disorder of the vertex attribute in the compression process of the mesh format three-dimensional geometric model, and realizing lossless compression.

[0115] Further, in order to effectively solve the problem that the mesh format three-dimensional geometric model cannot be face symbolized due to the existence of holes, thereby improving the compression efficiency of the mesh model, in some embodiments, the step S30 can include:

[0116] Step S301: performing face traversal on each frame three-dimensional geometric model in the three-dimensional geometric model with face connection attribute to detect mesh holes;

[0117] Step S302: marking the mesh face position without adjacent face as a mesh hole position in the face traversal process;

[0118] Step S303: adding a virtual vertex and a virtual face at the mesh hole position to fill the hole at the mesh hole position;

[0119] Step S304: obtaining a symbol library based on the EdgeBreaker face symbolization algorithm, the symbol library including a plurality of encoding symbols;

[0120] Step S305: generating a virtual encoding symbol according to the mesh hole position, and updating the symbol library based on the virtual encoding symbol;

[0121] Step S306: symbolizing the mesh face of the three-dimensional geometric model in the three-dimensional geometric model with face connection attribute based on the updated symbol library, to obtain a symbolized face.

[0122] It can be understood that the EdgeBreaker mesh face symbolization method has high efficiency and is often used as a symbolization method, but the method assumes that the three-dimensional mesh structure has no holes, which is almost impossible to guarantee in reality. The present application provides a solution to the problem of using EdgeBreaker as a symbolization method when encountering a three-dimensional mesh model with holes. On the basis of the five symbol libraries of "C", "L", "E", "S", and "R" defined in the EdgeBreaker algorithm, a new symbol "X" is added to represent a virtual face. In the symbolization process, first detect the mesh hole, and the detection method is as follows: if any side of the current face has no adjacent face, then the side is determined as the boundary of the hole, and the face is determined as the mesh hole boundary face. If the hole is detected, add a virtual vertex (new vertex number) and connect all the boundaries of the hole, which may generate multiple virtual faces and complete the filling of the mesh hole. In the face symbolization process, if a face containing a previously added virtual vertex is encountered during the iteration process, the face is also determined as a virtual face and symbolized using "X". If no hole is detected, i.e., the face belongs to the internal face of the mesh, the standard EdgeBreaker symbolization method is called. In the reverse symbolization process at the decoding end, if "X" is encountered, no new vertex is generated, and if other symbols are encountered, the standard EdgeBreaker reverse symbolization process is used.

[0123] It can be understood that the compression device pre-processes the mesh with holes before compressing the three-dimensional geometric model data in the form of a mesh. Specifically, when traversing the mesh patches, if any side of the patch has no adjacent patch, it is determined as a mesh hole, a virtual vertex is added in the hole, and the connected mesh is added with a virtual vertex and a virtual patch to fill the hole, as shown in FIG. 2. Figure 4 , Figure 4 FIG. 2 is a schematic diagram of a virtual face filling a hole in an embodiment; in the decoding process, the virtual vertex and the virtual patch are identified and discarded, thereby restoring the original mesh structure.

[0124] For the added virtual patch, a new symbol is added to the patch symbolization algorithm to represent the virtual patch. In the encoding process, if a virtual patch is encountered, the new symbol is used for symbolization, and in the reverse symbolization process, if the symbol is encountered, the patch is ignored.

[0125] Further, in order to avoid the vertex attribute disorder of the three-dimensional geometric model in mesh format, ensure that the correspondence between the vertex attribute and the mesh patch vertex number is not affected by compression, and ensure data integrity, in some embodiments, after the above step S301, the method can further include:

[0126] Step S3011: constructing a sorting buffer of vertex attributes and a sorting counter;

[0127] Step S3012: In the process of patch traversal, in response to passing a new mesh vertex, record the vertex order value of the mesh vertex by means of the sorting counter, and generate a mapping relationship between the vertex attribute of the mesh vertex and the vertex order value, and write the mapping relationship into the sorting buffer;

[0128] Step S3013: In response to the end of patch traversal, reorder the vertex attributes of each mesh vertex based on the mapping relationship in the sorting buffer.

[0129] It can be understood that when compressing the mesh format three-dimensional geometric model, the vertex attributes need to be reordered, because the mesh format three-dimensional geometric model has both patch connection attributes and vertex attributes, and the compression device reorders the vertex attributes based on the correspondence between the vertex number and the vertex attribute in the mesh patch after decompression, which specifically includes:

[0130] 1. Reordering must occur before vertex attribute compression;

[0131] 2. Establish a vertex number mapping list and a counter, and allocate buffer space for later vertex attribute reordering;

[0132] 3. In the process of mesh symbolization, when traversing the mesh patch, each time a new vertex is passed, record the counter value and the original vertex number in the table at the same time, and then increment the counter by 1;

[0133] 4. At the end of symbolization, use the value pair in the table to read the corresponding data from the vertex attribute with the original vertex number as the index, and write the data into the reordered buffer area with the new vertex number in the second column as the index;

[0134] 5. After all vertex data migration is completed, exchange the memory addresses of the original vertex attribute storage area and the reordered buffer area.

[0135] In some embodiments, in order to avoid compression loss, a processing method is provided to maintain the correspondence between vertex attributes and mesh patch vertex numbers from being affected by compression before the three-dimensional geometric model with patch connection attributes is compressed. To achieve this purpose, the compression device first compresses and decompresses the patch connection attributes (hereinafter referred to as S attributes) and then reorders the vertex attributes (hereinafter referred to as V attributes). A vertex number mapping list with two columns and a counter are maintained during the processing, and the counter is initialized to 1. In the process of symbolizing S attributes, when a new patch is encountered, if any vertex of the patch has not appeared in the list, the vertex number in the S attributes is first recorded in the first column of the list, and the value of the counter is recorded in the second column, and then the counter is incremented by 1. If the vertex number has appeared in the list, no additional processing is performed. This method is followed until each patch is symbolized. Next, the vertex number mapping list is used to reorder the vertex attributes. A reordering buffer area is first created, which is comparable in size to the current vertex attributes. Each column of the list is processed in a loop, the value in the first column of the list is used to locate the V attribute element as the copy source, the value in the second column of the list is used to locate the reordering buffer area as the copy target, and the V attribute is copied from the source to the target. Finally, the reordering buffer and the V attribute storage area are address swapped. The swapped area can be retained for the next reordering process without the need to apply for system storage resources in advance.

[0136] Step S40: Inter-frame correlation classification of the three-dimensional geometric model with vertex attributes is performed to determine the inter-frame correlation attributes between the frames of the three-dimensional geometric model with vertex attributes.

[0137] In some embodiments, the inter-frame correlation attributes include a first correlation attribute, a second correlation attribute, and a third correlation attribute, the first correlation attribute is a non-inter-frame correlation attribute, the second correlation attribute is a two-frame correlation attribute, and the third correlation attribute is a more than two-frame correlation attribute.

[0138] In some embodiments, for vertex attributes, according to the inter-frame correlation of the attributes, three types are divided: one, more than two-frame correlation attribute (referred to as V3 type), two, two-frame correlation attribute (referred to as V2 type), and three, non-inter-frame correlation attribute (referred to as V1 type). For any vertex attribute with multi-channel characteristics, such as three-channel vertex color and three-channel XYZ vertex coordinate, the data is loaded into the system memory in a channel priority manner. This storage method optimizes the access to the memory in the later encoding process and improves the real-time efficiency.

[0139] Step S50: Residual prediction is performed on the three-dimensional geometric model with vertex attributes based on the inter-frame correlation attributes to determine the quantized residual.

[0140] In a specific implementation, the compression device can determine the quantized residual based on the inter-frame associated attribute and the corresponding residual prediction strategy selection:

[0141] For a three-dimensional geometric model with vertex attributes, the inter-frame associated characteristics are determined, a suitable change rate is designed based on the inter-frame associated characteristics, the current vertex attribute to be compressed is predicted using the change rate, the difference between the predicted attribute value and the current attribute value, i.e., the predicted residual, is taken for encoding, and the target quantization bit width is selected according to the value range of the predicted residual and the quantization requirement for the residual quantization.

[0142] For a three-dimensional geometric model with face connection attributes, a new symbol is added to the EdgeBreaker face symbolization algorithm to represent a virtual face, and the modified symbolization algorithm is used to symbolize the mesh faces.

[0143] Further, in order to accurately predict the vertex attribute residual, in some embodiments, the above step S50 can include:

[0144] Step S501: determining the change rate of the vertex attribute of the three-dimensional geometric model in the three-dimensional geometric model with vertex attributes based on the inter-frame associated attribute;

[0145] Step S502: predicting the attribute value of the current vertex to be compressed according to the change rate to obtain a predicted attribute value;

[0146] Step S503: determining a predicted residual based on the predicted attribute value and the current attribute value of the current vertex to be compressed;

[0147] Step S504: determining a target quantization bit width according to the value range of the predicted residual and the quantization requirement, and quantizing the predicted residual based on the target quantization bit width to obtain a quantized residual.

[0148] It should be noted that there are many types of vertex attributes, such as common position, color. Since it is a sequence, there is a change rate between frames. For example, the color of the first frame is 254, and the color of the same vertex in the second frame is 255, so the change here is 1. A three-dimensional geometric structure is usually composed of hundreds of thousands or millions of vertices, and compression is processed one by one, so the vertex that has not been processed is called the current vertex to be compressed.

[0149] It should be noted that the inter-frame associated attribute includes a first associated attribute (referred to as V1 type), a second associated attribute (referred to as V2 type), and a third associated attribute (referred to as V3 type), the first associated attribute is an attribute without inter-frame association, the second associated attribute is an attribute with two-frame association, and the third associated attribute is an attribute with more than two-frame association.

[0150] The application provides a method for predicting residual binary coding of three-dimensional data. For V3 type, velocity or acceleration is generated when three-dimensional model moves, which is directly reflected on vertex coordinate attribute. In order to further improve compression ratio, the current V3 attribute is predicted, and the prediction method is as follows: for vertex attribute numerical type floating point data, the pre-processing is quantization, and the process is to convert floating point data type into integer type. The pre-quantization method is as follows: first, the upper and lower value boundaries and are determined and H for the vertex attribute without value range limitation, such as vertex position coordinates X, Y and Z, the upper and lower value boundaries are provided to the system by the outside bit, the longer the length, the smaller the precision loss, but the larger the data amount, and vice versa. All values in the range of to are mapped to 0 to -1. The specific mapping (and demapping) scheme can be designed according to the required precision distribution, for example, the uniform mapping is selected without special requirements: the quantized value of the given floating point value (belonging to the range of , ) is

[0151]

[0152] The inverse quantized value is

[0153]

[0154] Select appropriate quantization length for each vertex attribute , write the value of into the output code stream, and transmit first, and then the upper and lower value ranges of the vertex attribute residual and . For the vertex attribute with inherent range, the inherent range is directly selected as and . For the vertex attribute which is not common and does not have inherent range, the user provides and . For the common vertex attribute without inherent range, the default and are selected. For the input residual value exceeding the range of , , the residual is truncated to or . 、 、 As a parameter, select an appropriate quantization function Quantize the residual so that the output value is completely in the range of 0 to -1. The predicted residual is non-uniformly quantized with reference to the following formula:

[0155]

[0156] wherein, represents the quantized residual, and represent the lower limit value and the upper limit value in the value range of the predicted residual, respectively, represents the floating-point value selected in the value range, represents the target quantization bit width, represents the adjustment factor of non-uniform quantization (a higher-precision quantization in a specified range can be obtained by adjusting the value), represents the predicted residual to be quantized.

[0157] The binary coding mode for the quantized vertex attribute residual: the positive and negative attributes of the residual are extracted, and a binary probability model is set for the sign bit (positive = 0, negative = 1); the absolute value of the residual is taken to obtain the absolute residual amount, and the binary coding is performed using a unary code for the case where the absolute residual amount is less than M. For the case where the absolute residual amount is greater than M, the part less than M is binary coded using a unary code, and the part greater than M is coded using a Golomb code. An independent probability model is set for each bit of the unary coded part, and the probability model used is updated when one bit is coded, i.e., the probability of 0 or 1 is redistributed, so as to achieve the purpose of adaptive coding in the coding process. The part coded using the Golomb code is coded using a fixed and non-updated uniform distribution probability model.

[0158] Further, in order to improve the accuracy and efficiency of residual prediction, appropriate prediction strategies are selected for different inter-frame associated attributes, and the above step S502 can include:

[0159] Step S5021: acquiring buffered data of a current data frame and historical data frames in a data buffer, the data buffer being a pre-constructed buffer area with a buffer upper limit exceeding three frames of data;

[0160] Step S5022: constructing a prediction function based on the buffered data of the current data frame and the historical data frames;

[0161] Step S5023: predicting the attribute value of a current vertex to be compressed based on the prediction function to obtain a predicted attribute value.

[0162] It should be noted that the present application also proposes a method for systematically predicting and obtaining residual errors of vertex attributes in three-dimensional data. For vertex attributes, according to the inter-frame correlation of attributes, there are three types: attributes with more than two frames of correlation (i.e. third correlation attribute, referred to as V3 type); attributes with two frames of correlation (i.e. second correlation attribute, referred to as V2 type), and attributes without inter-frame correlation (i.e. first correlation attribute, referred to as V1 type).

[0163] It can be understood that the compression device opens at least three frame data buffer areas to save the historical compressed and decompressed attribute data during the encoding process, which are referred to as 0, 1 and 2 buffer positions. When a frame of data is encoded, the data in the 0 position and the 1 position are exchanged, and then the data in the 1 position and the 2 position are exchanged, and finally the current encoding data is decompressed and restored and placed in the 0 position. If a higher order prediction method is used, N buffer areas are opened, and the nth position and the n+1 position are exchanged in order from n=0, and then incremented. The prediction module uses the buffer area data to predict and encode the V3 type and V2 type attributes, and uses the prediction function to predict the current frame to be encoded from the reconstructed frame filled in the buffer area , and uses the prediction function to generate the prediction value of the current frame , and takes the prediction difference of the vertex attribute as the prediction residual error , that is .

[0164] In some embodiments, the compression device buffers the current frame data in the buffer area and the reconstructed data of the previous three frames , , The specific form of the prediction function of the V3 type attribute is , the specific form of the prediction function of the V2 type attribute is , and the form of the prediction function of the attribute without inter-frame correlation is , , is the already encoded and decoded data of the same frame. Here and functions respectively predict the acceleration and speed of the change, and implements intra-frame prediction.

[0165] One implementation is to assume that the acceleration of the change is constant, i.e. the acceleration is constant, and the prediction function is:

[0166]

[0167] One The implementation assumes that the changing speed is constant, and the prediction function is:

[0168]

[0169] An implementation assumes that the adjacent attributes in the same frame have relevance, and the prediction function is:

[0170]

[0171] That is, since the data cached by the V3 type attribute covers the V2 and V1 types, the V3 type can select to use or the prediction function. Since the data cached by the V2 type attribute covers the V1 type, the V2 type attribute can select to use the prediction function.

[0172] wherein, represents the prediction function of the third associated attribute, represents the prediction function of the second associated attribute, represents the prediction function of the second associated attribute, , and represent the first three frames of data of the current data frame, is the data that has been encoded and decoded in the current data frame.

[0173] The prediction attribute value and the original attribute value of the current frame form an array with the same number of channels, respectively, and the sum of the differences of each array is calculated to determine the optimal prediction mode when using multi-mode prediction (i.e., using , , several of them). The implementation of the difference evaluation can use the sum of absolute differences (SAD) or the mean square error (MSE) method. According to the accuracy of the prediction results under multiple modes, V3 and V2 type data select the prediction mode with the lowest difference evaluation value as the final prediction model. For multi-channel data, each channel can independently complete prediction, i.e., the output bitstream must write the prediction mode of each channel, or the same prediction mode can be applied to multiple channels. Whether to use multi-channel consistent prediction can be determined according to the limitation of the calculation amount of the encoder and the real-time performance in actual application, etc. The multi-channel prediction mode and the change rate prediction mode are first entered into the bitstream by the encoder. When decoding at the receiving end, the decoder parses the bitstream. Since the prediction mode type is decoded first, the decoding system uses the same prediction method as when encoding to obtain the prediction value . Then, the residual compression data following the prediction value in the bitstream is parsed and decoded to obtain . The prediction value , using the residual to compensate, thereby obtaining the original compressed frame .

[0174] Step S60: binarize the quantized residual and / or the symbolized patch to obtain a binary bit string.

[0175] In a specific implementation, the compression device uses unary coding to binarize all symbols in the symbolization calculation of the patch. Using a plurality of types of three-dimensional geometric sequence data sets for symbolization processing, the probability of occurrence of each symbol (i.e., prior probability) is counted, and the probability of occurrence of any other symbol under the condition that a certain symbol has occurred (i.e., conditional probability) is counted.

[0176] According to the order from large to small of the prior probability, a unary code from short to long is assigned; for each bit of the unary code, a probability model for binary arithmetic coding is set; according to the statistical results, symbols with conditional probability characteristics, i.e., a certain symbol conditional probability and prior probability differ greatly, are set with an additional conditional probability model. The statistical values are used to initialize all probability models.

[0177] The model will update the probability model or conditional probability model corresponding thereto according to its position when ending a bit coding according to the arithmetic coding algorithm. If a reset frame is encountered, the statistical values are used to initialize all probability models.

[0178] Further, in order to improve the binarization efficiency of the symbolized patch, in some embodiments, the above step S60 can include:

[0179] Step S601: binarize the quantized residual to obtain a binary bit string;

[0180] and / or,

[0181] Step S602: determine the symbol occurrence frequency of each coding symbol and the symbol co-occurrence frequency of each coding symbol with other symbols based on the symbolized patch;

[0182] Step S603: divide the symbol co-occurrence frequency of each coding symbol by the symbol occurrence frequency to obtain the prior probability of each coding symbol;

[0183] Step S604: sort the coding symbols based on the prior probability, and binarize the symbolized patch based on the sorting result to obtain a binary bit string.

[0184] It can be understood that the present application provides a binary arithmetic coding method for symbolized geometric patches. Binary coding is performed on the symbolized patches using unary code, and six symbols are mapped to six binary bit strings of "0", "10", "11000", "11100", "11110", and "11111". The specific mapping relationship is determined by probability statistics. The above symbolization processing is performed on three-dimensional geometric sequence data sets in various scenarios, the number of occurrences of each symbol is counted, and the probability (prior probability) of the occurrence of each symbol is obtained by dividing the total number of symbols. The number of occurrences of any other symbol under the condition that a certain symbol M occurs (i.e. the co-occurrence number of symbols) is also counted, for example, for the symbol "L", the number of occurrences of "CL", "EL", "SL", "RL", "XL", and "LL" (i.e. the number of occurrences of "L" under the condition that "C", "E", "S", "R", "X", and "L" occur) is counted. The number is divided by the number of occurrences of each symbol to obtain the conditional probability of symbol M in the occurrence of any symbol. The six symbols are sorted in descending order of probability value, and the symbol types are then mapped to the above six binary bit string sequences in turn.

[0185] The binary coding method for symbols: multiple probability models are set for five-bit positions respectively, and each model tracks the probability and conditional probability of a certain symbol, i.e. the distribution and conditional distribution of each bit 0 and 1. If statistics show that the conditional probability of a certain symbol is significantly different from its probability, it means that the symbol has statistical correlation with the conditioned symbol, and the conditional probability model is switched to for coding when coding. When a bit position is coded, the corresponding probability model is updated, i.e. the probability of 0 or 1 is redistributed, so as to achieve the purpose of adaptive coding in the coding process.

[0186] Step S70: bit string coding is performed on the binary bit string based on adaptive binary arithmetic coding, to generate compressed data of vertex attributes and / or compressed data of patch connection attributes, and to realize compression of the three-dimensional geometric model sequence.

[0187] In a specific implementation, the compression device first identifies the attributes of the input three-dimensional geometric model sequence data, selects a corresponding compression strategy based on the data attributes, and refers to the compression strategy selection table shown in FIG. 6. Figure 5 Figure 5 The flowchart of the coding and compression in an embodiment is shown in FIG. 7. The compression device first determines whether the input three-dimensional geometric model is in point cloud format:

[0188] ​If the input data is in point cloud format, vertex attribute reordering is performed, inter-frame correlation of vertex attributes is identified, a corresponding residual prediction strategy is selected based on inter-frame correlation attribute of the three-dimensional geometric model sequence, predicted vertex attribute values are obtained, prediction residuals are determined based on the vertex attribute values and the compressed vertex attribute values, the prediction residuals are binarized, and then binary arithmetic coding is performed to achieve dynamic compression of the three-dimensional geometric model sequence.

[0189] If the input data is not in point cloud format, mesh hole filling is performed first to ensure that the three-dimensional geometric model sequence has a complete mesh structure, an improved EdgeBreaker face symbolization algorithm is used to symbolize the mesh faces of the three-dimensional geometric model sequence to obtain symbolized faces, vertex attributes are reordered after face traversal, and then the vertex attributes are compressed, and then the prediction residuals of the symbolized faces and the vertex attributes are binarized and then binary arithmetic coding is performed to achieve dynamic compression of the three-dimensional geometric model sequence.

[0190] In some embodiments, for input data that is mesh attribute, hole filling is performed on the mesh first. The processing method first detects the position of the mesh hole. If a certain edge cannot form a certain geometric primitive (usually a triangle), the edge is determined as the starting position of the boundary of the mesh hole, and moving along the boundary can obtain the vertex and edge information of all hole boundaries. A new (virtual) vertex is added in the middle, connecting all the boundaries to generate a new (virtual) mesh face, and the hole filling is completed. Since the virtual face is not in the traditional EdgeBreaker algorithm, for this case, a new face symbol is added to represent the virtual face. The improved EdgeBreaker is used to symbolize the mesh information. Due to the uncertainty of traversal in the symbolization process, the order of the vertex attributes will be affected. Therefore, the mesh face symbolization is performed first, and then the vertex attributes are reordered to maintain the correspondence between the faces and the vertices. The improved EdgeBreaker symbolization algorithm generates a symbol sequence composed of six symbols. Each symbol is binarized. Binarization converts the symbol to a 0 / 1 sequence. This conversion method first uses different mesh structures to statistically analyze the probability of each symbol, and uses unary code for binarization, assigning shorter codes to symbols with higher probabilities and longer codes to symbols with lower probabilities. Independent probability models are set for the six bits of the binary sequence. Using these probability models, adaptive binary arithmetic coding is used to encode the binary sequence bit by bit to form the final mesh face compression code stream.

[0191] In some embodiments, if the input data is point cloud attributes or vertex attributes in a mesh. First, classify different attributes into three categories, the first category is vertex attributes without inter-frame correlation, the second category is one-frame correlation attributes, and the third category is more than one-frame correlation attributes. The classification is pre-set, such as vertex normal is the first or second category, vertex coordinates are the third category, vertex color is the second category, and vertex UV coordinates are the first category. If the vertex attribute belongs to the first category, the attribute uses intra-frame prediction to complete residual calculation, intra-frame prediction refers to using the encoded data in the current frame to predict the to-be-encoded data. One method is to directly use the already encoded value as the prediction, and a more complex method is to use the average or weighted average of multiple already encoded data as the prediction. Regardless of the type, the to-be-encoded data is finally subtracted from the prediction value to obtain the residual. The residual is binarized to adapt to binary arithmetic coding. Unlike pixels in traditional video, which are in integer form, most attributes in three-dimensional data are floating-point type. Therefore, the obtained residual is mostly floating-point type, so the floating-point residual needs to be quantized first. When quantizing, the upper and lower bounds of the value range of different attributes need to be determined, which is related to different application scenarios and attribute types. For example, vertex coordinates come from a reconstructed dancer, so the upper and lower bounds of the vertex coordinate attribute can be set to + / - 10 meters, because the dancer is unlikely to move more than 10 meters. For example, vertex UV coordinates are naturally between [0, 1]. Next, according to the different accuracy requirements, the floating-point residual is quantized to obtain integer values. Here, the quantization is set according to the user's required accuracy, which can be commonly used by 8 bits, 16 bits, or any bits. The longer the number of bits, the greater the accuracy obtained, but the compression ratio will be lost. For example, assuming the interval is between [-10, 10] (vertex position), use 16 bits for quantization, then each bit can represent 0.3 millimeters. When quantizing, non-uniform quantization can also be selected, which can obtain higher accuracy in a certain region (the accuracy of the remaining part will be reduced), such as the need for high accuracy in the near range ([-5, 5]), which can be designed with a quantization function to obtain more bits in this region.

[0192] In some embodiments, if the input three-dimensional geometric model sequence is in point cloud format, the three-dimensional geometric model sequence is segmented to obtain a plurality of point cloud groups;

[0193] The three-dimensional geometric model with vertex attributes is classified according to inter-frame correlation, the inter-frame correlation attributes between each frame of the three-dimensional geometric model with vertex attributes are determined, the residual prediction of the three-dimensional geometric model with vertex attributes is performed based on the inter-frame correlation attributes, and the quantized residual is determined;

[0194] The quantized residual is binarized to obtain a binary bit string, the binary bit string is bit string encoded based on adaptive binary arithmetic coding, compressed data of the vertex attribute is generated, and compression of the three-dimensional geometric model of the vertex attribute is achieved.

[0195] In some embodiments, if the input three-dimensional geometric model sequence is in a mesh format, the three-dimensional geometric model sequence is segmented based on the three-dimensional model data type to obtain a plurality of sub-meshes; and mesh faces of the three-dimensional geometric model having face connection attributes are symbolized based on an EdgeBreaker face symbolization algorithm to obtain symbolized faces.

[0196] The three-dimensional geometric model having vertex attributes is inter-frame correlation classified to determine inter-frame correlation attributes between frames of the three-dimensional geometric model having vertex attributes, residual prediction is performed on the three-dimensional geometric model having vertex attributes based on the inter-frame correlation attributes, and quantized residual is determined.

[0197] The quantized residual or the symbolized faces are binarized to obtain a binary bit string, the binary bit string is bit string encoded based on adaptive binary arithmetic coding, compressed data of the vertex attribute and compressed data of the face connection attribute are generated, and compression of the three-dimensional geometric model of the vertex attribute and the face connection attribute is achieved.

[0198] The embodiment identifies a three-dimensional model data type and an attribute type of the three-dimensional geometric model sequence in response to a three-dimensional geometric model sequence input, the three-dimensional model data type includes a point cloud sequence and a mesh sequence, the attribute type of the point cloud sequence includes a vertex attribute, the attribute type of the mesh sequence includes a vertex attribute and a face connection attribute, the three-dimensional geometric model sequence is segmented based on the three-dimensional model data type, a plurality of sub-structures are obtained, the sub-structure includes a point cloud group and a sub-mesh, the mesh face of the three-dimensional geometric model with the face connection attribute is symbolized based on an EdgeBreaker face symbolization algorithm, a symbolized face is obtained, the three-dimensional geometric model with the vertex attribute is classified in terms of inter-frame correlation, an inter-frame correlation attribute between frames in the three-dimensional geometric model with the vertex attribute is determined, the three-dimensional geometric model with the vertex attribute is predicted based on the inter-frame correlation attribute, a quantized residual error is determined, the quantized residual error and / or the symbolized face is binarized, a binary bit string is obtained, the binary bit string is bit string encoded based on adaptive binary arithmetic coding, compressed data of the vertex attribute and / or compressed data of the face connection attribute are generated, and compression of the three-dimensional geometric model sequence is implemented. Since the embodiment identifies the data type and the attribute type of the three-dimensional geometric model sequence, the three-dimensional geometric model sequence is adaptively segmented and processed in a complex and dynamic manner, the redundancy of various attributes in frames and between frames is deeply mined without obvious quality loss, the multi-dimensional attributes of the dynamic three-dimensional geometric model sequence are effectively processed, the compression efficiency of the three-dimensional geometric sequence is greatly improved, all to-be-encoded information is binarized, the calculation amount in the entropy coding process is greatly reduced, the real-time requirement of the three-dimensional geometric model sequence compression is met, and the compression ratio and the compression efficiency are greatly improved.

[0199] In addition, the embodiment further provides a computer readable storage medium, and the computer readable storage medium stores a three-dimensional geometric model sequence compression program based on binary arithmetic coding. The three-dimensional geometric model sequence compression program based on binary arithmetic coding implements the steps of the three-dimensional geometric model sequence compression method based on binary arithmetic coding when executed by a processor.

[0200] The computer readable storage medium provided in the application may be, for example, a U disk, but is not limited to an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, system, or device, or any combination of the above. More specific examples of the computer readable storage medium may include, but are not limited to, an electrical connection with one or more conductive wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the embodiment, the computer readable storage medium may be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, system, or device. The program code contained on the computer readable storage medium can be transmitted by any suitable medium, including but not limited to an electrical wire, an optical cable, an RF (Radio Frequency), and the like, or any suitable combination of the above.

[0201] The computer readable storage medium described above may be included in the three-dimensional geometric model sequence compression device based on binary arithmetic coding, or may exist independently without being assembled into the three-dimensional geometric model sequence compression device based on binary arithmetic coding.

[0202] In addition, the embodiment of the application also provides a computer program product, including a three-dimensional geometric model sequence compression program based on binary arithmetic coding, and the three-dimensional geometric model sequence compression program based on binary arithmetic coding realizes the steps of the three-dimensional geometric model sequence compression method based on binary arithmetic coding when executed by a processor.

[0203] The computer program product embodiment of the application is basically the same as the above-mentioned three-dimensional geometric model sequence compression method based on binary arithmetic coding, and will not be repeated here.

[0204] Reference Figure 6 , Figure 6 The structure block diagram of an embodiment of the three-dimensional geometric model sequence compression device based on binary arithmetic coding of the application is shown in the figure.

[0205] As Figure 6 shown, the three-dimensional geometric model sequence compression device based on binary arithmetic coding provided in the embodiment of the application includes:

[0206] The data recognition module 10 is configured to recognize a three-dimensional model data type and an attribute type of the three-dimensional geometric model sequence in response to a three-dimensional geometric model sequence input, the three-dimensional model data type comprising a point cloud sequence and a mesh sequence, the attribute type of the point cloud sequence comprising a vertex attribute, and the attribute type of the mesh sequence comprising a vertex attribute and a face connection attribute;

[0207] The model segmentation module 20 is configured to segment the three-dimensional geometric model sequence based on the three-dimensional model data type to obtain a plurality of substructures, the substructure comprising a point cloud cluster and a sub-mesh;

[0208] The face symbolization module 30 is configured to symbolize a mesh face of the three-dimensional geometric model with the face connection attribute based on an EdgeBreaker face symbolization algorithm to obtain a symbolized face;

[0209] The inter-frame association analysis module 40 is configured to perform inter-frame association classification on the three-dimensional geometric model with the vertex attribute to determine an inter-frame association attribute between frames in the three-dimensional geometric model with the vertex attribute;

[0210] The residual prediction module 50 is configured to perform residual prediction on the three-dimensional geometric model with the vertex attribute based on the inter-frame association attribute to determine a quantized residual;

[0211] The binarization module 60 is configured to binarize the quantized residual and / or the symbolized face to obtain a binary bit string;

[0212] The binary compression module 70 is configured to perform bit string coding on the binary bit string based on adaptive binary arithmetic coding to generate compressed data of the vertex attribute and / or compressed data of the face connection attribute, and to realize compression of the three-dimensional geometric model sequence.

[0213] The embodiment identifies a three-dimensional model data type and an attribute type of the three-dimensional geometric model sequence in response to a three-dimensional geometric model sequence input, the three-dimensional model data type includes a point cloud sequence and a mesh sequence, the attribute type of the point cloud sequence includes a vertex attribute, the attribute type of the mesh sequence includes a vertex attribute and a face connection attribute, segments the three-dimensional geometric model sequence based on the three-dimensional model data type, obtains a plurality of substructures, the substructure includes a point cloud group and a sub-mesh, symbolizes a mesh face of the three-dimensional geometric model with the face connection attribute based on an EdgeBreaker face symbolization algorithm, obtains a symbolized face, performs inter-frame correlation classification on the three-dimensional geometric model with the vertex attribute, determines an inter-frame correlation attribute between frames in the three-dimensional geometric model with the vertex attribute, performs residual prediction on the three-dimensional geometric model with the vertex attribute based on the inter-frame correlation attribute, determines a quantized residual, binarizes the quantized residual and / or the symbolized face, obtains a binary bit string, performs bit string coding on the binary bit string based on adaptive binary arithmetic coding, generates compressed data of the vertex attribute and / or compressed data of the face connection attribute, and realizes compression of the three-dimensional geometric model sequence. Since the embodiment identifies the data type and the attribute type of the three-dimensional geometric model sequence, the complex and dynamic three-dimensional geometric model sequence is adaptively segmented and processed, the redundancy of various attributes in the frame and between frames is deeply mined under the premise of no obvious quality loss, the multi-dimensional attributes of the dynamic three-dimensional geometric model sequence are effectively processed, and therefore the compression efficiency of the three-dimensional geometric sequence is greatly improved. Binarizing all to-be-encoded information greatly reduces the amount of calculation in the entropy coding process, thereby meeting the real-time requirement of the three-dimensional geometric model sequence compression and greatly improving the compression ratio and the compression efficiency.

[0214] The three-dimensional geometric model sequence compression device based on binary arithmetic coding provided in the application adopts the three-dimensional geometric model sequence compression method based on binary arithmetic coding in the above embodiment, and can solve the technical problem of three-dimensional geometric model sequence compression based on binary arithmetic coding. Compared with the prior art, the three-dimensional geometric model sequence compression device based on binary arithmetic coding provided in the application has the same beneficial effects as the three-dimensional geometric model sequence compression method based on binary arithmetic coding provided in the above embodiment, and other technical features in the three-dimensional geometric model sequence compression device based on binary arithmetic coding are the same as the features disclosed in the above embodiment method, which will not be repeated here.

[0215] It should be understood that the above is only an example, and the technical solutions of the application are not limited in any way. In specific applications, those skilled in the art can set up as needed, and the application does not limit this.

[0216] It should be noted that the above-described workflow is merely illustrative and does not limit the scope of protection of the present application, and in actual application, a person skilled in the art can select part or all of them to achieve the purpose of the embodiment scheme according to actual needs, which is not limited here.

[0217] In addition, technical details not described in detail in the present embodiment can be found in the binary arithmetic coding-based three-dimensional geometric model sequence compression method provided by any embodiment of the present application, which will not be repeated here.

[0218] It should be noted that in this paper, the term "includes", "contains" or any other variant thereof is intended to cover non-exclusive inclusion, so that the process, method, article or system including a series of elements not only includes those elements, but also includes other elements not explicitly listed, or includes elements inherent to such process, method, article or system. Without more limitations, the element defined by the sentence "includes a" does not exclude the presence of other identical elements in the process, method, article or system including the element.

[0219] The above-mentioned embodiment numbers of the present application are only for description, not representing the advantages and disadvantages of the embodiments.

[0220] From the above description of the embodiments, those skilled in the art can clearly understand that the above-mentioned embodiment method can be realized by means of software and necessary general hardware platform, of course, it can also be realized by hardware, but in many cases, the former is a better embodiment. Based on such understanding, the technical solutions of the present application or the part that contributes to the prior art can be embodied in the form of software product, which is stored in a storage medium (such as read-only memory / random access memory, magnetic disk, optical disk), including a plurality of instructions for making a terminal device (which can be a mobile phone, computer, server, air conditioner, or network device, etc.) execute the method described in each embodiment of the present application.

[0221] The above is only the preferred embodiment of the present application, and does not limit the patent scope of the present application, and any equivalent structure or equivalent process transformation made by using the content of the present application specification and drawings, or directly or indirectly applied to other related technical fields, are also included in the patent protection scope of the present application.

Claims

1. A method for compressing three-dimensional geometric model sequences based on binary arithmetic coding, characterized in that, The three-dimensional geometric model sequence compression method based on binary arithmetic coding includes: In response to a sequence of 3D geometric models input, the data type and attribute type of the 3D geometric model sequence are identified. The data type of the 3D geometric model includes point cloud sequences and mesh sequences. The attribute type of the point cloud sequence includes vertex attributes. The attribute type of the mesh sequence includes vertex attributes and patch connectivity attributes. Based on the three-dimensional model data type, the three-dimensional geometric model sequence is segmented to obtain multiple substructures, the substructures including point cloud clusters and sub-mesh; The EdgeBreaker patch symbolization algorithm is used to symbolize the mesh patches of a 3D geometric model with patch connectivity properties to obtain symbolized patches. The inter-frame correlation of a 3D geometric model with vertex attributes is classified to determine the inter-frame correlation attributes between frames in the 3D geometric model with vertex attributes. Based on the inter-frame correlation attributes, residual prediction is performed on the 3D geometric model with vertex attributes to determine the quantized residual; The quantization residual and / or the symbolic patch are binaryized to obtain a binary bit string; The binary bit string is bit-string encoded based on adaptive binary arithmetic encoding to generate compressed data of vertex attributes and / or compressed data of face connection attributes, thereby achieving compression of the three-dimensional geometric model sequence. The step of performing residual prediction on the 3D geometric model with vertex attributes based on the inter-frame correlation attributes, and determining the quantized residual, includes: The rate of change of vertex attributes of the three-dimensional geometric model with vertex attributes is determined based on the inter-frame correlation attribute. The attribute values ​​of the current vertex to be compressed are predicted based on the rate of change, and the predicted attribute values ​​are obtained. The prediction residual is determined based on the predicted attribute value and the current attribute value of the current vertex to be compressed; The target quantization bit width is determined based on the value range of the predicted residual and the quantization requirements. The predicted residual is then quantized based on the target quantization bit width to obtain the quantized residual. The predicted residual is quantized using the following formula: in, Indicates the quantized residual. and These represent the lower and upper limits of the predicted residual's value range, respectively. This represents a floating-point value selected from the range of possible values. Indicates the target quantization bit width. This indicates the adjustment factor for non-uniform quantization. This represents the prediction residual to be quantified.

2. The three-dimensional geometric model sequence compression method based on binary arithmetic coding as described in claim 1, characterized in that, The segmentation of the three-dimensional geometric model sequence based on the three-dimensional model data type yields multiple sub-structures, including: The segmentation strategy is determined based on the data type of the 3D model. The three-dimensional geometric model sequence is segmented according to the segmentation strategy to obtain multiple substructures; The segmentation strategies include spatial segmentation strategies, sprawling segmentation strategies, and semantic segmentation strategies. The spatial partitioning strategy includes: Determine the three-dimensional spatial boundary of the three-dimensional geometric model of the frame to be segmented in the three-dimensional geometric model sequence, and divide the three-dimensional space where the three-dimensional geometric model is located into multiple subspaces based on the three-dimensional spatial boundary; Based on the point cloud attribute information or mesh attribute information of the three-dimensional geometric model, the three-dimensional geometric model in each subspace is segmented into point clouds or meshes to obtain multiple point cloud clusters or multiple sub-meshes. The mesh attribute information includes the vertex position information of the mesh facets and the centroid position of the mesh facets. The propagation segmentation strategy includes: Determine the number of target facets in the submesh required during the propagation process; Based on the breadth-first principle, the mesh patches in the 3D geometric model of the frame to be segmented are spread and traversed. Based on the spread and traversal results and the number of target patches, the 3D geometric model is segmented into meshes to obtain multiple sub-mesh. The semantic segmentation strategy includes: The three-dimensional geometric model of the frame to be segmented is rendered to generate a two-dimensional color image, and semantic recognition is performed on the two-dimensional color image to generate a semantic map; Based on the semantic graph, the three-dimensional geometric model is subjected to three-dimensional semantic segmentation to obtain multiple point cloud clusters or multiple sub-mesh.

3. The three-dimensional geometric model sequence compression method based on binary arithmetic coding as described in claim 2, characterized in that, The inter-frame association attribute includes a first association attribute, a second association attribute, and a third association attribute. The first association attribute is one that does not have inter-frame association attributes, the second association attribute is one that has two-frame association attributes, and the third association attribute is one that has more than two-frame association attributes. The step of predicting the attribute values ​​of the current vertex to be compressed based on the rate of change to obtain predicted attribute values ​​includes: Obtain the buffered data of the current data frame and historical data frames in the data buffer, wherein the data buffer is a pre-built buffer area with a buffer limit of more than three frames of data; A prediction function is constructed based on the buffered data of the current data frame and historical data frames. The prediction function includes: in, The prediction function representing the third associated attribute. The prediction function representing the second associated attribute. The prediction function representing the second associated attribute. , and These represent the first three frames of data in the current data frame. This refers to the data that has been encoded and decoded in the current data frame; The attribute values ​​of the current vertex to be compressed are predicted based on the prediction function to obtain the predicted attribute values.

4. The method for compressing three-dimensional geometric model sequences based on binary arithmetic coding as described in any one of claims 1 to 3, characterized in that, The EdgeBreaker patch symbolization algorithm is used to symbolize the mesh patches of a 3D geometric model with patch connectivity attributes to obtain symbolized patches, including: Perform patch traversal on each frame of the 3D geometric model with patch connectivity to detect mesh holes; During the traversal of the mesh, the position of any mesh mesh with no adjacent meshes on any side is marked as the position of the mesh hole. Virtual vertices and virtual faces are added at the locations of the mesh holes to fill the holes. A symbol library is obtained based on the EdgeBreaker patch symbolization algorithm, and the symbol library includes multiple encoded symbols; Virtual coded symbols are generated based on the positions of the grid holes, and the symbol library is updated based on the virtual coded symbols; Symbolized patches are obtained by symbolizing the mesh patches of a 3D geometric model with patch connectivity properties based on the updated symbol library.

5. The three-dimensional geometric model sequence compression method based on binary arithmetic coding as described in claim 4, characterized in that, After performing patch traversal on each frame of the 3D geometric model with patch connectivity, the process further includes: Construct a sort buffer and sort counter for vertex attributes; During the patch traversal, in response to passing a new mesh vertex, the vertex order value of the mesh vertex is recorded by the sorting counter, and a mapping relationship between the vertex attributes of the mesh vertex and the vertex order value is generated, and the mapping relationship is written into the sorting buffer; In response to the end of the face traversal, the vertex attributes of each mesh vertex are reordered based on the mapping relationship in the sorting buffer.

6. The three-dimensional geometric model sequence compression method based on binary arithmetic coding as described in claim 5, characterized in that, The step of binarying the quantization residual and / or the symbolic patch to obtain a binary bit string includes: The quantization residual is binary-coded to obtain a binary bit string; And / or, based on the symbolized patch, determine the number of times each coded symbol appears and the number of times each coded symbol co-occurs with other symbols; Divide the number of times each coded symbol co-occurs by the number of times the symbol appears to obtain the prior probability of each coded symbol. The encoded symbols are sorted based on the prior probabilities, and the symbolized patches are binary mapped based on the sorting results to obtain a binary bit string.

7. A three-dimensional geometric model sequence compression device based on binary arithmetic coding, characterized in that, The three-dimensional geometric model sequence compression device based on binary arithmetic coding includes: The data recognition module is used to identify the three-dimensional model data type and attribute type of the three-dimensional geometric model sequence in response to the input of the three-dimensional geometric model sequence. The three-dimensional model data type includes point cloud sequence and mesh sequence. The attribute type of the point cloud sequence includes vertex attribute. The attribute type of the mesh sequence includes vertex attribute and patch connection attribute. The model segmentation module is used to segment the three-dimensional geometric model sequence based on the three-dimensional model data type to obtain multiple substructures, wherein the substructures include point cloud clusters and sub-mesh; The patch symbolization module is used to symbolize the mesh patches of a 3D geometric model with patch connectivity attributes based on the EdgeBreaker patch symbolization algorithm to obtain symbolized patches; The inter-frame correlation analysis module is used to classify the inter-frame correlation of a 3D geometric model with vertex attributes and determine the inter-frame correlation attributes between frames in the 3D geometric model with vertex attributes. The residual prediction module is used to perform residual prediction on the three-dimensional geometric model with vertex attributes based on the inter-frame correlation attributes, and determine the quantized residual; A binary conversion module is used to binary convert the quantization residual and / or the symbolic patch to obtain a binary bit string; The binary compression module is used to perform bit string encoding on the binary bit string based on adaptive binary arithmetic encoding to generate compressed data of vertex attributes and / or compressed data of face connection attributes, thereby realizing the compression of the three-dimensional geometric model sequence. The residual prediction module is further configured to: determine the rate of change of vertex attributes of the 3D geometric model with vertex attributes based on the inter-frame correlation attributes; predict the attribute value of the current vertex to be compressed based on the rate of change to obtain the predicted attribute value; determine the prediction residual based on the predicted attribute value and the current attribute value of the current vertex to be compressed; determine the target quantization bit width based on the value range of the prediction residual and the quantization requirements; and quantize the prediction residual based on the target quantization bit width to obtain the quantized residual, using the following formula for quantization: in, Indicates the quantized residual. and These represent the lower and upper limits of the predicted residual's value range, respectively. This represents a floating-point value selected from the range of possible values. Indicates the target quantization bit width. This indicates the adjustment factor for non-uniform quantization. This represents the prediction residual to be quantified.

8. A three-dimensional geometric model sequence compression device based on binary arithmetic coding, characterized in that, The three-dimensional geometric model sequence compression device based on binary arithmetic encoding includes: a memory, a processor, and a three-dimensional geometric model sequence compression program based on binary arithmetic encoding stored in the memory and executable on the processor. The three-dimensional geometric model sequence compression program based on binary arithmetic encoding is configured to implement the three-dimensional geometric model sequence compression method based on binary arithmetic encoding as described in any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a three-dimensional geometric model sequence compression program based on binary arithmetic encoding, which, when executed by a processor, implements the three-dimensional geometric model sequence compression method based on binary arithmetic encoding as described in any one of claims 1 to 6.

Images