Three-dimensional geometric model sequence compression method, device and equipment based on binary arithmetic coding and storage medium

Abstract

The invention discloses a three-dimensional geometric model sequence compression method, device and equipment based on binary arithmetic coding and a storage medium, and the method comprises the steps: segmenting a three-dimensional geometric model sequence based on the three-dimensional model data type of the three-dimensional geometric model sequence, and obtaining a plurality of substructures, performing symbolization on a grid patch of the three-dimensional geometric model with the patch connection attribute, performing inter-frame relevance classification on the three-dimensional geometric model with the vertex attribute, performing residual error prediction on the three-dimensional geometric model with the vertex attribute based on the inter-frame relevance attribute, performing binarization on a quantized residual error and / or a symbolized patch to obtain a binary bit string, and performing binarization on the quantized residual error and / or the symbolized patch to obtain a binary bit string; bit string coding is carried out on the binary bit string, compression of the three-dimensional geometric model sequence is achieved, multi-dimensional attributes of the dynamic three-dimensional geometric model sequence are effectively processed, the compression ratio and the compression efficiency are greatly improved, and the timeliness requirement of the three-dimensional geometric model sequence is met.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of computer graphics processing, and particularly relates to a three-dimensional geometric model sequence compression method and device based on binary arithmetic coding, equipment and a storage medium. BACKGROUND

[0002] Computer graphics processing is an important field in computer science, and three-dimensional model processing is an indispensable part of computer graphics. In recent years, with the rapid development of computer processing power and computer vision algorithms, a new modeling method, real scene modeling, has emerged. Real scene modeling usually uses color cameras, infrared cameras, laser radars and other sensing devices to take multiple-angle photos or scan real three-dimensional objects in reality, so as to obtain three-dimensional geometric and color appearance features at each angle, and then combine the features at each angle by a three-dimensional fusion algorithm to obtain a complete model. This model can be identical to the model obtained by traditional modeling methods in terms of data structure. The advantage of real scene modeling is that it can model real objects with high efficiency (can be completed within a few seconds). Based on the reconstruction of static objects, dynamic three-dimensional real scene modeling has been developed. Dynamic three-dimensional real scene modeling is a sequence formed by multiple three-dimensional models in time sequence. However, dynamic three-dimensional real scene modeling has the characteristic of large data volume, resulting in the need for large capacity storage of three-dimensional geometric model sequences. In order to solve this problem, data compression of dynamic three-dimensional real scene models is needed. However, the data composition of three-dimensional models is very complex, usually involving attributes of different properties. The current three-dimensional model compression method cannot effectively handle the multi-dimensional attributes in three-dimensional models, resulting in low compression ratio and dynamic compression efficiency. SUMMARY

[0003] The main purpose of the present application is to provide a three-dimensional geometric model sequence compression method and device based on binary arithmetic coding, equipment and a storage medium, which aims to solve the technical problem that the prior art cannot effectively handle the multi-dimensional attributes in three-dimensional models, resulting in low compression ratio and compression efficiency.

[0004] To achieve the above purpose, the present application provides a three-dimensional geometric model sequence compression method based on binary arithmetic coding, which comprises the following steps: In response to the input of a three-dimensional geometric model sequence, the data type and attribute type of the three-dimensional model sequence are identified, the data type of the three-dimensional model includes a point cloud sequence and a mesh sequence, the attribute type of the point cloud sequence includes vertex attributes, and the attribute type of the mesh sequence includes vertex attributes and face connection attributes; The three-dimensional geometric model sequence is segmented based on the data type of the three-dimensional model, and a plurality of sub-structures are obtained, the sub-structure includes a point cloud group and a sub-mesh; The EdgeBreaker patch symbolization algorithm is used to symbolize the mesh patches of the three-dimensional geometric model with patch connection attributes, to obtain symbolized patches; The three-dimensional geometric model with vertex attributes is classified according to inter-frame correlation, to determine the inter-frame correlation attributes between frames in the three-dimensional geometric model with vertex attributes; The three-dimensional geometric model with vertex attributes is predicted based on the inter-frame correlation attributes, to determine the quantized residual error; The quantized residual error and / or the symbolized patches are binarized, to obtain a binary bit string; The binary bit string is encoded based on adaptive binary arithmetic coding, to generate compressed data of vertex attributes and / or compressed data of patch connection attributes, to realize compression of the three-dimensional geometric model sequence.

[0005] Optionally, the three-dimensional geometric model sequence is segmented based on the three-dimensional model data type, to obtain a plurality of sub-structures, including: A segmentation strategy is determined based on the three-dimensional model data type; The three-dimensional geometric model sequence is segmented according to the segmentation strategy, to obtain a plurality of sub-structures; The segmentation strategy includes a spatial segmentation strategy, a propagation segmentation strategy, and a semantic segmentation strategy; The spatial segmentation strategy includes: The three-dimensional spatial boundary of the three-dimensional geometric model of the frame to be segmented in the three-dimensional geometric model sequence is determined, and the three-dimensional space where the three-dimensional geometric model is located is segmented into a plurality of sub-spaces based on the three-dimensional spatial boundary; The three-dimensional geometric model in each sub-space is segmented based on the point cloud attribute information or the mesh attribute information of the three-dimensional geometric model, to obtain a plurality of point cloud clusters or a plurality of sub-meshes, and the mesh attribute information includes vertex position information of mesh patches and gravity center positions of mesh patches; The propagation segmentation strategy includes: The target patch number of the sub-mesh required in the propagation process is determined; Each mesh patch in the three-dimensional geometric model of the frame to be segmented is traversed based on the breadth-first principle, and the three-dimensional geometric model is segmented based on the traversal result and the target patch number, to obtain a plurality of sub-meshes; The semantic segmentation strategy includes: The three-dimensional geometric model of the frame to be segmented is rendered to generate a two-dimensional color image, and the two-dimensional color image is subjected to semantic recognition to generate a semantic map; The three-dimensional geometric model is subjected to three-dimensional semantic segmentation based on the semantic map, to obtain a plurality of point cloud clusters or a plurality of sub-meshes.

[0006] Optionally, the residual prediction of the three-dimensional geometric model with vertex attributes based on the inter-frame association attribute comprises: determining a change rate of the vertex attribute of the three-dimensional geometric model in the three-dimensional geometric model with vertex attributes based on the inter-frame association attribute; predicting the attribute value of the current vertex to be compressed according to the change rate to obtain a predicted attribute value; determining a predicted residual based on the predicted attribute value and the current attribute value of the current vertex to be compressed; determining a target quantization bit width according to the value range of the predicted residual and the quantization requirement, and quantizing the predicted residual based on the target quantization bit width to obtain a quantized residual, wherein the predicted residual is quantized according to the following formula: wherein, represents the quantized residual, and respectively represent a lower limit value and an upper limit value in the value range of the predicted residual, represents a floating-point value selected in the value range, represents the target quantization bit width, represents an adjustment factor of non-uniform quantization, represents the predicted residual to be quantized.

[0007] Optionally, the inter-frame association attribute comprises a first association attribute, a second association attribute and a third association attribute, the first association attribute is an attribute without inter-frame association, the second association attribute is an attribute with two-frame association, and the third association attribute is an attribute with more than two-frame association. The prediction of the attribute value of the current vertex to be compressed according to the change rate to obtain a predicted attribute value comprises: obtaining buffered data of the current data frame and the historical data frame in a data buffer, wherein the data buffer is a buffer area with a pre-constructed buffer upper limit exceeding three frames of data; constructing a prediction function based on the buffered data of the current data frame and the historical data frame, wherein the prediction function comprises: wherein, represents the prediction function of the third association attribute, represents the prediction function of the second association attribute, represents the prediction function of the second association attribute, , and respectively represent the first three frames of data of the current data frame, is the data in the current data frame that has been encoded and decoded; predicting the attribute value of the current vertex to be compressed based on the prediction function, to obtain a predicted attribute value.

[0008] Optionally, the EdgeBreaker patch coding algorithm is used to code the mesh patches of the three-dimensional geometric model with patch connection attributes, to obtain coded patches, including: patch traversal is performed on each frame of the three-dimensional geometric model with patch connection attributes, to detect mesh holes; any mesh patch with no adjacent patch is marked as a mesh hole position during patch traversal; a virtual vertex and a virtual patch are added at the mesh hole position to fill the hole at the mesh hole position; an encoding symbol library is obtained based on the EdgeBreaker patch coding algorithm, and the encoding symbol library includes a plurality of encoding symbols; a virtual encoding symbol is generated according to the mesh hole position, and the encoding symbol library is updated based on the virtual encoding symbol; the mesh patches of the three-dimensional geometric model with patch connection attributes are coded based on the updated encoding symbol library, to obtain coded patches.

[0009] Optionally, after the patch traversal is performed on each frame of the three-dimensional geometric model with patch connection attributes, the method further includes: an ordering buffer and an ordering counter of vertex attributes are constructed; during patch traversal, in response to passing through a new mesh vertex, a vertex order value of the mesh vertex is recorded by the ordering counter, a mapping relationship between the vertex attributes of the mesh vertex and the vertex order value is generated, and the mapping relationship is written into the ordering buffer; in response to the end of patch traversal, the vertex attributes of each mesh vertex are reordered based on the mapping relationship in the ordering buffer.

[0010] Optionally, the quantized residual and / or the coded patches are binarized to obtain a binary bit string, including: the quantized residual is binarized to obtain a binary bit string; and / or, the symbol occurrence frequency of each encoding symbol and the symbol co-occurrence frequency of each encoding symbol and other symbols are determined based on the coded patches; the symbol co-occurrence frequency of each encoding symbol is divided by the symbol occurrence frequency to obtain the prior probability of each encoding symbol. The coding symbols are sorted based on the prior probability, and the symbolized facets are binary mapped based on the sorting result to obtain a binary bit string.

[0011] In addition, to achieve the above object, the application further provides a three-dimensional geometric model sequence compression device based on binary arithmetic coding, which comprises: The three-dimensional geometric model sequence compression device based on binary arithmetic coding comprises: A data identification module is configured to identify a three-dimensional model data type and an attribute type of the three-dimensional geometric model sequence in response to a three-dimensional geometric model sequence input, wherein the three-dimensional model data type comprises a point cloud sequence and a mesh sequence, and the attribute type of the point cloud sequence comprises a vertex attribute, and the attribute type of the mesh sequence comprises a vertex attribute and a facet connection attribute. A model segmentation module is configured to segment the three-dimensional geometric model sequence based on the three-dimensional model data type to obtain a plurality of substructures, wherein the substructures comprise a point cloud cluster and a sub-mesh. A facet symbolization module is configured to symbolize mesh facets of the three-dimensional geometric model with the facet connection attribute based on an EdgeBreaker facet symbolization algorithm to obtain symbolized facets. An inter-frame association analysis module is configured to classify three-dimensional geometric models with vertex attributes based on inter-frame association to determine inter-frame association attributes between frames of the three-dimensional geometric models with vertex attributes. A residual prediction module is configured to predict residuals of the three-dimensional geometric models with vertex attributes based on the inter-frame association attributes to determine quantized residuals. A binarization module is configured to binarize the quantized residuals and / or the symbolized facets to obtain a binary bit string. A binary compression module is configured to encode the binary bit string based on adaptive binary arithmetic coding to generate compressed data of vertex attributes and / or compressed data of facet connection attributes, thereby realizing compression of the three-dimensional geometric model sequence.

[0012] In addition, to achieve the above object, the application further provides a three-dimensional geometric model sequence compression device based on binary arithmetic coding, which comprises a memory, a processor and a computer program stored in the memory and executable on the processor, wherein the computer program is configured to implement the steps of the three-dimensional geometric model sequence compression method based on binary arithmetic coding.

[0013] In addition, to achieve the above object, the application further provides a computer readable storage medium, which has a computer program stored thereon, and the computer program is executed by a processor to implement the steps of the binary arithmetic coding based three-dimensional geometric model sequence compression method.

[0014] In addition, to achieve the above object, the application further provides a computer program product, which comprises a computer program, and the computer program is executed by a processor to implement the steps of the binary arithmetic coding based three-dimensional geometric model sequence compression method.

[0015] The application identifies the three-dimensional model data type and attribute type of the three-dimensional geometric model sequence in response to the three-dimensional geometric model sequence input, the three-dimensional model data type comprises a point cloud sequence and a mesh sequence, the attribute type of the point cloud sequence comprises a vertex attribute, the attribute type of the mesh sequence comprises a vertex attribute and a face connection attribute, the three-dimensional geometric model sequence is segmented based on the three-dimensional model data type, a plurality of substructures are obtained, the substructure comprises a point cloud group and a sub-mesh, the mesh face of the three-dimensional geometric model with the face connection attribute is symbolized based on an EdgeBreaker face symbolization algorithm, a symbolized face is obtained, the three-dimensional geometric model with the vertex attribute is classified based on inter-frame correlation, the inter-frame correlation attribute between frames in the three-dimensional geometric model with the vertex attribute is determined, the three-dimensional geometric model with the vertex attribute is predicted based on the inter-frame correlation attribute, a quantized residual error is determined, the quantized residual error and / or the symbolized face is binarized, a binary bit string is obtained, the binary bit string is bit string encoded based on adaptive binary arithmetic coding, the compression data of the vertex attribute and / or the compression data of the face connection attribute are generated, and the three-dimensional geometric model sequence compression is implemented. BRIEF DESCRIPTION OF DRAWINGS

[0016] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the accompanying drawings needed to be used in the embodiments or prior art description will be briefly introduced as follows. Obviously, the drawings are only for the purpose of illustrating the embodiments of the present application, and thus the person of ordinary skill in the art can further obtain other drawings based on these drawings without any creative effort.

[0017] Figure 1 is a structural schematic diagram of a binary arithmetic coding based three-dimensional geometric model sequence compression device of a hardware running environment related to the embodiment of the present application. Figure 2 is a flow schematic diagram of a binary arithmetic coding based three-dimensional geometric model sequence compression method of an embodiment of the present application. Figure 3 is a three-dimensional geometric model sequence compression flow schematic diagram in a binary arithmetic coding based three-dimensional geometric model sequence compression method of an embodiment of the present application. Figure 4 is a schematic diagram of a virtual surface filling hole in a binary arithmetic coding based three-dimensional geometric model sequence compression method of an embodiment of the present application. Figure 5 is a flow schematic diagram of an encoding compression in a binary arithmetic coding based three-dimensional geometric model sequence compression method of an embodiment of the present application. Figure 6 is a structural block diagram of a binary arithmetic coding based three-dimensional geometric model sequence compression device of an embodiment of the present application.

[0018] The implementation, functional features and advantages of the present application will be further described with reference to the embodiments and the accompanying drawings. DETAILED DESCRIPTION

[0019] It should be understood that the specific embodiments described herein are merely intended to explain the present application, and are not intended to limit the present application.

[0020] Reference Figure 1 , Figure 1 is a structural schematic diagram of a binary arithmetic coding based three-dimensional geometric model sequence compression device of a hardware running environment related to the embodiment of the present application.

[0021] As Figure 1As shown in the figure, the binary arithmetic coding based three-dimensional geometric model sequence compression device can include a processor 1001, for example, a Central Processing Unit (CPU), a communication bus 1002, a user interface 1003, a network interface 1004, and a memory 1005. The communication bus 1002 is used to realize the connection communication between the components. The user interface 1003 can include a display screen, an input unit such as a keyboard, and can also include a standard wired interface, a wireless interface. The network interface 1004 can optionally include a standard wired interface, a wireless interface (such as a Wireless-Fidelity (WI-FI) interface). The memory 1005 can be a high-speed Random Access Memory (RAM), and can also be a stable Non-Volatile Memory (NVM), for example, a disk memory. The memory 1005 can also be a storage device independent of the aforementioned processor 1001.

[0022] Those skilled in the art can understand that Figure 1 The structure shown in the figure does not constitute a limitation on the binary arithmetic coding based three-dimensional geometric model sequence compression device, and can include more or fewer components than the figure, or combine certain components, or different component arrangements.

[0023] As Figure 1 As shown in the figure, the memory 1005 as a computer readable storage medium can include an operating system, a network communication module, a user interface module, and a binary arithmetic coding based three-dimensional geometric model sequence compression program.

[0024] In Figure 1 The binary arithmetic coding based three-dimensional geometric model sequence compression device shown in the figure, the network interface 1004 is mainly used for data communication with a network server; the user interface 1003 is mainly used for data interaction with a user; the processor 1001 and the memory 1005 in the binary arithmetic coding based three-dimensional geometric model sequence compression device of the application can be arranged in the binary arithmetic coding based three-dimensional geometric model sequence compression device, and the binary arithmetic coding based three-dimensional geometric model sequence compression device calls the binary arithmetic coding based three-dimensional geometric model sequence compression program stored in the memory 1005 through the processor 1001, and executes the binary arithmetic coding based three-dimensional geometric model sequence compression method provided by the embodiment of the application.

[0025] The embodiment of the application provides a binary arithmetic coding based three-dimensional geometric model sequence compression method, which is described with reference to Figure 2 , Figure 2 A flowchart of an embodiment of the method for compressing a sequence of three-dimensional geometric models based on binary arithmetic coding.

[0026] In this embodiment, the method for compressing a sequence of three-dimensional geometric models based on binary arithmetic coding comprises the following steps: Step S10: In response to input of the sequence of three-dimensional geometric models, identify the three-dimensional model data type and attribute type of the sequence of three-dimensional geometric models.

[0027] It should be noted that the present embodiment is applied to compress a sequence of three-dimensional geometric models, and solves the problems of large data volume, difficult network transmission, and large storage resource occupation by efficiently compressing the sequence of three-dimensional geometric models.

[0028] The main method for compressing a sequence of three-dimensional geometric models in the form of point cloud in the prior art is to project the point cloud onto multiple two-dimensional planes to form a two-dimensional video sequence of multiple views, and then use mature two-dimensional video coding for compression. This method is only applicable to point cloud, and the entire process has a large amount of calculation and obvious quality loss. Therefore, this method is usually only used in static three-dimensional geometric model compression.

[0029] The focus of the prior art for compressing a sequence of three-dimensional geometric models in the form of a mesh composed of triangular faces is generally on how to symbolize the connection information of the triangular faces, and little attention is paid to how to efficiently encode the symbol information. The mainstream method mainly uses Huffman coding or arithmetic coding to encode and decode the symbol sequence. Although these methods can achieve the purpose of reducing data volume, the effect is very limited (i.e., the compression ratio before and after compression is not high), and they do not consider or have the ability of real-time encoding and decoding. The compression of vertex attributes in the sequence of three-dimensional geometric models in the form of a mesh is considered even less, and generally only the simple difference between the front and back frames is made for the vertex position attributes, and then Huffman coding or arithmetic coding is used for the difference. This compression method has the same problem as the compression of face information, i.e., it cannot achieve high efficiency and real-time performance.

[0030] It should be understood that the execution subject of the present embodiment can be a computing service device with data processing, network communication, and program running functions, such as a tablet computer, a personal computer, a mobile phone, or a terminal electronic device capable of realizing the above functions. The following takes a three-dimensional geometric model sequence compression device based on binary arithmetic coding (referred to as a compression device) as an example to describe the present embodiment and each of the following embodiments.

[0031] It should be noted that the compression device based on binary arithmetic coding is described with reference to Figure 3 , Figure 3This is a schematic diagram of the 3D geometric model sequence compression process in one embodiment of the present invention. This embodiment identifies the format and attributes of the model sequence, then segments the 3D geometric model in point cloud format / mesh format, and then binarizes the segmented model to achieve deep mining of the redundancy of various attributes within and between frames without obvious quality loss, thereby greatly improving the compression efficiency of the 3D geometric sequence. Binarization of all information to be encoded significantly reduces the amount of computation in the entropy coding process, thereby meeting real-time requirements.

[0032] It should be noted that the three-dimensional model data type includes a point cloud sequence and a mesh sequence. The attribute type of the point cloud sequence includes vertex attributes, and the attribute type of the mesh sequence includes vertex attributes and facet connection attributes.

[0033] It should be noted that mesh sequences and point cloud sequences are two common formats for representing 3D geometric model sequences. The mesh format is a model sequence composed of 3D models with mesh attributes, while the point cloud format is a model sequence composed of 3D models with point cloud attributes. The mesh format contains both vertex attributes and patch connectivity attributes, while the point cloud format only contains vertex attributes.

[0034] Step S20: Segment the three-dimensional geometric model sequence based on the three-dimensional model data type to obtain multiple substructures.

[0035] It should be noted that the substructure includes a point cloud cluster and a subgrid.

[0036] In some embodiments, the compression device segments the 3D geometric sequence, dividing a 3D geometric model in point cloud format into multiple sub-point cloud clusters or a 3D geometric model in grid format into multiple sub-grids, and independently processes and encodes the sub-3D structures. The segmentation method can be a fixed spatial partitioning method or a spatial spreading method, or it can be a 3D semantic recognition method based on the semantics of the object and then segmentation based on the recognized semantics, or a combination of both.

[0037] Furthermore, in order to improve the efficiency of model segmentation and provide appropriate segmentation strategies for three-dimensional geometric models with different format attributes, in some embodiments, step S20 may include: Step S201: determining a segmentation strategy based on the 3D model data type; Step S202: Segment the three-dimensional geometric model sequence according to the segmentation strategy to obtain multiple substructures.

[0038] It should be noted that the segmentation strategies include spatial segmentation strategy, spreading segmentation strategy and semantic segmentation strategy; The space segmentation strategy comprises: determining a three-dimensional space boundary of a three-dimensional geometric model of a frame to be segmented in the sequence of three-dimensional geometric models, and segmenting a three-dimensional space where the three-dimensional geometric model is located into a plurality of subspaces based on the three-dimensional space boundary; performing point cloud segmentation or mesh segmentation on the three-dimensional geometric model in each subspace based on point cloud attribute information or mesh attribute information of the three-dimensional geometric model, to obtain a plurality of point cloud clusters or a plurality of sub-meshes, wherein the mesh attribute information comprises vertex position information of a mesh patch and a gravity center position of the mesh patch. The propagation segmentation strategy comprises: determining a target patch number of a sub-mesh required in a propagation process; performing propagation traversal on each mesh patch in the three-dimensional geometric model of the frame to be segmented based on a breadth-first principle, and performing mesh segmentation on the three-dimensional geometric model based on a propagation traversal result and the target patch number, to obtain a plurality of sub-meshes. The semantic segmentation strategy comprises: rendering the three-dimensional geometric model of the frame to be segmented to generate a two-dimensional color image, and performing semantic recognition on the two-dimensional color image to generate a semantic map; performing three-dimensional semantic segmentation on the three-dimensional geometric model based on the semantic map, to obtain a plurality of point cloud clusters or a plurality of sub-meshes.

[0039] It can be understood that the compression device performs regional segmentation on three-dimensional data, and then performs predictive coding on the segmented data. If the input three-dimensional format is a point cloud, the segmentation only classifies the point cloud into a plurality of point cloud clusters in form. If the input three-dimensional format is a mesh, the mesh is physically split into sub-meshes, and the splitting process does not cut the patch edge, but copies the patch edge to maintain the structure of the mesh patch. In this process, first, the patch edge to be split is determined, the numbers of the two end vertices are found from the edge, and a new vertex number is added to point to the same vertex attribute. The method of splitting the patch can be one of space segmentation, propagation segmentation, and semantic segmentation according to different requirements and the structure of the model.

[0040] It should be understood that in the space method, in view of the characteristics that the three-dimensional model is large, complex, and has no subjective structure, the upper and lower boundaries of the three-dimensional model in the X, Y, and Z directions in the three-dimensional space are first determined in the segmentation process, thereby forming a cuboid space that wraps the three-dimensional model. The space is equally segmented in the X, Y, and Z dimensions to form a plurality of continuous and closely connected subspaces.

[0041] For three-dimensional data in the form of a point cloud, the point cloud in each subspace forms an independent point cloud cluster, thereby realizing segmentation of the point cloud.

[0042] For three-dimensional data in the form of a mesh, first, the gravity center position of the patch is calculated using the vertex position information of the patch, and the sub-mesh formed by the gravity center of the patch in each subspace, thereby realizing segmentation of the mesh patch.

[0043] The propagation method is mainly suitable for three-dimensional data in grid form. First, the maximum number of patches required in the propagation process is determined. Propagation starts with an arbitrary patch. The breadth-first principle is used in the propagation, that is, all patches connected to the same vertex are traversed first, and untraversed vertices are recorded. When the patches of a vertex are all accessed, the oldest accessed patch is selected to repeat the same operation. A counter is maintained during the traversal process. When the counter reaches the upper limit of the target sub-grid, all patches in this round of traversal form a segmented sub-grid, and the counter is reset. This method is used until all patches are accessed.

[0044] The semantic segmentation method is suitable for objects with natural semantic features. This method uses image recognition means to segment three-dimensional objects with semantic features, such as identifying the head, hands, thighs, and body of a human three-dimensional geometric model, and then segmenting each part on the three-dimensional model.

[0045] The specific process is as follows. First, render the three-dimensional model to generate a two-dimensional color image, and project the patch or point cloud numbers to the forming plane to generate a number map. Apply a semantic recognition algorithm to the rendered two-dimensional image to form a semantic map. Using the semantic map and the number map, classify the vertex numbers (for point clouds) or patch numbers (for grids) according to different semantics, thereby achieving three-dimensional semantic segmentation. The segmented sub-point cloud or sub-grid will be independently predicted and encoded.

[0046] Step S30: Based on the EdgeBreaker patch symbolization algorithm, the grid patches of the three-dimensional geometric model with patch connection attributes are symbolized to obtain symbolized patches.

[0047] In a specific implementation, the compression device directly loads the patch connection attributes in the grid data into the system memory. For a grid with holes, first perform virtual filling, then symbolize the grid structure, and finally binary the symbol representing the patch.

[0048] In some embodiments, if the sequence of three-dimensional geometric models to be compressed is a sequence in point cloud format, step S30 is skipped and step S40 is directly executed; if the sequence of three-dimensional geometric models to be compressed is a sequence in grid format, step S30 is directly executed to compress the patch connection attributes, and then the vertex attributes are compressed. This is because the three-dimensional geometric model in point cloud format only has vertex attributes and does not need to be compressed. The three-dimensional geometric model in grid format contains both vertex attributes and patch connection attributes, so the patch connection attributes need to be compressed first, and then the vertex attributes are compressed.

[0049] In some embodiments, when compressing the mesh format three-dimensional geometric model by the compression device, the patch connection attribute is compressed first, then the vertex attribute is reordered, and after the reordering, the vertex attribute is compressed, so as to avoid the problem of loss or disorder of the vertex attribute in the compression process of the mesh format three-dimensional geometric model, and lossless compression is realized.

[0050] Further, in order to effectively solve the problem that the mesh format three-dimensional geometric model cannot be patch coded due to the existence of holes, and improve the compression efficiency of the mesh model, in some embodiments, the step S30 can include: Step S301: patch traversal is performed on each frame three-dimensional geometric model in the three-dimensional geometric model with patch connection attribute, so as to detect the mesh hole; Step S302: the position of the mesh patch without adjacent patch on any side is marked as the mesh hole position in the patch traversal process; Step S303: a virtual vertex and a virtual patch are added at the mesh hole position to fill the hole at the mesh hole position; Step S304: a symbol library is obtained based on the EdgeBreaker patch coding algorithm, and the symbol library includes a plurality of coding symbols; Step S305: a virtual coding symbol is generated according to the mesh hole position, and the symbol library is updated based on the virtual coding symbol; Step S306: the mesh patch of the three-dimensional geometric model in the three-dimensional geometric model with patch connection attribute is coded based on the updated symbol library, and a coded patch is obtained.

[0051] It can be understood that the EdgeBreaker mesh face symbolization method has high efficiency and is often used as a symbolization method, but the method assumes that the three-dimensional mesh structure has no holes, which is almost impossible to guarantee in reality. The present application provides a solution to the problem of using EdgeBreaker as a symbolization method when encountering a three-dimensional mesh model with holes. On the basis of the five symbol libraries of "C", "L", "E", "S", and "R" defined in the EdgeBreaker algorithm, a new symbol "X" is added to represent a virtual face. In the symbolization process, first detect the mesh hole, and the detection method is as follows: if any side of the current face has no adjacent face, then the side is determined as the boundary of the hole, and the face is determined as the mesh hole boundary face. If the hole is detected, add a virtual vertex (new vertex number) and connect all the boundaries of the hole, which may generate multiple virtual faces and complete the filling of the mesh hole. In the face symbolization process, if a face containing a previously added virtual vertex is encountered during the iteration process, the face is also determined as a virtual face and symbolized using "X". If no hole is detected, i.e. the face belongs to the internal face of the mesh, the standard EdgeBreaker symbolization method is called. In the reverse symbolization process at the decoding end, if "X" is encountered, no new vertex is generated, and if other symbols are encountered, the standard EdgeBreaker reverse symbolization process is used.

[0052] It can be understood that the compression device pre-processes the mesh with holes before compressing the three-dimensional geometric model data in the form of a mesh. Specifically, when traversing the mesh patches, if any side of the patch has no adjacent patch, it is determined as a mesh hole, a virtual vertex is added in the hole, and the connected mesh is added with a virtual vertex and a virtual patch to fill the hole, as shown in FIG. 1. Figure 4 , Figure 4 FIG. 1 is a schematic diagram of a virtual face filling hole in an embodiment; in the decoding process, the virtual vertex and the virtual patch are identified and discarded, thereby restoring the original mesh structure.

[0053] For the added virtual patch, a new symbol is added to the patch symbolization algorithm to represent the virtual patch. In the encoding process, if a virtual patch is encountered, the new symbol is used for symbolization, and in the reverse symbolization process, if the symbol is encountered, the patch is ignored.

[0054] Further, in order to avoid the vertex attribute disorder of the three-dimensional geometric model in mesh format, ensure that the correspondence between the vertex attribute and the mesh patch vertex number is not affected by compression, and ensure data integrity, in some embodiments, after the above step S301, the method can further include: Step S3011: constructing a sorting buffer of vertex attributes and a sorting counter; Step S3012: In the process of patch traversal, in response to passing a new mesh vertex, record the vertex order value of the mesh vertex by means of the sorting counter, and generate a mapping relationship between the vertex attribute of the mesh vertex and the vertex order value, and write the mapping relationship into the sorting buffer; Step S3013: In response to the end of patch traversal, reorder the vertex attributes of each mesh vertex based on the mapping relationship in the sorting buffer.

[0055] It can be understood that when compressing the mesh format three-dimensional geometric model, the vertex attributes need to be reordered, because the mesh format three-dimensional geometric model has both patch connection attributes and vertex attributes, and the compression device reorders the vertex attributes based on the correspondence between the vertex number and the vertex attribute in the mesh patch after compression and decompression, which specifically includes: 1. Reordering must occur before vertex attribute compression; 2. Establish a vertex number mapping list and a counter, and allocate buffer space for later vertex attribute reordering; 3. In the process of mesh symbolization, when traversing the mesh patch, each time a new vertex is passed, the counter value and the original vertex number are recorded in the table at the same time, and then the counter is incremented by 1; 4. At the end of symbolization, use the value pair in the table to read the corresponding data from the vertex attribute with the original vertex number as the index, and write the data into the reordered buffer with the new vertex number in the second column as the index; 5. After all vertex data migration is completed, exchange the memory addresses of the original vertex attribute storage area and the reordered buffer.

[0056] In some embodiments, in order to avoid compression loss, a processing method is provided to maintain the correspondence between vertex attributes and mesh patch vertex numbers from being affected by compression before the three-dimensional geometric model with patch connection attributes is compressed. To achieve this purpose, the compression device first compresses and decompresses the patch connection attributes (hereinafter referred to as S attributes) and then reorders the vertex attributes (hereinafter referred to as V attributes). A vertex number mapping list with two columns and a counter are maintained during the processing, and the counter is initialized to 1. In the process of symbolizing S attributes, when a new patch is encountered, if any vertex of the patch has not appeared in the list, the vertex number in the S attributes is first recorded in the first column of the list, and the value of the counter is recorded in the second column, and then the counter is incremented by 1. If the vertex number has appeared in the list, no additional processing is performed. This method is followed until each patch is symbolized. Next, the vertex number mapping list is used to reorder the vertex attributes. A reordering buffer area is first created, which is comparable in size to the current vertex attributes. Each column of the list is processed in a loop, the value in the first column of the list is used to locate the V attribute element as the copy source, the value in the second column of the list is used to locate the reordering buffer area as the copy target, and the V attribute is copied from the source to the target. Finally, the reordering buffer and the V attribute storage area are address swapped. The swapped area can be retained for the next reordering process without the need to apply for system storage resources in advance.

[0057] Step S40: Inter-frame correlation classification of the three-dimensional geometric model with vertex attributes is performed to determine the inter-frame correlation attributes between the frames of the three-dimensional geometric model with vertex attributes.

[0058] In some embodiments, the inter-frame correlation attributes include a first correlation attribute, a second correlation attribute, and a third correlation attribute, the first correlation attribute is an attribute without inter-frame correlation, the second correlation attribute is an attribute with two-frame correlation, and the third correlation attribute is an attribute with more than two-frame correlation.

[0059] In some embodiments, for vertex attributes, according to the inter-frame correlation of the attributes, three types are divided: one, an attribute with more than two-frame correlation (referred to as V3 type), two, an attribute with two-frame correlation (referred to as V2 type), and three, an attribute without inter-frame correlation (referred to as V1 type). For any vertex attribute with multi-channel characteristics, such as three-channel vertex color and three-channel XYZ vertex coordinate, the data is loaded into the system memory in a channel-priority manner. This storage method optimizes the access to the memory in the later encoding process and improves the real-time efficiency.

[0060] Step S50: Residual prediction is performed on the three-dimensional geometric model with vertex attributes based on the inter-frame correlation attributes to determine the quantized residual.

[0061] In a specific implementation, the compression device can determine the quantized residual based on the inter-frame associated attribute and the corresponding residual prediction strategy selection: For a three-dimensional geometric model with vertex attributes, the inter-frame associated characteristics are determined, a suitable change rate is designed based on the inter-frame associated characteristics, the current vertex attribute to be compressed is predicted using the change rate, the difference between the predicted attribute value and the current attribute value, i.e., the predicted residual, is taken to perform encoding, and the target quantization bit width is selected according to the value range of the predicted residual and the quantization requirement for the residual quantization.

[0062] For a three-dimensional geometric model with face connection attributes, a new symbol is added to the EdgeBreaker face symbolization algorithm to represent a virtual face, and the modified symbolization algorithm is used to symbolize the mesh faces.

[0063] Further, in order to accurately predict the vertex attribute residual, in some embodiments, the above step S50 can include: Step S501: determining the change rate of the vertex attribute of the three-dimensional geometric model in the three-dimensional geometric model with vertex attributes based on the inter-frame associated attribute; Step S502: predicting the attribute value of the current vertex to be compressed according to the change rate to obtain a predicted attribute value; Step S503: determining a predicted residual based on the predicted attribute value and the current attribute value of the current vertex to be compressed; Step S504: determining a target quantization bit width according to the value range of the predicted residual and the quantization requirement, and quantizing the predicted residual based on the target quantization bit width to obtain a quantized residual.

[0064] It should be noted that there are many types of vertex attributes, such as common position, color. Since it is a sequence, there is a change rate between frames. For example, the color of the first frame is 254, and the color of the same vertex in the second frame is 255, so the change here is 1. A three-dimensional geometric structure is usually composed of hundreds of thousands or millions of vertices, and compression is processed one by one, so the vertex that has not been processed is called the current vertex to be compressed.

[0065] It should be noted that the inter-frame associated attribute includes a first associated attribute (referred to as V1 type), a second associated attribute (referred to as V2 type), and a third associated attribute (referred to as V3 type), the first associated attribute is an attribute without inter-frame association, the second associated attribute is an attribute with two-frame association, and the third associated attribute is an attribute with more than two-frame association.

[0066] The application provides a method for predicting residual binary coding of three-dimensional data. For V3 type, such as three-dimensional model motion, speed or acceleration is generated, which is directly reflected on the vertex coordinate attribute. In order to further improve the compression ratio, the current V3 attribute is predicted, and the prediction method is as follows: for the vertex attribute numerical type of floating point type data, the pre-processing of quantization is carried out, and the process of the processing is to convert the floating point type data into an integer type. The pre-quantization method is as follows: first, the upper and lower value boundaries and are determined and H, for the attribute with the default upper and lower value boundaries, such as the vertex UV coordinate (given in the [0, 1] interval), the given upper and lower value boundaries are directly used. The target quantization length bit is provided to the system by the outside world, the longer the length, the smaller the precision loss, but the larger the data quantity, and vice versa. All values in the range of to are mapped to 0 to -1. The specific mapping (and reflection) scheme can be designed according to the required precision distribution, for example, if there is no special requirement, the uniform mapping can be selected: the quantized value of the given floating point value (belonging to the range of , ) is: The inverse quantized value is: Select an appropriate quantization length for each vertex attribute , write the value of into the output code stream, and first transmit , and then select the upper and lower value ranges of the vertex attribute residual and . For the vertex attribute with inherent range, the inherent range is directly selected as and , for the vertex attribute which is not common and does not have inherent range, the user provides and , for the common vertex attribute which does not have inherent range, the default and are selected. For the input residual value exceeding the range of , , the residual is truncated to or . Take , , as parameters, and select an appropriate quantization function​​ The residual is quantized so that the output value is completely within the range of 0 to -1. The prediction residual is non-uniformly quantized with reference to the following formula: wherein, represents the quantized residual, and represent the lower limit value and the upper limit value in the value range of the prediction residual, respectively, represents the floating-point value selected in the value range, represents the target quantization bit width, represents the adjustment factor of non-uniform quantization (a higher precision quantization in a specified area can be obtained by adjusting the value), represents the prediction residual to be quantized.

[0067] The binary coding mode for the quantized vertex attribute residual: the positive and negative attributes of the residual are extracted, and a binary probability model is set for the sign bit (positive = 0, negative = 1); the absolute value of the residual is taken to obtain the absolute residual amount, and the absolute residual amount is less than M. The case is binary using unary code. For the case where the absolute residual amount is greater than M, the part less than M is binary using unary code, and the part greater than M is coded using the Golomb code. A separate probability model is set for each bit of the unary code binary part, and the probability model used is updated when a bit is coded, i.e. the probability of 0 or 1 is redistributed, so as to achieve the purpose of adaptive coding in the coding process. The part coded by Golomb code is coded using a fixed and non-updated uniform distribution probability model.

[0068] Further, in order to improve the accuracy and efficiency of residual prediction, a suitable prediction strategy is selected for different inter-frame associated attributes. The above step S502 can include: Step S5021: obtaining the buffered data of the current data frame and the historical data frame in the data buffer, wherein the data buffer is a pre-constructed buffer area with a buffer upper limit exceeding three frames of data; Step S5022: constructing a prediction function based on the buffered data of the current data frame and the historical data frame; Step S5023: predicting the attribute value of the current vertex to be compressed based on the prediction function to obtain a predicted attribute value.

[0069] It should be noted that the present application also proposes a method for systematically predicting and obtaining residual errors of vertex attributes in three-dimensional data. For vertex attributes, according to the inter-frame correlation of the attributes, they are divided into three types: attributes with more than two frames of correlation (i.e. third correlation attribute, referred to as V3 type); attributes with two frames of correlation (i.e. second correlation attribute, referred to as V2 type); and attributes without inter-frame correlation (i.e. first correlation attribute, referred to as V1 type).

[0070] It can be understood that the compression device opens at least three frame data buffer areas to save the historical compressed and decompressed attribute data during the encoding process, which are referred to as 0, 1 and 2 buffer positions. When a frame of data is encoded, the data in the 0 position and the 1 position are exchanged, and then the data in the 1 position and the 2 position are exchanged, and finally the current encoding data is decompressed and restored and placed in the 0 position. If a higher order prediction method is used, N buffer areas are opened, and the nth position and the n+1 position are exchanged in order from n=0, and then incremented. The prediction module uses the buffer data to predict and encode the V3 type and V2 type attributes, and uses the prediction function to predict the current frame to be encoded from the reconstructed frame filled in the buffer , and uses the prediction function to generate the prediction value of the current frame , and takes the prediction difference of the vertex attribute as the prediction residual error , that is .

[0071] In some embodiments, the compression device buffers the current frame data in the buffer area and the reconstructed data of the previous three frames , , . The specific form of the prediction function of the V3 type attribute is , the specific form of the prediction function of the V2 type attribute is , and the form of the prediction function of the attribute without inter-frame correlation is . , is the already encoded and decoded data of the same frame. Here and functions respectively predict the acceleration and speed of the change, and implements intra-frame prediction.

[0072] One implementation is to assume that the acceleration of the change is constant, i.e. the acceleration is constant, and the prediction function is: One implementation is to assume that the speed of the change is constant, and the prediction function is: A sort of The implementation method assumes that adjacent attributes are correlated under the same frame condition, and the prediction function is: That is, since the data of the V3 type attribute cache covers the V2 and V1 types, the V3 type can choose to use or Prediction function. Since the data cached by V2 type attributes covers V1 type, V2 type attributes can choose to use Prediction function.

[0073] in, represents the prediction function of the third associated attribute, represents the prediction function of the second associated attribute, represents the prediction function of the second associated attribute, 、 and Respectively represent the first three frames of data of the current data frame, The encoded and decoded data in the current data frame.

[0074] The predicted attribute value and the original attribute value of the current frame are formed into arrays with the same number of channels, and the sum of the differences of each array is calculated to determine when to use multi-mode prediction (i.e., using , , The difference evaluation can be implemented using the sum of absolute differences (SAD) or mean square error (MSE) method. For V3 and V2 type data, the prediction mode with the lowest difference evaluation value is selected as the final prediction model based on the accuracy of the prediction results under multiple modes. For multi-channel data, each channel can complete the prediction independently, that is, the output bitstream must be written into the prediction mode of each channel, or the same prediction mode can be applied to multiple channels. Whether to use multi-channel consistent prediction can be determined based on the actual application. The multi-channel prediction mode and the rate of change prediction mode are first entered into the bitstream by the encoder. When decoding at the receiving end, the decoder parses the bitstream. Since the prediction mode type is decoded first, the decoding system uses the prediction mode to repeat the same prediction method as that during encoding and obtains the prediction value. Then parse and decode the residual compressed data following the code stream and obtain . For the predicted value , use the residual to compensate and get the original compressed frame .

[0075] Step S60: binarize the quantized residual and / or the symbolized patch to obtain a binary bit string.

[0076] In a specific implementation, the compression device uses unary coding to binarize all symbols in the face patch symbolization calculation. A plurality of types of three-dimensional geometry sequence data sets are used for symbolization processing, the probability of occurrence of each symbol (i.e., prior probability) is counted, and the probability of occurrence of any other symbol under the condition that a certain symbol is known to occur (i.e., conditional probability) is counted.

[0077] According to the order from large to small of the prior probability, a unary code from short to long is assigned; for each bit of the unary code, a probability model for binary arithmetic coding is set; according to the statistical results, a symbol with conditional probability characteristics, i.e., a symbol whose conditional probability and prior probability differ greatly, is set with an additional conditional probability model. The statistical values are used to initialize all probability models.

[0078] The model will update the probability model or the conditional probability model corresponding thereto according to its position when ending the coding of a bit according to the arithmetic coding algorithm. If a reset frame is encountered, the statistical values are used to initialize all probability models.

[0079] Further, in order to improve the binarization efficiency of the symbolized face patch, in some embodiments, the above step S60 can include: Step S601: Binarizing the quantized residual to obtain a binary bit string; and / or, Step S602: Determining the symbol occurrence frequency of each coding symbol and the symbol co-occurrence frequency of each coding symbol with other symbols based on the symbolized face patch; Step S603: Dividing the symbol co-occurrence frequency of each coding symbol by the symbol occurrence frequency to obtain the prior probability of each coding symbol; Step S604: Sorting the coding symbols based on the prior probability, and performing binary mapping on the symbolized face patch based on the sorting result to obtain a binary bit string.

[0080] It can be understood that the present application provides a binary arithmetic coding method for symbolized geometric patches. Binary coding is performed on the symbolized patches using unary code, and six symbols are mapped to six binary bit strings of "0", "10", "11000", "11100", "11110", and "11111". The specific mapping relationship is determined by probability statistics. The above symbolization processing is performed on three-dimensional geometric sequence data sets in various scenarios, the number of occurrences of each symbol is counted, and the probability (prior probability) of the occurrence of each symbol is obtained by dividing the total number of symbols. The number of occurrences of any other symbol under the condition that a certain symbol M occurs (i.e. the co-occurrence number of symbols) is also counted, for example, for the symbol "L", the number of occurrences of "CL", "EL", "SL", "RL", "XL", and "LL" (i.e. the number of occurrences of "L" under the condition that "C", "E", "S", "R", "X", and "L" occur) is counted. The number is divided by the number of occurrences of each symbol to obtain the conditional probability of symbol M in the occurrence of any symbol. The six symbols are sorted in descending order of probability value, and the symbol types are then mapped to the above six binary bit string sequences in turn.

[0081] The binary coding method for symbols: multiple probability models are set for five-bit positions respectively, and each model tracks the probability and conditional probability of a certain symbol, i.e. the distribution and conditional distribution of each bit 0 and 1. If statistics show that the conditional probability of a certain symbol is significantly different from its probability, it means that the symbol has statistical correlation with the conditioned symbol, and the conditional probability model is switched to for coding when coding. When a bit position is coded, the corresponding probability model is updated, i.e. the probability of 0 or 1 is redistributed, so as to achieve the purpose of adaptive coding in the coding process.

[0082] Step S70: bit string coding is performed on the binary bit string based on adaptive binary arithmetic coding, to generate compressed data of vertex attributes and / or compressed data of patch connection attributes, and to realize compression of the three-dimensional geometric model sequence.

[0083] In a specific implementation, the compression device first identifies the attributes of the input three-dimensional geometric model sequence data, selects a corresponding compression strategy based on the data attributes, and refers to the compression strategy selection table shown in FIG. 6. Figure 5 Figure 5 The flowchart of the coding and compression in an embodiment is shown in FIG. 7. The compression device first determines whether the input three-dimensional geometric model is in point cloud format: ​If the input data is in point cloud format, vertex attribute reordering is performed, inter-frame correlation of vertex attributes is identified, a corresponding residual prediction strategy is selected based on inter-frame correlation attribute of the three-dimensional geometric model sequence, predicted vertex attribute values are obtained, prediction residuals are determined based on the vertex attribute values and the compressed vertex attribute values, the prediction residuals are binarized, and then binary arithmetic coding is performed to achieve dynamic compression of the three-dimensional geometric model sequence.

[0084] If the input data is not in point cloud format, mesh hole filling is performed first to ensure that the three-dimensional geometric model sequence has a complete mesh structure, an improved EdgeBreaker face symbolization algorithm is used to symbolize the mesh faces of the three-dimensional geometric model sequence to obtain symbolized faces, vertex attributes are reordered after face traversal, and then the vertex attributes are compressed, and then the prediction residuals of the symbolized faces and the vertex attributes are binarized and then binary arithmetic coding is performed to achieve dynamic compression of the three-dimensional geometric model sequence.

[0085] In some embodiments, for input data that is mesh attribute, hole filling is performed on the mesh first. The processing method first detects the position of the mesh hole. If a certain edge cannot form a certain geometric primitive (usually a triangle), the edge is determined as the starting position of the boundary of the mesh hole, and moving along the boundary can obtain the vertex and edge information of all hole boundaries. A new (virtual) vertex is added in the middle, connecting all the boundaries to generate a new (virtual) mesh face, and the hole filling is completed. Since the virtual face is not in the traditional EdgeBreaker algorithm, for this case, a new face symbol is added to represent the virtual face. The improved EdgeBreaker is used to symbolize the mesh information. Due to the uncertainty of traversal in the symbolization process, the order of the vertex attributes will be affected. Therefore, the mesh face symbolization is performed first, and then the vertex attributes are reordered to maintain the correspondence between the faces and the vertices. The improved EdgeBreaker symbolization algorithm generates a symbol sequence composed of six symbols. Each symbol is binarized. Binarization converts the symbol to a 0 / 1 sequence. This conversion method first uses different mesh structures to statistically analyze the probability of each symbol, and uses unary code for binarization, assigning shorter codes to symbols with higher probabilities and longer codes to symbols with lower probabilities. Independent probability models are set for the six bits of the binary sequence. Using these probability models, adaptive binary arithmetic coding is used to encode the binary sequence bit by bit to form the final mesh face compression code stream.

[0086] In some embodiments, if the input data is point cloud attributes or vertex attributes in a mesh. First, classify different attributes into three categories, the first category is vertex attributes without inter-frame correlation, the second category is one-frame correlation attributes, and the third category is more than one-frame correlation attributes. The classification is pre-set, such as vertex normal is the first or second category, vertex coordinates are the third category, vertex color is the second category, and vertex UV coordinates are the first category. If the vertex attribute belongs to the first category, the attribute uses intra-frame prediction to complete residual calculation, intra-frame prediction refers to using the encoded data in the current frame to predict the to-be-encoded data. One method is to directly use the already encoded value as the prediction, and a more complex method is to use the average or weighted average of multiple already encoded data as the prediction. Regardless of the type, the to-be-encoded data is finally subtracted from the prediction value to obtain the residual. The residual is binarized to adapt to binary arithmetic coding. Unlike traditional video, in which pixels are in integer form, in three-dimensional data, most attributes are in floating-point form. Therefore, the obtained residual is mostly in floating-point form, so it is necessary to first quantize the floating-point residual. When quantizing, the upper and lower bounds of the value range of different attributes need to be determined, which is related to different application scenarios and attribute types. For example, vertex coordinates come from a reconstructed dancer, so the upper and lower bounds of the vertex coordinate attribute can be set to + / - 10 meters, because the dancer is unlikely to move more than 10 meters. For example, vertex UV coordinates are naturally between [0, 1]. Next, according to the different accuracy requirements, the floating-point residual is quantized to obtain integer values. Here, the quantization is set according to the user's required accuracy, which can be commonly used by 8 bits, 16 bits, or any bits. The longer the number of bits, the greater the accuracy obtained, but the compression ratio will be lost. For example, assuming that the interval is between [-10, 10] (vertex position), 16 bits are used for quantization, and each bit can represent 0.3 millimeters. When quantizing, non-uniform quantization can also be selected, which can obtain higher accuracy in a certain region (the accuracy of the remaining part will be reduced), such as the need for high accuracy in the near range ([-5, 5]), which can be designed with a quantization function to obtain more bits in this region.

[0087] In some embodiments, if the input three-dimensional geometric model sequence is in point cloud format, the three-dimensional geometric model sequence is segmented to obtain a plurality of point cloud groups; The three-dimensional geometric model with vertex attributes is classified according to inter-frame correlation, the inter-frame correlation between each frame of the three-dimensional geometric model with vertex attributes is determined, residual prediction is performed on the three-dimensional geometric model with vertex attributes based on the inter-frame correlation, and a quantized residual is determined; The quantized residual is binarized to obtain a binary bit string, the binary bit string is encoded based on adaptive binary arithmetic coding, compressed data of the vertex attribute is generated, and compression of the three-dimensional geometric model with vertex attributes is realized.

[0088] In some embodiments, if the input three-dimensional geometric model sequence is in a mesh format, the three-dimensional geometric model sequence is segmented based on the three-dimensional model data type to obtain a plurality of sub-meshes; the mesh facets of the three-dimensional geometric model with facet connection attributes are symbolized based on the EdgeBreaker facet symbolization algorithm to obtain symbolized facets; performing inter-frame correlation classification on a three-dimensional geometric model with vertex attributes, determining inter-frame correlation attributes between frames in the three-dimensional geometric model with vertex attributes, performing residual prediction on the three-dimensional geometric model with vertex attributes based on the inter-frame correlation attributes, and determining a quantized residual; The quantized residual or the symbolized patch is binarized to obtain a binary bit string, and the binary bit string is bit string encoded based on adaptive binary arithmetic coding to generate compressed data of vertex attributes and compressed data of patch connection attributes, thereby realizing three-dimensional geometric model compression of vertex attributes and patch connection attributes.

[0089] The embodiment identifies a three-dimensional model data type and an attribute type of the three-dimensional geometric model sequence in response to a three-dimensional geometric model sequence input, the three-dimensional model data type includes a point cloud sequence and a mesh sequence, the attribute type of the point cloud sequence includes a vertex attribute, the attribute type of the mesh sequence includes a vertex attribute and a face connection attribute, the three-dimensional geometric model sequence is segmented based on the three-dimensional model data type, a plurality of sub-structures are obtained, the sub-structure includes a point cloud group and a sub-mesh, the mesh face of the three-dimensional geometric model with the face connection attribute is symbolized based on an EdgeBreaker face symbolization algorithm, a symbolized face is obtained, the three-dimensional geometric model with the vertex attribute is classified in terms of inter-frame correlation, an inter-frame correlation attribute between frames in the three-dimensional geometric model with the vertex attribute is determined, the three-dimensional geometric model with the vertex attribute is predicted based on the inter-frame correlation attribute, a quantized residual error is determined, the quantized residual error and / or the symbolized face is binarized, a binary bit string is obtained, the binary bit string is bit string encoded based on adaptive binary arithmetic coding, compressed data of the vertex attribute and / or compressed data of the face connection attribute are generated, and compression of the three-dimensional geometric model sequence is implemented. Since the data type and the attribute type of the three-dimensional geometric model sequence are identified, the three-dimensional geometric model sequence is adaptively segmented and processed in a complex and dynamic manner, the redundancy of various attributes in frames and between frames is deeply mined without obvious quality loss, the multi-dimensional attributes of the dynamic three-dimensional geometric model sequence are effectively processed, the compression efficiency of the three-dimensional geometric sequence is greatly improved, the calculation amount in the entropy coding process is greatly reduced by binarizing all to-be-encoded information, the real-time requirement of the three-dimensional geometric model sequence compression is met, and the compression ratio and the compression efficiency are greatly improved.

[0090] In addition, the embodiment of the present application further provides a computer readable storage medium, and the computer readable storage medium stores a three-dimensional geometric model sequence compression program based on binary arithmetic coding. When the three-dimensional geometric model sequence compression program based on binary arithmetic coding is executed by a processor, the steps of the three-dimensional geometric model sequence compression method based on binary arithmetic coding are implemented.

[0091] The computer readable storage medium provided in the application may be, for example, a U disk, but is not limited to an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, system, or device, or any combination of the above. More specific examples of the computer readable storage medium may include, but are not limited to, an electrical connection with one or more conductive wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the embodiment, the computer readable storage medium may be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, system, or device. The program code contained on the computer readable storage medium can be transmitted by any suitable medium, including but not limited to an electrical wire, an optical cable, an RF (Radio Frequency), and the like, or any suitable combination of the above.

[0092] The computer readable storage medium described above may be included in the three-dimensional geometric model sequence compression device based on binary arithmetic coding, or may exist independently without being assembled into the three-dimensional geometric model sequence compression device based on binary arithmetic coding.

[0093] In addition, the embodiment of the application also provides a computer program product, including a three-dimensional geometric model sequence compression program based on binary arithmetic coding, and the three-dimensional geometric model sequence compression program based on binary arithmetic coding realizes the steps of the three-dimensional geometric model sequence compression method based on binary arithmetic coding when executed by a processor.

[0094] The computer program product embodiment of the application is basically the same as the above-mentioned three-dimensional geometric model sequence compression method based on binary arithmetic coding, and will not be repeated here.

[0095] Reference Figure 6 , Figure 6 The structure block diagram of an embodiment of the three-dimensional geometric model sequence compression device based on binary arithmetic coding of the application is shown in the figure.

[0096] As Figure 6 shown, the three-dimensional geometric model sequence compression device based on binary arithmetic coding provided in the embodiment of the application includes: The data recognition module 10 is configured to recognize a three-dimensional model data type and an attribute type of the three-dimensional geometric model sequence in response to a three-dimensional geometric model sequence input, the three-dimensional model data type including a point cloud sequence and a mesh sequence, the attribute type of the point cloud sequence including a vertex attribute, and the attribute type of the mesh sequence including a vertex attribute and a face connection attribute. The model segmentation module 20 is configured to segment the three-dimensional geometric model sequence based on the three-dimensional model data type to obtain a plurality of substructures, the substructure including a point cloud cluster and a sub-mesh. The face symbolization module 30 is configured to symbolize a mesh face of the three-dimensional geometric model having the face connection attribute based on an EdgeBreaker face symbolization algorithm to obtain a symbolized face. The inter-frame association analysis module 40 is configured to perform inter-frame association classification on the three-dimensional geometric model having the vertex attribute to determine an inter-frame association attribute between frames in the three-dimensional geometric model having the vertex attribute. The residual prediction module 50 is configured to perform residual prediction on the three-dimensional geometric model having the vertex attribute based on the inter-frame association attribute to determine a quantized residual. The binarization module 60 is configured to binarize the quantized residual and / or the symbolized face to obtain a binary bit string. The binary compression module 70 is configured to perform bit string coding on the binary bit string based on adaptive binary arithmetic coding to generate compressed data of the vertex attribute and / or compressed data of the face connection attribute, and to realize compression of the three-dimensional geometric model sequence.

[0097] The embodiment identifies a three-dimensional model data type and an attribute type of the three-dimensional geometric model sequence in response to a three-dimensional geometric model sequence input, the three-dimensional model data type includes a point cloud sequence and a mesh sequence, the attribute type of the point cloud sequence includes a vertex attribute, the attribute type of the mesh sequence includes a vertex attribute and a face connection attribute, segments the three-dimensional geometric model sequence based on the three-dimensional model data type, obtains a plurality of substructures, the substructure includes a point cloud group and a sub-mesh, symbolizes a mesh face of the three-dimensional geometric model with the face connection attribute based on an EdgeBreaker face symbolization algorithm, obtains a symbolized face, performs inter-frame correlation classification on the three-dimensional geometric model with the vertex attribute, determines an inter-frame correlation attribute between frames in the three-dimensional geometric model with the vertex attribute, performs residual prediction on the three-dimensional geometric model with the vertex attribute based on the inter-frame correlation attribute, determines a quantized residual, binarizes the quantized residual and / or the symbolized face, obtains a binary bit string, performs bit string coding on the binary bit string based on adaptive binary arithmetic coding, generates compressed data of the vertex attribute and / or compressed data of the face connection attribute, and realizes compression of the three-dimensional geometric model sequence. Since the embodiment identifies the data type and the attribute type of the three-dimensional geometric model sequence, the complex and dynamic three-dimensional geometric model sequence is adaptively segmented and processed, the redundancy of various attributes in the frame and between frames is deeply mined under the premise of no obvious quality loss, the multi-dimensional attributes of the dynamic three-dimensional geometric model sequence are effectively processed, and therefore the compression efficiency of the three-dimensional geometric sequence is greatly improved. Binarizing all to-be-encoded information greatly reduces the amount of calculation in the entropy coding process, thereby meeting the real-time requirement of the three-dimensional geometric model sequence compression and greatly improving the compression ratio and the compression efficiency.

[0098] The three-dimensional geometric model sequence compression device based on binary arithmetic coding provided in the application adopts the three-dimensional geometric model sequence compression method based on binary arithmetic coding in the above embodiment, and can solve the technical problem of three-dimensional geometric model sequence compression based on binary arithmetic coding. Compared with the prior art, the three-dimensional geometric model sequence compression device based on binary arithmetic coding provided in the application has the same beneficial effects as the three-dimensional geometric model sequence compression method based on binary arithmetic coding provided in the above embodiment, and other technical features in the three-dimensional geometric model sequence compression device based on binary arithmetic coding are the same as the features disclosed in the above embodiment method, which will not be repeated here.

[0099] It should be understood that the above is only an example, and the technical solutions of the application are not limited in any way. In specific applications, those skilled in the art can set up as needed, and the application does not limit this.

[0100] It should be noted that the above-described workflow is merely illustrative and does not limit the scope of protection of the present application, and in actual application, a person skilled in the art can select part or all of them to achieve the purpose of the embodiment scheme according to actual needs, which is not limited here.

[0101] In addition, technical details not described in detail in the present embodiment can be found in the binary arithmetic coding-based three-dimensional geometric model sequence compression method provided by any embodiment of the present application, which will not be repeated here.

[0102] It should be noted that in this paper, the term "includes", "contains" or any other variant thereof is intended to cover non-exclusive inclusion, so that the process, method, article or system including a series of elements not only includes those elements, but also includes other elements not explicitly listed, or includes elements inherent to such process, method, article or system. Without more limitations, the element defined by the sentence "includes a" does not exclude the presence of other identical elements in the process, method, article or system including the element.

[0103] The above-mentioned embodiment numbers of the present application are only for description, not representing the advantages and disadvantages of the embodiments.

[0104] Through the above description of the embodiments, those skilled in the art can clearly understand that the above-mentioned embodiment method can be realized by means of software and necessary general hardware platform, of course, it can also be realized by hardware, but in many cases, the former is a better embodiment. Based on such understanding, the technical solutions of the present application or the part that contributes to the prior art can be embodied in the form of software product, which is stored in a storage medium (such as read-only memory / random access memory, magnetic disk, optical disk), including a plurality of instructions for making a terminal device (which can be a mobile phone, computer, server, air conditioner, or network device, etc.) execute the method described in each embodiment of the present application.

[0105] The above is only the preferred embodiment of the present application, and does not limit the patent scope of the present application, and any equivalent structure or equivalent process transformation made by using the content of the present application specification and drawings, or directly or indirectly applied to other related technical fields, are also included in the patent protection scope of the present application.

Claims

1. A three-dimensional geometric model sequence compression method based on binary arithmetic coding, characterized in that: The three-dimensional geometric model sequence compression method based on binary arithmetic coding includes: In response to a 3D geometric model sequence input, identifying a 3D model data type and an attribute type of the 3D geometric model sequence, wherein the 3D model data type includes a point cloud sequence and a mesh sequence, the attribute type of the point cloud sequence includes vertex attributes, and the attribute type of the mesh sequence includes vertex attributes and facet connectivity attributes; Segmenting the three-dimensional geometric model sequence based on the three-dimensional model data type to obtain a plurality of substructures, wherein the substructures include point cloud clusters and subgrids; Symbolize the mesh patches of the three-dimensional geometric model with patch connection attributes based on the EdgeBreaker patch symbolization algorithm to obtain symbolized patches; Performing inter-frame correlation classification on the three-dimensional geometric model with vertex attributes, and determining inter-frame correlation attributes between frames in the three-dimensional geometric model with vertex attributes; Performing residual prediction on the three-dimensional geometric model with vertex attributes based on the inter-frame correlation attributes to determine a quantized residual; Binarizing the quantized residual and / or the symbolized patch to obtain a binary bit string; The binary bit string is bit-string encoded based on adaptive binary arithmetic coding to generate compressed data of vertex attributes and / or compressed data of patch connection attributes, thereby achieving compression of the three-dimensional geometric model sequence.

2. The three-dimensional geometric model sequence compression method based on binary arithmetic coding according to claim 1, characterized in that: The segmenting of the three-dimensional geometric model sequence based on the three-dimensional model data type to obtain a plurality of substructures includes: determining a segmentation strategy based on the three-dimensional model data type; Segmenting the three-dimensional geometric model sequence according to the segmentation strategy to obtain a plurality of substructures; The segmentation strategies include spatial segmentation strategy, spreading segmentation strategy and semantic segmentation strategy; The space segmentation strategy includes: Determining a three-dimensional space boundary of a three-dimensional geometric model of a frame to be segmented in the three-dimensional geometric model sequence, and segmenting the three-dimensional space where the three-dimensional geometric model is located into a plurality of subspaces based on the three-dimensional space boundary; performing point cloud segmentation or mesh segmentation on the three-dimensional geometric model in each subspace based on point cloud attribute information or mesh attribute information of the three-dimensional geometric model to obtain a plurality of point cloud clusters or a plurality of sub-meshes, wherein the mesh attribute information includes vertex position information of mesh facets and a center of gravity position of the mesh facets; The spreading segmentation strategy includes: Determine the target number of sub-grid faces required during the spreading process; Performing a propagation traversal on each mesh face in the three-dimensional geometric model of the to-be-segmented frame based on a breadth-first principle, and performing mesh segmentation on the three-dimensional geometric model based on the propagation traversal result and the target number of faces to obtain a plurality of sub-grids; The semantic segmentation strategy includes: Rendering the three-dimensional geometric model of the frame to be segmented to generate a two-dimensional color image, and performing semantic recognition on the two-dimensional color image to generate a semantic map; The three-dimensional geometric model is subjected to three-dimensional semantic segmentation based on the semantic graph to obtain a plurality of point cloud clusters or a plurality of sub-grids.

3. The three-dimensional geometric model sequence compression method based on binary arithmetic coding according to claim 2, characterized in that: The performing residual prediction on the three-dimensional geometric model with vertex attributes based on the inter-frame correlation attributes to determine a quantized residual includes: Determining a rate of change of a vertex attribute of a three-dimensional geometric model in the three-dimensional geometric model having vertex attributes based on the inter-frame correlation attribute; Predicting the attribute value of the current vertex to be compressed according to the change rate to obtain a predicted attribute value; Determining a prediction residual based on the predicted attribute value and the current attribute value of the current vertex to be compressed; The target quantization bit width is determined according to the value range of the prediction residual and the quantization requirement, and the prediction residual is quantized based on the target quantization bit width to obtain a quantized residual. The prediction residual is quantized according to the following formula: in, represents the quantized residual, and Respectively represent the lower and upper limits of the range of prediction residuals, Indicates the floating point value selected in the value range. Indicates the target quantization bit width, represents the adjustment factor of non-uniform quantization, Represents the prediction residual to be quantized.

4. The three-dimensional geometric model sequence compression method based on binary arithmetic coding according to claim 3, characterized in that: The inter-frame correlation attribute includes a first correlation attribute, a second correlation attribute, and a third correlation attribute, wherein the first correlation attribute is an attribute of not having an inter-frame correlation, the second correlation attribute is an attribute of having a two-frame correlation, and the third correlation attribute is an attribute of having a correlation of more than two frames; The step of predicting the attribute value of the current vertex to be compressed according to the change rate to obtain the predicted attribute value includes: Obtaining buffered data of a current data frame and a historical data frame in a data buffer, wherein the data buffer is a pre-built buffer area with a buffer upper limit exceeding three frames of data; A prediction function is constructed based on the buffered data of the current data frame and the historical data frame, the prediction function comprising: in, represents the prediction function of the third associated attribute, represents the prediction function of the second associated attribute, represents the prediction function of the second associated attribute, 、 and Respectively represent the first three frames of data of the current data frame, The data that has been encoded and decoded in the current data frame; The attribute value of the current vertex to be compressed is predicted based on the prediction function to obtain a predicted attribute value.

5. The three-dimensional geometric model sequence compression method based on binary arithmetic coding according to any one of claims 1 to 4, characterized in that: The method of symbolizing mesh patches of a three-dimensional geometric model having patch connection attributes based on an EdgeBreaker patch symbolization algorithm to obtain symbolized patches includes: Performing face traversal on each frame of the three-dimensional geometric model in the three-dimensional geometric model with face connection attributes to detect mesh holes; During the face traversal process, the mesh face positions that have no adjacent faces on any side are marked as mesh hole positions; Adding virtual vertices and virtual facets at the mesh hole positions to fill the mesh hole positions; Acquire a symbol library based on an EdgeBreaker face symbolization algorithm, wherein the symbol library includes a plurality of coding symbols; generating virtual coding symbols according to the grid hole positions, and updating the symbol library based on the virtual coding symbols; The mesh facets of the three-dimensional geometric model in the three-dimensional geometric model with facet connection attributes are symbolized based on the updated symbol library to obtain symbolized facets.

6. The three-dimensional geometric model sequence compression method based on binary arithmetic coding according to claim 5, characterized in that: After performing face traversal on each frame of the three-dimensional geometric model with face connection attributes, the method further includes: Build sort buffers and sort counters for vertex attributes; During the face traversal process, in response to passing through a new mesh vertex, recording the vertex order value of the mesh vertex through the sorting counter, generating a mapping relationship between the vertex attributes of the mesh vertex and the vertex order value, and writing the mapping relationship into the sorting buffer; In response to the completion of the patch traversal, the vertex attributes of each mesh vertex are reordered based on the mapping relationship in the sorting buffer.

7. The three-dimensional geometric model sequence compression method based on binary arithmetic coding according to claim 6, characterized in that: Binarizing the quantized residual and / or the symbolized patch to obtain a binary bit string includes: Binarizing the quantized residual to obtain a binary bit string; and / or, determining the number of symbol occurrences of each coded symbol and the number of symbol co-occurrences of each coded symbol with other symbols based on the symbolized patch; Divide the number of symbol co-occurrences of each coding symbol by the number of symbol occurrences to obtain the prior probability of each coding symbol; The coding symbols are sorted based on the prior probability, and the symbolized surface patches are binary mapped based on the sorting result to obtain a binary bit string.

8. A three-dimensional geometric model sequence compression device based on binary arithmetic coding, characterized in that: The three-dimensional geometric model sequence compression device based on binary arithmetic coding includes: a data identification module, configured to identify, in response to a 3D geometric model sequence input, a 3D model data type and an attribute type of the 3D geometric model sequence, wherein the 3D model data type includes a point cloud sequence and a mesh sequence, the attribute type of the point cloud sequence includes vertex attributes, and the attribute type of the mesh sequence includes vertex attributes and facet connectivity attributes; A model segmentation module, configured to segment the three-dimensional geometric model sequence based on the three-dimensional model data type to obtain a plurality of substructures, wherein the substructures include point cloud clusters and subgrids; A patch symbolization module is used to symbolize mesh patches of a three-dimensional geometric model with patch connection attributes based on an EdgeBreaker patch symbolization algorithm to obtain symbolized patches; An inter-frame correlation analysis module, configured to classify inter-frame correlations of a three-dimensional geometric model with vertex attributes, and determine inter-frame correlation attributes between frames in the three-dimensional geometric model with vertex attributes; A residual prediction module, configured to perform residual prediction on the three-dimensional geometric model with vertex attributes based on the inter-frame correlation attribute, and determine a quantized residual; A binarization module, configured to binarize the quantized residual and / or the symbolized patch to obtain a binary bit string; The binary compression module is used to perform bit string encoding on the binary bit string based on adaptive binary arithmetic coding, generate compressed data of vertex attributes and / or compressed data of facet connection attributes, and realize compression of three-dimensional geometric model sequence.

9. A three-dimensional geometric model sequence compression device based on binary arithmetic coding, characterized in that: The three-dimensional geometric model sequence compression device based on binary arithmetic coding includes: a memory, a processor, and a three-dimensional geometric model sequence compression program based on binary arithmetic coding stored on the memory and runnable on the processor. The three-dimensional geometric model sequence compression program based on binary arithmetic coding is configured to implement the three-dimensional geometric model sequence compression method based on binary arithmetic coding as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a three-dimensional geometric model sequence compression program based on binary arithmetic coding. When the three-dimensional geometric model sequence compression program based on binary arithmetic coding is executed by a processor, the three-dimensional geometric model sequence compression method based on binary arithmetic coding as described in any one of claims 1 to 7 is implemented.

Images