Gaussian point cloud lossless coding and decoding method for three-dimensional reconstruction

Abstract

The invention relates to a three-dimensional reconstruction-oriented Gaussian point cloud lossless coding and decoding method, a product and a storage medium. The method comprises the following steps: extracting a first Gaussian point cloud corresponding to anhor data in a numpy array format at a coding side; converting the first Gaussian point cloud in the numpy array format into a first Gaussian point cloud in a ply format; meanwhile, through a Morton code-based spatial sorting method, performing Morton code sorting on geometric data and attribute data of the first Gaussian point cloud in the numpy array format, and generating a Morton sequence index table; and carrying out AVS coding on the first Gaussian point cloud in the ply format to obtain compressed data in a binary code stream form. On the decoding side, AVS decoding is carried out when the compressed data in the binary code stream form and the Morton sequence index table are received, and a second Gaussian point cloud in the ply format is obtained through reduction; performing format conversion on the second Gaussian point cloud in the ply format to obtain a second Gaussian point cloud in a numpy array format; and rearranging the attribute data and the geometric data based on the Morton sequence index table to realize one-to-one correspondence of the geometric data and the attribute data between the first Gaussian point cloud and the second Gaussian point cloud before and after coding and decoding. Compared with an existing method, the data consistency before and after coding is improved.

Description

TECHNICAL FIELD

[0001] The application relates to the technical field of Gaussian point cloud coding, in particular to a Gaussian point cloud lossless coding method, product and storage medium for three-dimensional reconstruction. BACKGROUND

[0002] Traditional three-dimensional scene representation methods such as point cloud and mesh in the field of computer vision and graphics have problems such as insufficient flexibility or high computational complexity in dynamic scene modeling, and methods based on neural radiation field (NeRF) also face the bottleneck of slow rendering speed.

[0003] Three-dimensional Gaussian splatting (3D Gaussian Splatting) technology realizes real-time rendering while maintaining high-quality rendering effects by explicitly representing Gaussian distribution points in the scene, and has become one of the most promising dynamic scene representation schemes.

[0004] However, this technology faces the core problem of excessive data volume in practical application. Each 3D Gaussian point is represented by a 59-dimensional attribute vector. A medium-complexity static scene usually contains 50-100 million Gaussian points, and the single-frame data volume can reach 300-500 MB. For a dynamic scene sequence, if the frame rate is calculated at 25-30 fps, the original data rate will be as high as 7.5-15 Gbps, far exceeding the carrying capacity of the current network bandwidth. Especially in real-time interactive applications such as cloud VR and AR, this data volume makes it almost impossible to transmit high-quality three-dimensional video.

[0005] Existing solutions such as the i3DV platform introduce a binary hash table to cache the transformation relationship of Gaussian points between frames, which can compress the transmission code rate to the level of 40 Mbps, but when processing long-time sequences or high-dynamic scenes, a large amount of redundant spatio-temporal association information still needs to be stored. At the same time, traditional video coding standards (such as H.264 / AVC, H.265 / HEVC) are designed for regular pixel arrays and cannot be directly applied to the compression of unstructured Gaussian point cloud data. This technical bottleneck seriously restricts the practical deployment of three-dimensional Gaussian technology in resource-constrained environments such as mobile terminals and edge computing, and hinders the large-scale development of immersive interactive applications. SUMMARY

[0006] Therefore, it is necessary to provide a Gaussian point cloud lossless coding method, product and storage medium for three-dimensional reconstruction, which can reduce the storage and transmission burden of Gaussian point cloud data, is suitable for the coding needs of complex dynamic scenes, and provides reliable geometric-attribute consistency guarantee for the coding of dynamic three-dimensional scenes.

[0007] To solve the above technical problems, the technical solutions of the present application are as follows: In a first aspect, a Gaussian point cloud lossless coding method for three-dimensional reconstruction includes: On the encoding side: The first Gaussian point cloud corresponding to the anhor data in numpy array format generated based on the original video frame is extracted; wherein the first Gaussian point cloud corresponding to the anhor data in numpy array format is configured with geometry data and attribute data; The first Gaussian point cloud corresponding to the anhor data in numpy array format is converted into a first Gaussian point cloud in ply format; at the same time, the geometry data and attribute data of the first Gaussian point cloud corresponding to the anhor data in numpy array format are sorted by Morton code based on a spatial sorting method to generate a Morton sequence index table; The first Gaussian point cloud in ply format is subjected to AVS encoding to obtain binary code stream form compression data; On the decoding side: When the binary code stream form compression data and the Morton sequence index table are received, the binary code stream form compression data is subjected to AVS decoding to restore the second Gaussian point cloud in ply format; The second Gaussian point cloud in ply format is subjected to format conversion to obtain a second Gaussian point cloud in numpy array format; Based on the Morton sequence index table, the attribute data and geometry data of the second Gaussian point cloud in numpy array format are rearranged according to the index values, realizing one-to-one correspondence between the geometry data and attribute data of the first Gaussian point cloud corresponding to the anhor data in numpy array format before and after coding and decoding.

[0008] In a second aspect, a computer program product includes a computer program or computer executable instructions, which are executed by a processor to implement a Gaussian point cloud lossless coding method for three-dimensional reconstruction as described above.

[0009] In a third aspect, a computer readable storage medium stores at least one instruction, at least one program, a code set or an instruction set, which are loaded and executed by a processor to implement a Gaussian point cloud lossless coding method for three-dimensional reconstruction as described above.

[0010] Compared with the prior art, the beneficial effects of the technical solutions of the present application are: The application provides a Gauss point cloud lossless coding method for three-dimensional reconstruction. The coding order and decoding process of the point cloud are systematically optimized, and the compression performance is significantly improved under the premise of ensuring reconstruction accuracy. Data format conversion and coding processing are performed in the AVS coding process to ensure data consistency during compression and restoration. The point cloud data is rearranged and organized based on the Morton space ordering method to make the coding order more consistent with spatial proximity. The application can significantly reduce the coding rate while maintaining the rendering quality by considering the high-precision compression of geometric data and the inter-frame redundancy modeling of attribute data. BRIEF DESCRIPTION OF DRAWINGS

[0011] Figure 1 The application provides a Gauss point cloud lossless coding method for three-dimensional reconstruction.

[0012] Figure 2 The application provides a point cloud data coding flowchart.

[0013] Figure 3 The application provides a general framework based on the i3DV platform.

[0014] Figure 4 The application provides a Morton code space ordering optimization scheme flowchart.

[0015] Figure 5 The application provides a performance evaluation experiment result comparison chart based on the Dance_Dunhuag_Pairs scene in the PKU-MVHumans data set.

[0016] Figure 6 The application provides a performance evaluation experiment result comparison chart based on the Show_Groups_4K scene in the PKU-MVHumans data set.

[0017] Figure 7 The application provides a performance evaluation experiment result comparison chart based on the VRU_dg4 scene in the VRU-Basketball data set.

[0018] Figure 8 The application provides a performance evaluation experiment result comparison chart based on the VRU_gz scene in the VRU-Basketball data set. DETAILED DESCRIPTION

[0019] The terms "first", "second", and the like in the description and in the claims of the present application and the above drawings merely mean differentiating one feature from another, and do not necessarily imply a specific order or sequence. It should be understood that the terms so used are interchangeable under appropriate circumstances and that the embodiments of the present application are capable of functioning in other sequences than the one described. Also, the terms "comprise", "have" and any variations thereof are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements is not necessarily limited to those elements, but can include other elements not expressly listed or inherent to such process, method, article, or apparatus. The term "determine" broadly encompasses a wide variety of actions and is used interchangeably with "decide", "calculate", "compute", "process", "ascertain", "identify", "select", "choose", "establish", and the like. Other terms will be defined as described below.

[0020] It should be noted that when an element is referred to as being "connected" to another element, it can be directly connected to the other element, or connected to the other element through an intervening element. In addition, "connected" in the following embodiments, if there is a transmission of electrical signals or data between the connected objects, should be understood as "electrically connected", "communicatively connected" and the like.

[0021] It should be emphasized that the acquisition, transmission, storage, use, processing, etc. of data in the technical solutions of the embodiments of the present application all comply with the relevant provisions of national laws and regulations.

[0022] In the embodiments of the present application, some existing industry solutions of software, components, models, etc. may be mentioned, which should be considered as exemplary, and the purpose is only to illustrate the feasibility of the implementation of the technical solutions of the present application, but does not mean that the applicant has or will necessarily use the solutions.

[0023] The accompanying drawings are only used for illustrative purposes and should not be understood as limiting the patent; In order to better illustrate the embodiments, some components in the drawings may be omitted, enlarged or reduced, and do not represent the actual size of the product; For those skilled in the art, it is understandable that some well-known structures in the drawings and their descriptions may be omitted.

[0024] The technical solutions of the present application will be further described below in conjunction with the drawings and embodiments.

[0025] Embodiment 1 The embodiment provides a Gauss point cloud lossless coding method for three-dimensional reconstruction, which is applied to an i3DV platform (Interactive 3D Video, an interactive 3D video coding platform), and refers to FIG. 1 and FIG. 2, and comprises the following steps. At the encoding side: A first Gauss point cloud corresponding to anhor data in a numpy array format generated based on an original video frame is extracted; wherein the first Gauss point cloud corresponding to the anhor data in the numpy array format is configured with geometric data and attribute data; The first Gauss point cloud corresponding to the anhor data in the numpy array format is converted into a first Gauss point cloud in a ply format, and the conversion process ensures that the data structure is complete; meanwhile, the geometric data and the attribute data of the first Gauss point cloud corresponding to the anhor data in the numpy array format are subjected to Morton code sorting through a spatial sorting method based on a Morton code, to generate a Morton sequence index table; The first Gauss point cloud in the ply format is subjected to AVS coding, to obtain compressed data in a binary code stream form; The first Gauss point cloud in the ply format is subjected to AVS coding, to retain the original precision of all geometric and attribute information, avoid any data loss, and thus realize a high-fidelity compression effect, to obtain compressed data in a binary code stream form; At the decoding side: When the compressed data in the binary code stream form and the Morton sequence index table are received, the compressed data in the binary code stream form is subjected to AVS decoding, to restore a second Gauss point cloud in a ply format; The second Gauss point cloud in the ply format is subjected to format conversion, to obtain a second Gauss point cloud in a numpy array format; Based on the Morton sequence index table, the attribute data and the geometric data of the second Gauss point cloud in the numpy array format are rearranged according to index values, to realize one-to-one correspondence between the geometric data and the attribute data of the first Gauss point cloud corresponding to the anhor data in the numpy array format and the second Gauss point cloud in the numpy array format before and after coding and decoding, and thus the reliability of data processing and the decoding precision are significantly improved.

[0026] The above method systematically optimizes the organization structure, encoding order and decoding process of the point cloud, significantly improves the compression performance under the premise of ensuring the reconstruction accuracy. Data format conversion and encoding processing are performed in the AVS coding process to ensure data consistency during compression and restoration. The point cloud data is reorganized and optimized based on the Morton space ordering method to make the encoding order more consistent with spatial proximity. The application can significantly reduce the coding rate while maintaining the rendering quality by considering the high-precision compression of geometric data and the inter-frame redundancy modeling of attribute data.

[0027] In some preferred embodiments, the above Gauss point cloud lossless coding method for three-dimensional reconstruction is based on AVSPCRM (Audio Video Coding Standard Point cloud Reference Software) for AVS coding.

[0028] The first Gauss point cloud corresponding to the numpy array format anhor data generated based on the original video frame is obtained through the following two stages, as shown in FIG. 3: S1, construction and compression stage, including: extracting a plurality of frames of sparse point sets with consistent structures from the original video frame by the SFM algorithm, and generating a first anchor point set based on the sparse point set by clustering; extracting anchor points of the initial frame from the first anchor point set, and completing compression of the geometric data and attribute data of the anchor points of the initial frame by context modeling, quantization, entropy coding and dequantization to obtain the anchor points of the initial frame after reconstruction, and taking the compressed initial frame as a global reference frame; For any frame other than the initial frame: taking the previous frame as a reference, through a binary transformation caching mechanism based on three planes and significance control, predicting the offset of the geometric data and the change of the attribute data of the anchor points of the subsequent frame compared with the anchor points of the previous frame by a third multi-layer perception network to obtain the anchor point offset and the first attribute compression residual of the anchor points of the subsequent frame compared with the anchor points of the previous frame, and storing the anchor point offset and the first attribute compression residual; quickly reconstructing the anchor points of the subsequent frame by loading the anchor point state of the previous frame and combining the anchor point offset and the first attribute compression residual to obtain the anchor points of the subsequent frame after reconstruction; combining all the anchor points after reconstruction to obtain a second anchor point set; S2, rendering stage, including: taking the second anchor point set as input, predicting a group of Gauss points corresponding to each anchor point by a first multi-layer perception network to obtain the first Gauss point cloud corresponding to the numpy array format anhor data.

[0029] The process of obtaining the anchor points after reconstruction of the initial frame includes: A multi-resolution hash grid expression is constructed for the anchor points of the initial frame extracted from the first anchor point set, and local spatial context features are extracted; The local spatial context features are input into a second multi-layer perception network, the context modeling of the anchor points of the initial frame is performed through the second multi-layer perception network, and a context model is established; The context model is iteratively trained until a preset condition is met, including: The attribute data and the geometric data of the anchor points of the initial frame are predicted through the context model to obtain probability distribution information and corresponding quantization steps, the attribute data and the geometric data are quantized using the predicted quantization steps; the attribute data and the geometric data after quantization are entropy encoded based on the probability distribution information; and the context model is optimized through loss function back propagation; The optimized quantization steps and the optimized probability distribution information are obtained through the iteratively trained context model; The geometric data and the attribute data of the anchor points of the initial frame are quantized from continuous values to discrete values using the optimized quantization steps; The discrete values are encoded into shorter bit streams using the optimized probability distribution information and an entropy encoding method; The neighborhood information provided by the hash grid expression is used as context information, combined with the optimized probability distribution information and the optimized quantization steps, the bit stream is dequantized through linear interpolation, mapped back to the original range, and the result is optimized and adjusted according to the neighborhood information to improve the recovery accuracy, to obtain the anchor points after reconstruction of the initial frame.

[0030] The encoding and storage process of the anchor point offset and the first attribute compression residual in the method of the embodiment includes: The saliency grid is used as an auxiliary module to divide the three-dimensional space into voxel units, mark the dynamic change area of the anchor points of the subsequent frame compared with the anchor points of the previous frame through saliency scoring, and generate a binary mask, the binary mask is used to mark the dynamic change area and assist the three-plane structure and the third multi-layer perception network prediction; The binary mask is input into the third multi-layer perception network as an input to guide the third multi-layer perception network to focus on the dynamic area to predict the anchor point offset and the first attribute compression residual; After quantization processing of the inter-anchor point offset and the first attribute compression residual, the quantized representation of the inter-anchor point offset and the first attribute compression residual is stored by using the three-plane structure composed of three orthogonal two-dimensional planes.

[0031] The above-mentioned storage of the quantized representation of the inter-anchor point offset and the first attribute compression residual by using the three-plane structure composed of three orthogonal two-dimensional planes includes: The inter-anchor point offset and the first attribute compression residual are written into a binary hash table construction space index by using a hash function; The inter-anchor point offset and the first attribute compression residual are accessed through the space index, and the dynamic region weight extracted by the saliency grid is used as auxiliary information. The inter-anchor point offset and the first attribute compression residual are quantized by a rate-distortion optimization mechanism to obtain a reconstruction error and a coding bit rate. The overall compression process adopts a rate-distortion joint optimization strategy to minimize the reconstruction loss and the bit cost simultaneously, thereby achieving an effective balance between high compression efficiency and high quality reconstruction. The reconstruction error and the coding bit rate are balanced and optimized to obtain a compressed binary code stream and write the binary code stream into the binary hash table, thereby ensuring that the model maintains rendering quality while achieving high compression rate.

[0032] The method of the embodiment further includes: Based on a redundancy elimination method, the geometry data and the attribute data of the first Gaussian point cloud in ply format are converted into a geometry compression residual and a second attribute compression residual in the AVS encoding process, and the geometry data and the attribute data of the second Gaussian point cloud in ply format are reconstructed according to the geometry compression residual and the second attribute compression residual in the AVS decoding process. A bit allocation method is used to compress and store the geometry compression residual and the second attribute compression residual.

[0033] The above-mentioned conversion of the geometry data and the attribute data of the first Gaussian point cloud in ply format into a geometry compression residual and a second attribute compression residual in the AVS encoding process, and the reconstruction of the geometry data and the attribute data of the second Gaussian point cloud in ply format according to the geometry compression residual and the second attribute compression residual in the AVS decoding process include: In the AVS encoding process: The octree is divided by spatial recursion, and a three-dimensional space is recursively divided into eight nodes; if the number of Gaussian point clouds in any node exceeds a preset threshold, the node is further subdivided; otherwise, the subdivision is stopped and the node is marked as a leaf node; the geometry data and the attribute data of the first Gaussian point cloud in the ply format are stored in the octree structure, only the non-empty nodes are encoded, and spatial redundancy is eliminated; The octree nodes are traversed, the geometry data and the attribute data of the first Gaussian point cloud in the ply format are sorted, the nearest neighbor in the encoded node is found as a parent node for each node, and the prediction tree in a single chain structure is constructed; the anchor point coordinates of the current point are predicted by using the anchor point coordinates of the parent node, the difference between the predicted value and the actual value is calculated, and the geometry compression residual error is obtained; for the attribute data, the attribute difference between the anchor point of the current node and the anchor point of the previous node is calculated, and the second attribute compression residual error is obtained; In the AVS decoding process: The geometry data of the second Gaussian point cloud in the ply format is reconstructed by predicting the current coordinate value based on the differential prediction of the parent node and calculating the residual error; and the attribute data of the second Gaussian point cloud in the ply format is reconstructed by predicting the difference based on the sorted reference point set. The embodiment method sorts the geometry data and the attribute data of the first Gaussian point cloud corresponding to the anhor data in the numpy array format by using the Morton code, as shown in FIG. 4, which includes: Before AVS encoding, the overall data of the first Gaussian point cloud corresponding to the anhor data in the numpy array format is sorted by using the Morton code, and a unified Morton sequence index table is generated based on 3D coordinates; The overall data is divided into the geometry data and the attribute data, and is compressed respectively; After AVS decoding, the decoded geometry data and attribute data are reordered by using the saved Morton sequence index table to restore the order before encoding. The Morton sorting, data conversion and AVS PCRM lossless encoding and decoding modules work together to optimize the overall performance of the i3DV platform, and are suitable for the wide deployment requirements of real-time immersive applications.

[0034] Embodiment 2 To verify the effectiveness of the present application, a Gauss point cloud lossless coding method based on AVS PCRM and Morton code optimization for three-dimensional reconstruction is evaluated on the test sequence proposed in Embodiment 1. The test sequence includes four scenes, each containing 250 frames of data at a frame rate of 25 fps. Referring to Figure 5 and Figure 6 , the Dance_Dunhuag_Pairs scene and the Show_Groups_4K scene come from the PKU-MVHumans dataset (56-60 camera perspectives, 1920x1080 to 3840x2160 resolution), referring to Figure 7 and Figure 8 , the VRU_dg4 scene and the VRU_gz scene come from the VRU-Basketball dataset (34 camera perspectives, 1920x1080 resolution), covering the complexity and diversity of dynamic multi-perspective three-dimensional scenes. To ensure experimental consistency, all evaluations use the same test set division and are conducted under the experimental conditions defined in the proposal. Performance evaluation is based on objective indicators (PSNR, SSIM, LPIPS, file size, compression ratio, encoding / decoding time) and subjective quality comparison, jointly verifying the improvement of the method in storage, transmission and rendering efficiency.

[0035] The present application replaces the original scheme with the Gauss point cloud lossless coding method based on AVS PCRM and Morton code optimization for three-dimensional reconstruction on the i3DV platform, and conducts subjective and objective performance evaluation experiments on four standard datasets. The following are the results of objective indicators and subjective quality comparison.

[0036] Objective indicators: referring to Table 1, the comprehensive experimental evaluation results show that i3DV 2.0 and AVS PCRM perform consistently in terms of encoding quality, with minimal differences in PSNR, SSIM and LPIPS indicators, effectively maintaining the integrity of point cloud geometric structure and perceptual attributes. In contrast, AVS PCRM has a slight advantage in spatial compression ratio and decoding efficiency, especially in terms of point cloud position (XYZ) dimension code rate saving effect. Both have strong real-time processing capability in terms of encoding time, meeting the application needs of high-resolution, dynamic three-dimensional scenes. The present application, while ensuring lossless restoration, further improves compression efficiency and decoding performance through structural optimization and spatial redundancy compression mechanism, providing a stable and reliable solution for efficient encoding of dynamic Gauss point clouds.

[0037] Table 1 Comparison of objective indicators

[0038] Subjective quality: referring to Figures 5-8The subjective comparison experiment of the i3DV codec platform and the alternative point cloud compression scheme for the three-dimensional reconstruction-oriented Gauss point cloud lossless coding method based on the AVS PCRM and the Morton code optimization shows that the compressed model has no obvious difference in rendering quality from the original model.

[0039] It can be understood that the test of the embodiment corresponds to the method of embodiment 1 described above, and thus is not repeated here.

[0040] Embodiment 3 The embodiment provides a computer readable storage medium, and at least one instruction, at least one program, a code set or an instruction set are stored on the storage medium. The at least one instruction, at least one program, code set or instruction set is loaded and executed by a processor, so that the processor executes part or all steps of the method provided in embodiment 1 of the application.

[0041] It can be understood that the storage medium can be transient or non-transient. Exemplarily, the storage medium includes but is not limited to a U disk, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, and various media that can store program codes.

[0042] Exemplarily, the processor can be a central processing unit (CPU), a microprocessor unit (MPU), a digital signal processor (DSP) or a field programmable gate array (FPGA).

[0043] Exemplarily, the read-only memory includes but is not limited to a mask ROM, a PROM, an EPROM, an EEPROM, a Flash and the like.

[0044] Exemplarily, the random access memory includes but is not limited to a DRAM, an SRAM, an SDRAM, a DDR SDRAM and the like.

[0045] In some examples, a computer program product is provided, which can be implemented in a hardware, software or combination thereof. As a non-limiting example, the computer program product can be embodied in the storage medium, and can also be embodied in a software product, such as an SDK (Software Development Kit) and the like.

[0046] As a non-limiting example, a computer program product is provided, which includes a computer program or computer executable instructions stored in a computer readable storage medium. A processor of an electronic device reads the computer program or computer executable instructions from the computer readable storage medium, and the processor executes the computer executable instructions, so that the electronic device performs part or all of the steps of the method described in the embodiments of the present application.

[0047] In some examples, a computer program is provided, which includes computer readable code, which, when run in a computer device, causes a processor in the computer device to perform part or all of the steps of the method.

[0048] The embodiments also propose an electronic device, which includes a memory and a processor, the memory stores at least one instruction, at least one program, a code set or an instruction set, and the processor implements part or all of the steps of the method described in Embodiment 1 when executing the at least one instruction, at least one program, code set or instruction set.

[0049] In some examples, a hardware entity of the electronic device is provided, which includes a processor, a memory and a communication interface; wherein the processor generally controls the overall operation of the electronic device; the communication interface is used for the electronic device to communicate with other terminals or servers through a network; the memory is configured to store instructions and applications executable by the processor, and can also cache data to be processed by the processor and data to be processed or having been processed by each module in the electronic device (including but not limited to image data, audio data, voice communication data and video communication data), which can be realized by FLASH, EPROM (Erasable Programmable Read-Only Memory), EEPROM (Electrically Erasable Programmable Read-Only Memory) or RAM (Random Access Memory).

[0050] The processor can include one or more processing elements. Thus, the processor can include one or more integrated circuits (ICs) that are configured to perform the functions of the processor. In addition, each integrated circuit can include circuitry (e.g., first circuitry, second circuitry, and other circuitry, etc.) that is configured to perform the functions of the processor.

[0051] Further, the processor, the communication interface and the memory can transmit data through a bus, and the bus can include any number of interconnected buses and bridges, which connect various circuits of the one or more processors and the memory together.

[0052] The same or similar reference numerals can represent the same or similar components; The terms describing the positional relationship in the drawings are only used for example illustration, and cannot be understood as a limitation to the present application; It should be noted that the embodiments and the features in the embodiments in the present application can be combined with each other without conflict, if possible.

[0053] In different specific implementations, the method or system described in the present application can be implemented in software, hardware or their combination. In addition, the order of the steps of the method can be changed, and various elements can be added, reordered, combined, omitted, modified, etc.

[0054] Obviously, the above embodiments of the present application are only examples for clearly illustrating the present application, and are not intended to limit the implementation of the present application, and are not used to limit the present application. Based on the above description, other different forms of changes or variations can be made by those skilled in the art, and each separate structure / function module or unit can be integrated together to form an independent part, or each module can exist independently, or two or more modules can be integrated to form an independent part, and the structure and function of the separate components can be realized as a combined structure or component. Here, it is not necessary and impossible to enumerate all the implementation manners. Any modification, equivalent replacement and improvement made within the spirit and principle of the present application shall be included in the protection scope of the claims of the present application.

Claims

1. A Gauss point cloud lossless coding method for three-dimensional reconstruction, applied to an i3DV platform, characterized in that, Comprise: At the encoding side: extract the first Gaussian point cloud corresponding to the anhor data in numpy array format generated based on the original video frame; wherein the first Gaussian point cloud corresponding to the anhor data in numpy array format is configured with geometric data and attribute data; convert the first Gaussian point cloud corresponding to the anhor data in numpy array format into a first Gaussian point cloud in ply format; at the same time, through the spatial ordering method based on Morton code, the geometric data and attribute data of the first Gaussian point cloud corresponding to the anhor data in numpy array format are sorted by Morton code to generate a Morton sequence index table; AVS encoding is performed on the first Gaussian point cloud in ply format to obtain binary stream form compression data; At the decoding side: When receiving the binary stream form compression data and the Morton sequence index table, AVS decoding is performed on the binary stream form compression data to restore the second Gaussian point cloud in ply format; Format conversion is performed on the second Gaussian point cloud in ply format to obtain a second Gaussian point cloud in numpy array format; Based on the Morton sequence index table, the attribute data and geometric data of the second Gaussian point cloud in numpy array format are rearranged according to the index value, realizing one-to-one correspondence between the geometric data and attribute data of the first Gaussian point cloud corresponding to the anhor data in numpy array format before encoding and decoding and the second Gaussian point cloud in numpy array format.

2. The method of claim 1, wherein, The first Gaussian point cloud corresponding to the anhor data in numpy array format is obtained through the following two stages, including: S1, construction and compression stage, including: extracting a plurality of frames of sparse point sets with consistent structure from the original video frame through SFM algorithm, and generating a first anchor point set based on the sparse point set through clustering method; extracting anchor points of the initial frame from the first anchor point set, and completing compression of the geometric data and attribute data of the anchor points of the initial frame through context modeling, quantization, entropy encoding and dequantization to obtain reconstructed anchor points of the initial frame, and taking the compressed initial frame as a global reference frame; For any frame other than the initial frame: taking the previous frame as a reference, through a binary transformation buffer mechanism based on three planes and significance regulation, the anchor point offset of the subsequent frame compared with the anchor point of the previous frame and the attribute compression residual are obtained through a third multi-layer perception network to predict the offset of the geometric data and the change of the attribute data of the anchor point of the subsequent frame compared with the anchor point of the previous frame, and the anchor point offset and the first attribute compression residual are stored; the anchor point of the subsequent frame is quickly reconstructed by loading the anchor point state of the previous frame and combining the anchor point offset and the first attribute compression residual to obtain the reconstructed anchor point of the subsequent frame; combining all the reconstructed anchor points of the frames to obtain a second anchor point set; S2, rendering stage, including: The first Gaussian point cloud corresponding to the anhor data in the numpy array format is obtained by predicting a group of Gaussian points corresponding to each anchor point through a first multi-layer perception network with the second anchor point set as input.

3. The method of claim 2, wherein, The process of obtaining the anchor points after reconstruction of the initial frame includes: A multi-resolution hash grid expression is constructed for the anchor points of the initial frame extracted from the first anchor point set, and local spatial context features are extracted; The local spatial context features are input into a second multi-layer perception network, and the context model is established by modeling the context of the anchor points of the initial frame through the second multi-layer perception network; The context model is iteratively trained until a preset condition is met, including: The attribute data and the geometry data of the anchor points of the initial frame are predicted through the context model to obtain probability distribution information and corresponding quantization steps, and the attribute data and the geometry data are quantized using the predicted quantization steps; the attribute data and the geometry data after quantization are entropy encoded based on the probability distribution information; and the context model is optimized through loss function back propagation; The optimized quantization steps and the optimized probability distribution information are obtained through the iteratively trained context model; The geometry data and the attribute data of the anchor points of the initial frame are quantized from continuous values to discrete values using the optimized quantization steps; The discrete values are encoded into shorter bit streams using the optimized probability distribution information and an entropy encoding method; The neighbor information provided by the hash grid expression is used as context information, and the bit stream is dequantized through linear interpolation, mapped back to the original range, and optimized and adjusted according to the neighbor information, to obtain the anchor points after reconstruction of the initial frame, combined with the optimized probability distribution information and the optimized quantization steps.

4. The method of claim 2, wherein, The encoding and storage process of the anchor point offset and the first attribute compression residual includes: A saliency grid is used as an auxiliary module to divide the three-dimensional space into voxel units, mark the dynamic change area of the anchor points of the subsequent frame compared with the anchor points of the previous frame through saliency scoring, and generate a binary mask, which is used to mark the dynamic change area and assist the three-plane structure and the third multi-layer perception network in prediction. The binary mask is used as the input of the third multi-layer perception network to guide the third multi-layer perception network to focus on the dynamic area and predict the anchor point offset and the first attribute compression residual. After quantization processing of the anchor point offset and the first attribute compression residual, the three-plane structure composed of three orthogonal two-dimensional planes is used to store the quantized representation of the anchor point offset and the first attribute compression residual.

5. The method of claim 4, wherein, The three-plane structure composed of three orthogonal two-dimensional planes is used to store the quantized representation of the anchor point offset and the first attribute compression residual, including: The anchor point offset and the first attribute compression residual are written into a binary hash table to construct a spatial index using a hash function. Access the anchor point offset and the first attribute compression residual through the space index, take the dynamic region weight extracted by the saliency grid as auxiliary information, and quantize the anchor point offset and the first attribute compression residual through a rate-distortion optimization mechanism to obtain a reconstruction error and an encoding bit rate; Balance optimize the reconstruction error and the encoding bit rate to obtain a compressed binary code stream and write the binary code stream into the binary hash table.

6. The method of claim 1, wherein, The method further comprises: In an AVS encoding process, convert the geometry data and the attribute data of the first Gaussian point cloud in the ply format into a geometry compression residual and a second attribute compression residual, and in an AVS decoding process, reconstruct the geometry data and the attribute data of the second Gaussian point cloud in the ply format according to the geometry compression residual and the second attribute compression residual; wherein the geometry compression residual and the second attribute compression residual are compressed and stored by using a bit allocation method.

7. The method of claim 6, wherein, The method of converting the geometry data and the attribute data of the first Gaussian point cloud in the ply format into a geometry compression residual and a second attribute compression residual in an AVS encoding process, and reconstructing the geometry data and the attribute data of the second Gaussian point cloud in the ply format according to the geometry compression residual and the second attribute compression residual in an AVS decoding process, comprises: In an AVS encoding process: recursively divide an octree by using spatial recursion to divide a three-dimensional space into 8 nodes; if the number of Gaussian point cloud points in any node exceeds a preset threshold, continue to subdivide the node; otherwise, stop the subdivision and mark the node as a leaf node; store the geometry data and the attribute data of the first Gaussian point cloud in the ply format by layer through the octree structure, and only encode the non-empty nodes to eliminate spatial redundancy; traverse the octree nodes, find the nearest neighbor in the encoded nodes as the parent node for each node by traversing the sorted geometry data and attribute data of the first Gaussian point cloud in the ply format, and construct a single-chain prediction tree; predict the anchor point coordinates of the current node by using the anchor point coordinates of the parent node, calculate the difference between the predicted value and the actual value to obtain the geometry compression residual; for the attribute data, calculate the attribute difference between the anchor point of the current node and the anchor point of the previous node to obtain the second attribute compression residual; In an AVS decoding process: predict the current coordinate value by using the differential prediction based on the parent node for the geometry compression residual and calculate the residual to reconstruct the geometry data of the second Gaussian point cloud in the ply format; predict the attribute compression residual by using the sorted reference point set to perform differential prediction to reconstruct the attribute data of the second Gaussian point cloud in the ply format.

8. The method of claim 1-7, wherein, The method of sorting the geometry data and the attribute data of the first Gaussian point cloud corresponding to the numpy array format anhor data by using a Morton code comprises: Before AVS encoding, the overall data of the first Gaussian point cloud corresponding to the anhor data in numpy array format is sorted by Morton code, and a unified Morton sequence index table is generated based on 3D coordinates; The overall data is divided into the geometry data and the attribute data, and compression is performed respectively; After AVS decoding, the decoded geometry data and attribute data are reordered using the saved Morton sequence index table to restore the order before encoding.

9. A computer program product comprising computer programs or computer executable instructions, characterized in that, The computer program or computer executable instructions are executed by the processor to implement the method of any one of claims 1-8.

10. A computer-readable storage medium, characterized in that, The storage medium stores at least one instruction, at least one program, a code set or an instruction set, which is loaded and executed by the processor to implement the method of any one of claims 1-8.

Images