Decoding method of voxel grid, encoding method of voxel grid, apparatus

Abstract

The present disclosure provides a method and device for decoding and encoding a voxel grid. The method for decoding a voxel grid comprises: obtaining decoding metadata, a pruning mask, a coarse residual and a reference voxel index from a bitstream; constructing an empty voxel grid based on the decoding metadata; for any one sub-voxel in the empty voxel grid, filtering out unpruned voxels from a plurality of sub-voxels through the pruning mask of the sub-voxel; determining a corresponding reference voxel for each unpruned voxel according to the reference voxel index; for any one unpruned voxel in the plurality of unpruned voxels, determining a coarse reconstruction value of the unpruned voxel as a sum of the corresponding reference voxel and the coarse residual of the unpruned voxel; and using a coarse quantization step to dequantize the coarse reconstruction value of each unpruned voxel to obtain a first voxel value of the unpruned voxel. The present disclosure can fully utilize spatial correlation to achieve a higher compression ratio when compressing a voxel grid.

Description

Technical Field

[0001] This disclosure relates to the field of image processing technology, specifically to a voxel grid decoding method, a voxel grid encoding method, and an apparatus. Background Technology

[0002] Neural Radiance Field (NeRF) technology can reconstruct 3D scenes based on sparse views and then render images from any viewpoint, but NeRF technology suffers from high computational complexity.

[0003] Currently, while voxel-based radiation fields can effectively alleviate the computational complexity problem, this technique generates a massive number of additional parameters, resulting in a heavy computational burden.

[0004] Therefore, there is an urgent need for a data processing method for compressed voxel radiation fields to solve the above-mentioned technical problems. Summary of the Invention

[0005] In view of this, this disclosure proposes a voxel grid decoding method, a voxel grid encoding method, and an apparatus to solve the problem in related technologies that voxel radiation fields generate a large number of additional parameters, resulting in a large computational burden.

[0006] The first aspect of this disclosure proposes a method for decoding voxel grids, including:

[0007] Obtain decoding metadata, pruning mask, coarse residual, and reference voxel index from the bitstream;

[0008] An empty voxel grid is constructed based on the decoded metadata; the decoded metadata includes the size, resolution, and coarse quantization step size of the voxel grid; the voxel grid includes multiple sub-voxels;

[0009] For any child voxel in the empty voxel grid, the unpruned voxel is selected from the plurality of child voxels by using the pruning mask of the child voxel;

[0010] Based on the reference voxel index, determine the corresponding reference voxel for each unpruned voxel;

[0011] For any one of the multiple unpruned voxels, the sum of the reference voxel corresponding to the unpruned voxel and the coarse residual is determined as the coarse reconstruction value of the unpruned voxel.

[0012] The coarse reconstruction value of each unpruned voxel is dequantized using the coarse quantization step size to obtain the first voxel value of the unpruned voxel.

[0013] The embodiments of this disclosure construct an empty voxel grid by decoding metadata, and process each sub-voxel in the empty voxel grid using pruning masks, coarse residuals, and reference voxel indexes, which can make full use of spatial correlation to achieve a high compression ratio.

[0014] In this embodiment of the disclosure, filtering out unpruned voxels from the plurality of daughter voxels using a pruning mask includes:

[0015] If the pruning mask of the daughter voxel is the first value, then the daughter voxel is determined to be a pruned voxel.

[0016] If the pruning mask of the daughter voxel is the second value, then the daughter voxel is determined to be an unpruned voxel.

[0017] In this embodiment of the disclosure, the reference voxel index includes index values ​​of multiple unpruned voxels and a reference voxel corresponding to each index value; determining the corresponding reference voxel for each unpruned voxel based on the reference voxel index includes:

[0018] For any one of the multiple unpruned voxels, a reference voxel corresponding to the index value of the unpruned voxel is determined based on the reference voxel index.

[0019] In this embodiment of the disclosure, after obtaining the first voxel value of the unpruned voxel, the method further includes:

[0020] Key voxels were selected from multiple unpruned voxels;

[0021] For any one of the multiple key voxels, the fine residual of the key voxel is dequantized using a fine quantization step size to obtain the reconstructed residual.

[0022] The sum of the first voxel value and the reconstruction residual is used as the fine reconstruction value of the key voxel.

[0023] In this embodiment of the disclosure, key voxels are selected from a plurality of unpruned voxels, including:

[0024] Obtain the key voxel mask from the bitstream;

[0025] For any unpruned voxel, if the key voxel mask of the unpruned voxel is a preset mask value, then the unpruned voxel is determined to be a key voxel.

[0026] A second aspect of this disclosure provides a method for encoding voxel grids, comprising:

[0027] A first voxel mesh is generated based on images from multiple perspectives; the first voxel mesh is used to characterize the three-dimensional structure of the current scene; each first voxel in the first voxel mesh is used to characterize the radiation information at different locations;

[0028] The pruned voxels in the first voxel grid are pruned to obtain the second voxel grid; the pruned voxels refer to the first voxels whose importance scores are less than the first importance score threshold, and any importance score is used to characterize the contribution value of the first voxel to the image rendering result.

[0029] For any second voxel in the second voxel grid, a reference voxel corresponding to the second voxel is selected. The reference voxel is the second voxel in the second voxel grid that is adjacent to the second voxel and has the smallest sum of absolute errors.

[0030] The second voxel grid is encoded based on multiple second voxels and multiple reference voxels that correspond one-to-one with the multiple second voxels.

[0031] In this embodiment of the disclosure, the second voxel mesh is encoded based on a plurality of second voxels and a plurality of reference voxels corresponding one-to-one with the plurality of second voxels, including:

[0032] Calculate the second absolute error sum between each second voxel and its corresponding reference voxel to obtain multiple second absolute error sums that correspond one-to-one with the plurality of second voxels;

[0033] The average of the sum of the plurality of second absolute errors is used as the first weighting coefficient. The first loss function of the first voxel grid is weighted using the first weighting coefficient to obtain the second loss function.

[0034] The second voxel grid is adjusted using the second loss function to obtain a third voxel grid; the third voxel grid includes multiple third voxels;

[0035] The coarse quantization step size is used to quantize the multiple third voxels respectively to obtain multiple quantized third voxels;

[0036] Subtract each quantized third voxel from its corresponding reference voxel to obtain the corresponding coarse residual;

[0037] Based on the coarse residual corresponding to each quantized third voxel, each quantized third voxel is dequantized to obtain a coarse reconstructed voxel grid corresponding to the third voxel grid; the coarse reconstructed voxel grid includes multiple fourth voxels.

[0038] Multiple key voxels are selected from the plurality of fourth voxels; the key voxels refer to the fourth voxels whose importance score is greater than the second importance score threshold among the plurality of fourth voxels;

[0039] For any one of the multiple key voxels, a target second voxel with the same position as the key voxel is determined from the second voxel grid;

[0040] Calculate the third absolute error sum between the key voxel and the target second voxel to obtain multiple third absolute error sums that correspond one-to-one with the multiple key voxels;

[0041] The average of the sum of the plurality of third absolute errors is used as the second weighting coefficient, and the first loss function is weighted using the second weighting coefficient to obtain the third loss function;

[0042] The third loss function is used to adjust multiple key voxels in the coarsely reconstructed voxel grid to obtain a fourth voxel grid; the fourth voxel grid includes multiple adjusted key voxels.

[0043] For any one of the multiple adjusted key voxels, subtract the adjusted key voxel from the second voxel at the same position in the second voxel grid.

[0044] The subtraction result is quantified to obtain a fine residual.

[0045] The coarse residual, the fine residual, the reference voxel index, the pruned voxel mask, and the key voxel mask are written into the bitstream; the pruned voxel mask is used to determine the position of the pruned voxel in the empty voxel grid, and the key voxel mask is used to determine the position of the key voxel in the empty voxel grid.

[0046] A third aspect of this disclosure provides a voxel grid decoding apparatus, comprising:

[0047] The data acquisition module is used to obtain decoding metadata, pruning mask, coarse residual, and reference voxel index from the bitstream;

[0048] An empty voxel grid construction module is used to construct an empty voxel grid based on the decoded metadata; the decoded metadata includes the size, resolution, and coarse quantization step size of the voxel grid; the voxel grid includes multiple sub-voxels;

[0049] The unpruned voxel screening module is used to select unpruned voxels from the plurality of child voxels by using the pruning mask of the child voxel for any child voxel in the empty voxel grid.

[0050] The reference voxel determination module is used to determine the corresponding reference voxel for each unpruned voxel based on the reference voxel index.

[0051] The coarse reconstruction value determination module is used to determine the coarse reconstruction value of any one of the multiple unpruned voxels by summing the reference voxel corresponding to the unpruned voxel and the coarse residual.

[0052] The dequantization module is used to dequantize the coarse reconstruction value of each unpruned voxel using the coarse quantization step size to obtain the first voxel value of the unpruned voxel.

[0053] A fourth aspect of this disclosure provides a voxel grid encoding device, comprising:

[0054] The first voxel mesh generation module is used to generate a first voxel mesh based on images from multiple perspectives; the first voxel mesh is used to characterize the three-dimensional structure of the current scene; each first voxel in the first voxel mesh is used to characterize the radiation information at different locations;

[0055] The pruning module is used to prune the pruned voxels in the first voxel grid to obtain the second voxel grid; the pruned voxel refers to the first voxel whose importance score is less than the first importance score threshold, and any importance score is used to characterize the contribution value of the first voxel to the image rendering result.

[0056] The reference voxel determination module is used to select a reference voxel corresponding to any second voxel in the second voxel grid. The reference voxel is the second voxel in the second voxel grid that is adjacent to the second voxel and has the smallest sum of absolute errors.

[0057] The encoding module is used to encode the second voxel grid based on a plurality of second voxels and a plurality of reference voxels that correspond one-to-one with the plurality of second voxels.

[0058] An embodiment of the fifth aspect of this disclosure provides an electronic device including a memory and a processor, the memory and the processor being communicatively connected to each other, the memory storing computer instructions, and the processor executing the computer instructions to perform the voxel grid decoding method of the first aspect or the voxel grid encoding method of the second aspect.

[0059] An embodiment of the sixth aspect of this disclosure provides a computer-readable storage medium storing computer instructions for causing a computer to execute the voxel grid decoding method described in the first aspect or the voxel grid encoding method described in the second aspect.

[0060] Additional aspects and advantages of this disclosure will be set forth in part in the description which follows, and in part will be obvious from the description or may be learned by practice of this disclosure. Attached Figure Description

[0061] Various other advantages and benefits will become apparent to those skilled in the art upon reading the following detailed description of preferred embodiments. The accompanying drawings are for illustrative purposes only and are not intended to limit the scope of this disclosure. Furthermore, the same reference numerals denote the same parts throughout the drawings.

[0062] In the attached diagram:

[0063] Figure 1 A flowchart illustrating a voxel grid decoding method provided in an embodiment of this disclosure is shown.

[0064] Figure 2 A schematic diagram illustrating the decoding and encoding process of a voxel grid provided in an embodiment of this disclosure is shown;

[0065] Figure 3 A flowchart illustrating a voxel grid encoding method provided in an embodiment of this disclosure is shown.

[0066] Figure 4 A schematic diagram of the structure of a voxel grid decoding device provided in an embodiment of the present disclosure is shown;

[0067] Figure 5 A schematic diagram of the structure of a voxel grid encoding device provided in an embodiment of the present disclosure is shown;

[0068] Figure 6 A schematic diagram of the structure of an electronic device provided in an embodiment of the present disclosure is shown;

[0069] Figure 7 A schematic diagram of a storage medium provided according to an embodiment of the present disclosure is shown. Detailed Implementation

[0070] Exemplary embodiments of the present disclosure will now be described in more detail with reference to the accompanying drawings. While exemplary embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure may be implemented in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the disclosure to those skilled in the art.

[0071] It should be noted that, unless otherwise stated, the technical or scientific terms used in this disclosure shall have the ordinary meaning as understood by one of ordinary skill in the art to which this disclosure pertains.

[0072] The following describes the technical scenarios involved in the embodiments of this disclosure.

[0073] Generating models representing three-dimensional (3D) scenes and publishing them as photographs on the internet will profoundly transform entertainment and the future of media. This issue is receiving increasing attention as virtual reality technology develops.

[0074] Neural Radiance Field (NeRF) technology can reconstruct 3D scenes from sparse views and then render realistic images from arbitrary viewpoints. Early NeRF methods employed neural meshes for implicit 3D scene modeling. However, these methods suffer from high computational complexity, resulting in heavy training and rendering burdens. Recent research has established a promising direction: accelerating training and rendering by learning voxel-based radiance field representations. Voxel representations hold great potential in alleviating the challenges of computational complexity. However, explicit voxel mesh (EVG) representations generate a significantly larger number of additional parameters than neural mesh representations. The large number of parameters in voxel meshes imposes a heavy storage and computational burden, hindering further development and practical applications.

[0075] To address these challenges, recent research has applied data compression methods to voxel-based neural radiation fields, including importance pruning, frequency domain transformation, and vector quantization. While effective, these methods rarely consider the spatial correlation of voxel data. Furthermore, due to the complexity of modeling the encoding bit rate within these coding frameworks, these methods rarely consider compression issues when training voxel grids.

[0076] To address the aforementioned problems, this invention proposes a decoding method and an encoding method for voxel grids used to compress voxel radiation fields. This method can fully utilize spatial correlation to achieve a higher compression ratio. Furthermore, this invention proposes a rate-distortion loss function to jointly optimize rendering distortion and encoding bit rate.

[0077] According to an embodiment of this disclosure, a method for decoding voxel grids is provided. It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions. Furthermore, although a logical order is shown in the flowchart, in some cases, the steps shown or described may be executed in a different order than that shown here.

[0078] This embodiment provides a voxel grid decoding method, applied to the decoding side. Figure 1 This is a flowchart of a voxel mesh decoding method according to an embodiment of the present disclosure, such as... Figure 1 As shown, the process includes the following steps:

[0079] Step S101: Obtain decoding metadata, pruning mask, coarse residual, and reference voxel index from the bitstream.

[0080] The decoded metadata includes, but is not limited to, the size (i.e., length, width, and height), resolution, coarse quantization step size, and fine quantization step size of the empty voxel grid. The empty voxel grid can be constructed using the decoded metadata. The pruning mask can also be understood as a pruning placeholder code. The pruning mask can be used to filter out unpruned voxels from multiple sub-voxels of the empty voxel grid. The reference voxel index includes the index values ​​of multiple unpruned voxels and the reference voxel corresponding to each index value. The reference voxel index can be used to determine the corresponding reference voxel for each unpruned voxel.

[0081] Step S102: Construct an empty voxel grid based on the decoded metadata.

[0082] Specifically, an empty voxel grid containing multiple sub-voxels can be constructed by adjusting the size, resolution, and coarse quantization step size of the empty voxel grid.

[0083] More specifically, after constructing the empty voxel mesh, each sub-voxel within the empty voxel mesh can be processed using pruning masks, coarse residuals, and reference voxel indices to obtain the first voxel mesh; this first voxel mesh is used to characterize the three-dimensional structure of the current scene; each first voxel in the first voxel mesh is used to characterize the radiation information at different locations.

[0084] Step S103: For any child voxel in the empty voxel grid, filter out the unpruned voxels from the plurality of child voxels using the pruning mask of the child voxel.

[0085] In some specific embodiments, step S103 above includes steps S1031-S1032:

[0086] Step S1031: If the pruning mask of the daughter voxel is a first value, then the daughter voxel is determined to be a pruned voxel.

[0087] Step S1032: If the pruning mask of the daughter voxel is the second value, then the daughter voxel is determined to be an unpruned voxel.

[0088] In steps S1031-S1032 above, the first and second values ​​can be set by themselves and are not specifically limited here. For example, when the pruning mask of the child voxel is 0, the child voxel is a pruned voxel; conversely, when the pruning mask of the child voxel is 1, the child voxel is an unpruned voxel.

[0089] Step S104: Determine the corresponding reference voxel for each unpruned voxel based on the reference voxel index.

[0090] In some specific embodiments, step S104 above includes step S1041:

[0091] Step S1041: For any one of the multiple unpruned voxels, a reference voxel corresponding to the index value of the unpruned voxel is determined based on the reference voxel index.

[0092] An example will be used to illustrate step S1041 above:

[0093] The reference voxel index includes A1--B1, A2--B2, ..., An--Bn; where A1--B1 represents the index value A1 of the unpruned voxel 1 and the corresponding reference voxel B1. When the index value of the unpruned voxel is A2, the corresponding reference voxel can be determined as B2 using the above reference voxel index.

[0094] Step S105: For any one of the multiple unpruned voxels, the sum of the reference voxel corresponding to the unpruned voxel and the coarse residual is determined as the coarse reconstruction value of the unpruned voxel.

[0095] Step S106: Use the coarse quantization step size to dequantize the coarse reconstruction value of each unpruned voxel to obtain the first voxel value of the unpruned voxel.

[0096] The size of the coarse quantization step can be set according to the actual situation, and no specific limit is made here.

[0097] In steps S105-S106 above, the present embodiment determines the coarse reconstruction value of the unpruned voxel by the sum of the reference voxel and the coarse residual, and then performs inverse quantization on it, which helps to restore the first voxel mesh generated by the image from multiple perspectives well, and can ensure the accuracy and integrity of the voxel mesh restoration.

[0098] In some specific embodiments, after step S106 above, the method further includes:

[0099] Step S107: Select key voxels from multiple unpruned voxels.

[0100] In some specific embodiments, step S107 above includes steps S1071-S1072:

[0101] Step S1071: Obtain the key voxel mask from the bitstream.

[0102] Step S1072: For any unpruned voxel, if the key voxel mask of the unpruned voxel is a preset mask value, then the unpruned voxel is determined to be a key voxel.

[0103] In steps S1071-S1072, whether an unpruned voxel is a key voxel can be determined by whether the key voxel mask of the unpruned voxel is a preset mask value. For example, when the key voxel mask of the unpruned voxel is 1, the unpruned voxel is a key voxel; conversely, when the key voxel mask of the unpruned voxel is 0, the unpruned voxel is a non-key voxel.

[0104] Step S108: For any one of the multiple key voxels, use a fine quantization step size to inverse quantize the fine residual of the key voxel to obtain the reconstructed residual.

[0105] The size of the fine quantization step can be set according to the actual situation, and no specific limit is made here.

[0106] Step S109: The sum of the first voxel value and the reconstruction residual is used as the fine reconstruction value of the key voxel.

[0107] Specifically, in this embodiment, the fine residual of the key voxel is dequantized by using a fine quantization step size to obtain the reconstruction residual; then the sum of the first voxel value and the reconstruction residual is used as the fine reconstruction value of the key voxel, which helps to better restore the first voxel mesh generated by the image from multiple perspectives, and greatly ensures the accuracy and integrity of the voxel mesh restoration.

[0108] In some specific embodiments, the voxel mesh decoding method provided in this disclosure has the following process: Figure 2 The spatial predictive coding framework employs a process of "blank mesh → pruning → selecting reference index → ​​coarse reconstruction → fine reconstruction" to obtain the first voxel mesh representing the 3D structure of the current scene. This decoding method effectively alleviates the problem of high computational complexity.

[0109] According to an embodiment of this disclosure, an embodiment of a voxel mesh encoding method is provided. It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions. Furthermore, although a logical order is shown in the flowchart, in some cases, the steps shown or described may be executed in a different order than that shown here.

[0110] This embodiment provides a voxel grid encoding method, applied to the encoding side. Figure 3 This is a flowchart of a voxel grid encoding method according to an embodiment of the present disclosure, such as... Figure 3 As shown, the process includes the following steps:

[0111] Step S301: Generate a first voxel mesh based on images from multiple perspectives.

[0112] Specifically, the first voxel mesh can be understood as a voxel radiation field, used to characterize the three-dimensional structure of the current scene. Each first voxel in the first voxel mesh is used to characterize radiation information at different locations, including but not limited to color information, density information, illumination information, transparency information, reflectivity information, and other physical property information, thereby enabling the reconstruction of the scene's geometry.

[0113] In some specific embodiments, the first voxel mesh can be generated through steps S3011-S3013:

[0114] Step S3011: Randomly initialize the voxel grid and neural grid.

[0115] Step S3012: Based on the input camera pose, emit several rays through each image pixel position, perform uniform sampling on the rays, and for each sampling point, perform linear interpolation on the voxel grid to obtain color feature values ​​and volume density values. Input the color feature values ​​into the neural grid to calculate the color values, and render the corresponding image according to the volume rendering formula.

[0116]

[0117] 0=alpha(0i,0i)=1-exp(-0i0i)

[0118]

[0119] Where, α i T represents the probability that the light ray stops at point i. i σ represents the loss of light rays traveling from point i to the camera. i δ is the volume density value at point i. i c is the distance from point i to the adjacent sampling point. bg Indicates background color, c i This represents the color at point i.

[0120] Step S3013: Calculate the loss function based on the distortion between the rendered image and the real image (such as peak signal-to-noise ratio PSNR and / or structural similarity SSIM), then calculate the gradient of the loss function with respect to color feature values, volume density values, and neural grid parameters, and use stochastic gradient descent to iterate multiple times to obtain the final voxel model.

[0121] Step S302: Prune the pruned voxels in the first voxel grid to obtain the second voxel grid.

[0122] Specifically, a pruned voxel refers to the first voxel whose importance score is less than the first importance score threshold. Any importance score is used to characterize the contribution of the first voxel to the image rendering result. Since volume density determines the visibility of a specific voxel, the volume density value of the voxel can be used as the importance score.

[0123] In some specific embodiments, before step S302 above, the method further includes step S3021:

[0124] Step S3021: Select pruned voxels and key voxels from the plurality of first voxels.

[0125] Specifically, pruned voxels are used to characterize first voxels with importance scores less than a first importance score threshold, and key voxels are used to characterize first voxels with importance scores greater than a second importance score threshold, where the second importance score threshold is greater than the first importance score threshold. The first and second importance score thresholds can be set by the user according to actual circumstances, and are not specifically limited here.

[0126] More specifically, in step S3021, a corresponding pruning mask can be set for the pruned voxels so that, during the decoding stage, it can be determined whether a voxel is a pruned voxel based on the pruning mask. For example, if the pruning mask of a voxel is a first value, then the voxel is determined to be a pruned voxel. A corresponding key voxel mask can also be set for key voxels so that, during the decoding stage, it can be determined whether a voxel is a key voxel based on the key voxel mask. For example, if the key voxel mask of a voxel is a preset value, then the voxel is determined to be a key voxel.

[0127] Step S303: For any second voxel in the second voxel grid, select the reference voxel corresponding to the second voxel.

[0128] Specifically, the reference voxel refers to the second voxel in the second voxel grid that is adjacent to the second voxel and has the smallest sum of absolute errors.

[0129] In some specific embodiments, step S303 above includes steps S3031-S3033:

[0130] Step S3031: For any second voxel in the second voxel grid, calculate the first absolute error between the second voxel and each encoded adjacent second voxel.

[0131] Specifically, each second voxel can be treated as a high-dimensional vector, and the sum of absolute differences (SAD) between the second voxel and each encoded neighboring second voxel can be understood as the SAD of two high-dimensional vectors. The specific calculation method for SAD can be determined using existing techniques and is not specifically limited here.

[0132] Step S3032: The encoded adjacent second voxel with the smallest sum of the first absolute error is determined as the reference voxel corresponding to the second voxel.

[0133] Step S3033: Construct a reference voxel index based on multiple second voxels and the reference voxel corresponding to each second voxel.

[0134] Specifically, the reference voxel index includes the index values ​​of multiple second voxels and the adjacent positions corresponding to each index value; for example, when the index value of second voxel 1 is the first value, the voxel at the first adjacent position is selected as the reference voxel of second voxel 1; when the index value of second voxel 1 is the second value, the voxel at the second adjacent position is selected as the reference voxel of second voxel 1.

[0135] Step S304: Encode the second voxel grid based on multiple second voxels and multiple reference voxels that correspond one-to-one with the multiple second voxels.

[0136] Specifically, in the process of encoding the second voxel mesh, progressive encoding is mainly adopted, that is, the second voxel mesh is encoded in multiple levels, for example... Figure 2 As shown, the process of rate distortion optimization is: fine-tuning -- coarse quantization -- fine-tuning -- fine quantization.

[0137] This embodiment of the present disclosure can remove voxels with little impact on image rendering by pruning the pruned voxels in the first voxel grid, thereby improving the compression ratio without significantly affecting visual quality. Preferably, this embodiment of the present disclosure can fully utilize spatial correlation and achieve a high compression ratio by selecting a reference voxel corresponding to the second voxel and encoding the second voxel grid based on multiple second voxels and multiple reference voxels that correspond one-to-one with the multiple second voxels.

[0138] In some specific embodiments, step S304 above further includes steps a1-a3:

[0139] Step a1: Calculate the second absolute error sum between each second voxel and its corresponding reference voxel to obtain multiple second absolute error sums that correspond one-to-one with the multiple second voxels.

[0140] Step a2: The average value of the sum of the plurality of second absolute errors is used as the first weighting coefficient. The first loss function of the first voxel grid is weighted using the first weighting coefficient to obtain the second loss function.

[0141] Step a3: Adjust the second voxel grid using the second loss function to obtain a third voxel grid; the third voxel grid includes multiple third voxels.

[0142] Specifically, the second loss function can be used as a truth loss function for fine-tuning the second voxel grid.

[0143] This embodiment of the disclosure uses the average of multiple second absolute errors as a first weighting coefficient, and performs weighting processing using a first loss function to obtain a second loss function; then, it adjusts the second voxel grid using the second loss function to obtain a third voxel grid. This achieves the following technical effects: the finely adjusted grid can more effectively represent key information during compression, thereby improving the compression ratio while maintaining visual quality; and the weighting processing ensures that visually important features are preserved more during compression, reducing the impact on visual quality.

[0144] In some specific embodiments, after step a3 above, the method further includes:

[0145] Step b1: Use a coarse quantization step size to quantize the multiple third voxels respectively to obtain multiple quantized third voxels.

[0146] Specifically, uniform quantization is generally used, and coarse quantization step size is a larger quantization step size, which can be set according to the actual situation, and no specific limitation is made here.

[0147] Step b2: Subtract each quantized third voxel from its corresponding reference voxel to obtain the corresponding coarse residual.

[0148] Step b3: Based on the coarse residual corresponding to each quantized third voxel, perform dequantization on each quantized third voxel to obtain a coarse reconstructed voxel grid corresponding to the third voxel grid; the coarse reconstructed voxel grid includes multiple fourth voxels.

[0149] This embodiment of the present disclosure obtains the corresponding coarse residual by subtracting each quantized third voxel from its corresponding reference voxel; and performs dequantization processing on each quantized third voxel based on the coarse residual corresponding to each quantized third voxel to obtain a coarse reconstructed voxel grid corresponding to the third voxel grid. This allows for fine processing only on voxels with large variations, and can improve the grid compression ratio without sacrificing too much visual quality.

[0150] In some specific embodiments, after step b3 above, the method further includes steps c1-c5:

[0151] Step c1: Select multiple key voxels from the plurality of fourth voxels.

[0152] Specifically, the key voxel refers to the fourth voxel among a plurality of fourth voxels whose importance score is greater than the second importance score threshold.

[0153] Step c2: For any one of the multiple key voxels, determine the target second voxel that is at the same position as the key voxel from the second voxel grid.

[0154] Step c3: Calculate the third absolute error sum between the key voxel and the target second voxel to obtain multiple third absolute error sums corresponding one-to-one with the multiple key voxels.

[0155] Step c4: The average of the sum of the plurality of third absolute errors is used as the second weighting coefficient. The first loss function is weighted using the second weighting coefficient to obtain the third loss function.

[0156] Step c5: Adjust multiple key voxels in the coarse reconstructed voxel grid using the third loss function to obtain a fourth voxel grid; the fourth voxel grid includes multiple adjusted key voxels.

[0157] Specifically, the third loss function can be used as a rate-distortion loss function to fine-tune all key voxels.

[0158] The embodiments disclosed herein can improve mesh compression efficiency by screening key voxels and focusing on the compression of these voxels; and since only the screened key voxels need to be processed in detail, the overall computational load is reduced and computational efficiency is improved.

[0159] In some specific embodiments, after step c5, the method further includes:

[0160] Step d1: For any one of the multiple adjusted key voxels, subtract the adjusted key voxel from the second voxel at the same position in the second voxel grid.

[0161] Step d2 involves quantizing the subtraction result to obtain a fine residual.

[0162] Specifically, in the quantization process, a fine quantization step size can be used for uniform quantization. This fine quantization step size can be a relatively small quantization step size, which can be set according to the actual situation and is not specifically limited here.

[0163] This disclosure embodiment can retain the detailed changes of key voxels during the adjustment process by calculating fine residuals, thereby avoiding the loss of important data during mesh compression; in the decompression and reconstruction process, the use of fine residuals can reduce reconstruction errors and improve the quality of the final reconstructed mesh.

[0164] In some specific embodiments, after step d2, the method further includes:

[0165] The coarse residual, the fine residual, the reference voxel index, the pruned voxel mask, and the key voxel mask are written into the bitstream.

[0166] The pruned voxel mask is used to determine the position of the pruned voxel in the empty voxel grid, and the key voxel mask is used to determine the position of the key voxel in the empty voxel grid.

[0167] Corresponding to the above implementation of the voxel grid decoding method, this disclosure also provides a voxel grid decoding device for executing the voxel grid encoding method described in the above embodiments. Figure 4 As shown, the decoding device for the voxel grid includes:

[0168] The data acquisition module is used to obtain decoding metadata, pruning mask, coarse residual, and reference voxel index from the bitstream;

[0169] An empty voxel grid construction module is used to construct an empty voxel grid based on the decoded metadata; the decoded metadata includes the size, resolution, and coarse quantization step size of the voxel grid; the voxel grid includes multiple sub-voxels;

[0170] The unpruned voxel screening module is used to select unpruned voxels from the plurality of child voxels by using the pruning mask of the child voxel for any child voxel in the empty voxel grid.

[0171] The reference voxel determination module is used to determine the corresponding reference voxel for each unpruned voxel based on the reference voxel index.

[0172] The coarse reconstruction value determination module is used to determine the coarse reconstruction value of any one of the multiple unpruned voxels by summing the reference voxel corresponding to the unpruned voxel and the coarse residual.

[0173] The dequantization module is used to dequantize the coarse reconstruction value of each unpruned voxel using the coarse quantization step size to obtain the first voxel value of the unpruned voxel.

[0174] Optionally, the unpruned voxel screening module is further configured to: determine that the daughter voxel is a pruned voxel if the pruning mask of the daughter voxel is a first value; and determine that the daughter voxel is an unpruned voxel if the pruning mask of the daughter voxel is a second value.

[0175] Optionally, the reference voxel index includes index values ​​of multiple unpruned voxels and a reference voxel corresponding to each index value; the reference voxel determination module is further configured to: for any one of the multiple unpruned voxels, determine the reference voxel corresponding to the index value of the unpruned voxel based on the reference voxel index.

[0176] Optionally, the apparatus further includes: a fine reconstruction module, configured to, after obtaining the first voxel value of the unpruned voxel, select key voxels from a plurality of unpruned voxels; for any one of the key voxels, use a fine quantization step size to inverse quantize the fine residual of the key voxel to obtain a reconstruction residual; and use the sum of the first voxel value and the reconstruction residual as the fine reconstruction value of the key voxel.

[0177] Optionally, the fine reconstruction module is further configured to: obtain key voxel masks from the bitstream; and for any unpruned voxel, if the key voxel mask of the unpruned voxel is a preset mask value, determine that the unpruned voxel is a key voxel.

[0178] The voxel grid decoding apparatus and the voxel grid decoding method provided in the above embodiments of this disclosure are based on the same inventive concept and have the same beneficial effects as the methods adopted, run or implemented by the applications stored therein.

[0179] Corresponding to the above implementation of the voxel grid encoding method, this disclosure also provides a voxel grid encoding device for executing the voxel grid encoding method described in the above embodiments. Figure 5 As shown, the encoding device for the voxel grid includes:

[0180] The first voxel mesh generation module is used to generate a first voxel mesh based on images from multiple perspectives; the first voxel mesh is used to characterize the three-dimensional structure of the current scene; each first voxel in the first voxel mesh is used to characterize the radiation information at different locations;

[0181] The pruning module is used to prune the pruned voxels in the first voxel grid to obtain the second voxel grid; the pruned voxel refers to the first voxel whose importance score is less than the first importance score threshold, and any importance score is used to characterize the contribution value of the first voxel to the image rendering result.

[0182] The reference voxel determination module is used to select a reference voxel corresponding to any second voxel in the second voxel grid. The reference voxel is the second voxel in the second voxel grid that is adjacent to the second voxel and has the smallest sum of absolute errors.

[0183] The encoding module is used to encode the second voxel grid based on a plurality of second voxels and a plurality of reference voxels that correspond one-to-one with the plurality of second voxels.

[0184] Optionally, the apparatus further includes: a voxel screening module, used to screen pruned voxels and key voxels from the plurality of first voxels before pruning the pruned voxels in the first voxel grid; the pruned voxels are used to characterize first voxels with an importance score less than a first importance score threshold, and the key voxels are used to characterize first voxels with an importance score greater than a second importance score threshold, the second importance score threshold being greater than the first importance score threshold.

[0185] Optionally, the reference voxel determination module is further configured to: for any second voxel in the second voxel grid, calculate a first absolute error sum between the second voxel and each encoded adjacent second voxel; determine the encoded adjacent second voxel with the smallest first absolute error sum as the reference voxel corresponding to the second voxel; construct a reference voxel index based on multiple second voxels and the reference voxel corresponding to each second voxel; the reference voxel index includes multiple reference voxels that correspond one-to-one with the multiple second voxels.

[0186] Optionally, the encoding module is further configured to: calculate the second absolute error sum between each second voxel and its corresponding reference voxel, obtaining multiple second absolute error sums corresponding one-to-one with the multiple second voxels; use the average of the multiple second absolute error sums as a first weighting coefficient, and use the first weighting coefficient to weight the first loss function of the first voxel grid to obtain a second loss function; adjust the second voxel grid through the second loss function to obtain a third voxel grid; the third voxel grid includes multiple third voxels; quantize the multiple third voxels using a coarse quantization step size to obtain multiple quantized third voxels; subtract each quantized third voxel from its corresponding reference voxel to obtain the corresponding coarse residual; perform dequantization processing on each quantized third voxel according to the coarse residual corresponding to each quantized third voxel to obtain a coarse reconstructed voxel grid corresponding to the third voxel grid; the coarse reconstructed voxel grid includes multiple fourth voxels; and select multiple key voxels from the multiple fourth voxels. Voxels; the key voxels refer to the fourth voxels among a plurality of fourth voxels whose importance score is greater than the second importance score threshold; for any one of the multiple key voxels, a target second voxel with the same position as the key voxel is determined from the second voxel grid; the third absolute error sum between the key voxel and the target second voxel is calculated to obtain a plurality of third absolute error sums corresponding one-to-one with the multiple key voxels; the average of the plurality of third absolute error sums is used as a second weighting coefficient, and the first loss function is weighted using the second weighting coefficient to obtain a third loss function; the multiple key voxels in the coarse reconstruction voxel grid are adjusted using the third loss function to obtain a fourth voxel grid; the fourth voxel grid includes a plurality of adjusted key voxels; for any one of the multiple adjusted key voxels, the adjusted key voxel is subtracted from the second voxel with the same position in the second voxel grid; the subtraction result is quantized to obtain a fine residual.

[0187] Optionally, the device further includes a data writing module for writing the coarse residual, the fine residual, the reference voxel index, the pruned voxel mask, and the key voxel mask into the bitstream; the pruned voxel mask is used to determine the position of the pruned voxel in the empty voxel grid, and the key voxel mask is used to determine the position of the key voxel in the empty voxel grid.

[0188] The voxel grid encoding apparatus and the voxel grid encoding method provided in the above embodiments of this disclosure are based on the same inventive concept and have the same beneficial effects as the methods adopted, run or implemented by the applications stored therein.

[0189] This disclosure also provides an electronic device for performing the above-described voxel grid encoding method or voxel grid decoding method. Please refer to... Figure 6 This illustrates a schematic diagram of an electronic device provided by some embodiments of the present disclosure. For example... Figure 6 As shown, the electronic device 6 includes: a processor 600, a memory 601, a bus 602, and a communication interface 603. The processor 600, the communication interface 603, and the memory 601 are connected via the bus 602. The memory 601 stores a computer program that can run on the processor 600. When the processor 600 runs the computer program, it executes the voxel grid encoding method or the voxel grid decoding method provided in the foregoing embodiments of this disclosure.

[0190] The memory 601 may include high-speed random access memory (RAM) or non-volatile memory, such as at least one disk storage device. Communication between this system network element and at least one other network element is achieved through at least one communication interface 603 (which can be wired or wireless), such as the Internet, wide area network, local area network, or metropolitan area network.

[0191] Bus 602 can be an ISA bus, PCI bus, or EISA bus, etc. The bus can be divided into an address bus, a data bus, a control bus, etc. The memory 601 is used to store programs. After receiving an execution instruction, the processor 600 executes the program. The voxel grid encoding method or voxel grid decoding method disclosed in the foregoing embodiments can be applied to the processor 600, or implemented by the processor 600.

[0192] The processor 600 may be an integrated circuit chip with signal processing capabilities. In implementation, each step of the above method can be completed by the integrated logic circuitry in the hardware of the processor 600 or by instructions in software form. The processor 600 may be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it may also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), an off-the-shelf programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. It can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of this disclosure. The general-purpose processor may be a microprocessor or any conventional processor. The steps of the methods disclosed in the embodiments of this disclosure can be directly embodied in the execution of a hardware decoding processor, or executed by a combination of hardware and software modules in the decoding processor. The software modules may reside in random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, or other mature storage media in the art. The storage medium is located in memory 601. Processor 600 reads the information in memory 601 and, in conjunction with its hardware, completes the steps of the above method.

[0193] The electronic device provided in this disclosure and the voxel grid encoding method or voxel grid decoding method provided in this disclosure are based on the same inventive concept and have the same beneficial effects as the methods they employ, operate, or implement.

[0194] This disclosure also provides a computer-readable storage medium corresponding to the voxel grid encoding method or voxel grid decoding method provided in the foregoing embodiments. Please refer to... Figure 7 The computer-readable storage medium shown is an optical disc 30, on which a computer program (i.e., a program product) is stored. When the computer program is run by a processor, it executes the voxel grid encoding method or the voxel grid decoding method provided in any of the foregoing embodiments.

[0195] It should be noted that examples of the computer-readable storage medium may also include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other optical and magnetic storage media, which will not be elaborated here.

[0196] The computer-readable storage medium provided in the above embodiments of this disclosure and the voxel grid encoding method or voxel grid decoding method provided in the embodiments of this disclosure are based on the same inventive concept and have the same beneficial effects as the methods adopted, run or implemented by the applications stored therein.

[0197] It should be noted that:

[0198] Numerous specific details are set forth in the specification provided herein. However, it will be understood that embodiments of this disclosure may be practiced without these specific details. In some instances, well-known structures and techniques have not been shown in detail so as not to obscure the understanding of this specification.

[0199] Similarly, it should be understood that, in order to simplify this disclosure and aid in understanding one or more of the various inventive aspects, in the above description of exemplary embodiments of this disclosure, various features of this disclosure are sometimes grouped together in a single embodiment, figure, or description thereof. However, this approach to disclosure should not be construed as reflecting a schematic diagram in which the claimed disclosure requires more features than are expressly recited in each claim. Rather, as reflected in the following claims, inventive aspects lie in fewer than all features of a single foregoing disclosed embodiment. Therefore, the claims following the detailed description are hereby expressly incorporated into that detailed description, wherein each claim itself is a separate embodiment of this disclosure.

[0200] Furthermore, those skilled in the art will understand that although some embodiments described herein include certain features included in other embodiments but not others, combinations of features from different embodiments are intended to be within the scope of this disclosure and form different embodiments. For example, in the following claims, any of the claimed embodiments can be used in any combination.

[0201] The above description is merely a preferred embodiment of this disclosure, but the scope of protection of this disclosure is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this disclosure should be included within the scope of protection of this disclosure. Therefore, the scope of protection of this disclosure should be determined by the scope of the claims.

Claims

1. A method of decoding a voxel grid, characterized in that, The method comprises: acquiring decoding metadata, a pruning mask, a coarse residual and a reference voxel index from a code stream; constructing an empty voxel grid based on the decoding metadata; the decoding metadata comprises the size, resolution and coarse quantization step length of the voxel grid; the empty voxel grid comprises a plurality of sub-voxels; for any one sub-voxel in the empty voxel grid, screening out an unpruned voxel from the plurality of sub-voxels through the pruning mask of the sub-voxel; determining the corresponding reference voxel for each unpruned voxel according to the reference voxel index; for any one unpruned voxel in the plurality of unpruned voxels, determining the coarse reconstruction value of the unpruned voxel as the sum of the reference voxel corresponding to the unpruned voxel and the coarse residual; using the coarse quantization step length to dequantize the coarse reconstruction value of each unpruned voxel to obtain the first voxel value of the unpruned voxel; the reference voxel index comprises the index values of the plurality of unpruned voxels and the reference voxels corresponding to each index value; determining the corresponding reference voxel for each unpruned voxel according to the reference voxel index comprises: for any one unpruned voxel in the plurality of unpruned voxels, determining the reference voxel corresponding to the index value of the unpruned voxel based on the reference voxel index; the reference voxel refers to the sub-voxel in the empty voxel grid corresponding to the index value.

2. The method of claim 1, wherein, screening out the unpruned voxel from the plurality of sub-voxels through the pruning mask of the sub-voxel comprises: if the pruning mask of the sub-voxel is a first value, determining that the sub-voxel is a pruned voxel; if the pruning mask of the sub-voxel is a second value, determining that the sub-voxel is an unpruned voxel.

3. The method according to claim 1 or 2, characterized in that, After obtaining the first voxel value of the unpruned voxel, the method further comprises: screening out key voxels from the plurality of unpruned voxels; for any one key voxel in the plurality of key voxels, dequantizing the fine residual of the key voxel using a fine quantization step length to obtain a reconstruction residual; taking the sum of the first voxel value and the reconstruction residual as the fine reconstruction value of the key voxel.

4. The method of claim 3, wherein, screening out key voxels from the plurality of unpruned voxels comprises: acquiring a key voxel mask from the code stream; for any unpruned voxel, if the key voxel mask of the unpruned voxel is a preset mask value, determining that the unpruned voxel is a key voxel.

5. An encoding method of a voxel grid, characterized by, The method comprises: generating a first voxel grid according to images of a plurality of perspectives; the first voxel grid is used to represent the three-dimensional structure of a current scene; each first voxel in the first voxel grid is used to represent the radiation information at different positions; the radiation information comprises color information and density information; the radiation information is determined by: emitting a plurality of rays from each image pixel position according to the input camera pose, uniformly sampling on the rays, and for each sampling point, performing linear interpolation on the voxel grid to obtain color feature values and body density values; pruning the pruned voxels in the first voxel grid to obtain a second voxel grid; the pruned voxels refer to the first voxels with importance scores less than a first importance score threshold, and any importance score is used to represent a contribution value of the first voxel to a rendering result of an image; for any second voxel in the second voxel grid, a reference voxel corresponding to the second voxel is selected, the reference voxel refers to a second voxel in the second voxel grid adjacent to the second voxel and with a minimum absolute error sum; based on a plurality of second voxels and a plurality of reference voxels corresponding to the plurality of second voxels one by one, encoding the second voxel grid.

6. The method of claim 5, wherein, based on a plurality of second voxels and a plurality of reference voxels corresponding to the plurality of second voxels one by one, encoding the second voxel grid, comprising: calculating a second absolute error sum between each second voxel and its corresponding reference voxel to obtain a plurality of second absolute error sums corresponding to the plurality of second voxels one by one; using the average value of the plurality of second absolute error sums as a first weighting coefficient, weighting processing a first loss function of the first voxel grid using the first weighting coefficient to obtain a second loss function; adjusting the second voxel grid through the second loss function to obtain a third voxel grid; the third voxel grid includes a plurality of third voxels; using a coarse quantization step to quantize the plurality of third voxels respectively to obtain a plurality of quantized third voxels; subtracting each quantized third voxel from its corresponding reference voxel to obtain a corresponding coarse residual error; according to the coarse residual error corresponding to each quantized third voxel, performing dequantization processing on the each quantized third voxel to obtain a coarse reconstructed voxel grid corresponding to the third voxel grid; the coarse reconstructed voxel grid includes a plurality of fourth voxels; selecting a plurality of key voxels from the plurality of fourth voxels; the key voxels refer to the fourth voxels with importance scores greater than a second importance score threshold in the plurality of fourth voxels; for any one key voxel in the plurality of key voxels, determining a target second voxel with the same position as the key voxel from the second voxel grid; calculating a third absolute error sum between the key voxel and the target second voxel to obtain a plurality of third absolute error sums corresponding to the plurality of key voxels one by one; using the average value of the plurality of third absolute error sums as a second weighting coefficient, weighting processing the first loss function using the second weighting coefficient to obtain a third loss function; adjusting the plurality of key voxels in the coarse reconstructed voxel grid through the third loss function to obtain a fourth voxel grid; the fourth voxel grid includes a plurality of adjusted key voxels; for any one adjusted key voxel in the plurality of adjusted key voxels, subtracting the adjusted key voxel from a second voxel with the same position in the second voxel grid; quantizing the subtraction result to obtain a fine residual error; write the coarse residual, the fine residual, the reference voxel index, a pruned voxel mask and a key voxel mask into a bitstream; the pruned voxel mask is used to determine the position of a pruned voxel in an empty voxel grid, and the key voxel mask is used to determine the position of a key voxel in the empty voxel grid.

7. A decoding apparatus of a voxel grid, characterized by, The device comprises: a data acquisition module configured to acquire decoding metadata, a pruning mask, a coarse residual and a reference voxel index from a bitstream; an empty voxel grid construction module configured to construct an empty voxel grid based on the decoding metadata; the decoding metadata comprises the size, resolution and coarse quantization step length of a voxel grid; the voxel grid comprises a plurality of sub-voxels; an unpruned voxel screening module configured to screen out, for any one sub-voxel in the empty voxel grid, an unpruned voxel from the plurality of sub-voxels through the pruning mask of the sub-voxel; a reference voxel determination module configured to determine, for each unpruned voxel, a corresponding reference voxel according to the reference voxel index; a coarse reconstruction value determination module configured to determine, for any one unpruned voxel of the plurality of unpruned voxels, a coarse reconstruction value of the unpruned voxel as the sum of the reference voxel corresponding to the unpruned voxel and the coarse residual; a dequantization module configured to dequantize the coarse reconstruction value of each unpruned voxel using the coarse quantization step length to obtain a first voxel value of the unpruned voxel; The reference voxel index comprises index values of a plurality of unpruned voxels and reference voxels corresponding to each index value; determining, for each unpruned voxel, a corresponding reference voxel according to the reference voxel index comprises: determining, for any one unpruned voxel of the plurality of unpruned voxels, a reference voxel corresponding to the index value of the unpruned voxel based on the reference voxel index; the reference voxel refers to a sub-voxel in the empty voxel grid corresponding to the index value.

8. An encoding apparatus of a voxel grid, characterized by, The device comprises: a first voxel grid generation module configured to generate a first voxel grid according to images of a plurality of perspectives; the first voxel grid is used to represent the three-dimensional structure of a current scene; each first voxel in the first voxel grid is used to represent radiation information at different positions; the radiation information comprises color information and density information; The radiation information is determined by: emitting a plurality of rays from each image pixel position according to an input camera pose, uniformly sampling on the rays, and for each sampling point, performing linear interpolation on the voxel grid to obtain color feature values and body density values; a pruning module configured to prune pruned voxels in the first voxel grid to obtain a second voxel grid; the pruned voxels refer to first voxels with importance scores less than a first importance score threshold; any importance score is used to represent the contribution value of the first voxel to the image rendering result; a reference voxel determination module configured to select, for any second voxel in the second voxel grid, a reference voxel corresponding to the second voxel; the reference voxel refers to a second voxel in the second voxel grid adjacent to the second voxel and having the smallest absolute error. The encoding module is configured to encode the second voxel grid based on a plurality of second voxels and a plurality of reference voxels corresponding to the plurality of second voxels one by one.

9. A computer device, comprising: The method comprises the following steps: A memory and a processor are in communication connection with each other, and the memory stores computer instructions. The processor executes the computer instructions to perform the method in any one of claims 1 to 6.

Images