Method and system for fast determination of dynamic 3D point cloud coding mode based on lightweight neural network

Abstract

A method and system for quickly determining a dynamic 3D point cloud coding mode based on a lightweight neural network relates to the field of video coding technology. The present invention performs a predictive coding process of a merge / skip mode according to the original coding scheme of HEVC, extracts the feature information of the current coding CU and processes it to obtain the input feature x1 of the T1 network, inputs x1 into the T1 network to obtain a judgment result, and if the T1 network determines that the merge / skip mode is the best mode, the decision process of the subsequent mode is skipped and the division stage is directly entered, otherwise the predictive coding process of the inter 2N×2N mode is continued; the feature information of the current coding CU is extracted and processed to obtain the input feature x2 of the T2 network, and x2 is input into the T2 network to obtain a judgment result, if the T2 network determines that the merge / skip mode or the inter 2N×2N mode is the best coding mode, the subsequent mode decision process is skipped and the division stage is directly entered, otherwise the subsequent mode decision process is continued. Compared with the existing method, the present invention can greatly shorten the inter-frame coding time of dynamic 3D point clouds.

Description

Technical Field

[0001] The present invention relates to the field of video coding technology, and in particular to a method and system for quickly determining a dynamic 3D point cloud coding mode based on a lightweight neural network. Background Art

[0002] 3D industrial production and commercial scanning equipment, such as VR equipment, RGBD cameras and light detection and ranging, are becoming more common and cheaper than ever before. These sensing devices can scan and generate a large amount of 3D data, and because they can represent 3D data in a more realistic mode, 3D visual representation methods such as polygonal meshes, light fields and point clouds are becoming more and more popular. Among these three-dimensional digital representation formats, the acquisition of point clouds has achieved a good compromise between convenience, authenticity and data operability. Therefore, point cloud technology is also being used more and more frequently.

[0003] Point cloud technology has laid a solid foundation for the advancement of visual technology. In recent years, 3D technologies that have appeared more and more frequently in daily life include immersive virtual reality (VR), augmented reality (AR) and mixed reality (MR). These advanced technologies have a disruptive role in many applications, including exploration of historical sites and art museums, immersive real-time remote communications, interactive portrait games, mobile navigation, etc. However, due to the extremely large amount of data carried by point clouds, it is not possible to present these large amounts of raw point cloud data in a cached or streamed form, which means that encoding and compression of point clouds is a very complex process and encoding takes a very long time. In order to overcome such obstacles, there is a need to generate efficient point cloud compression (PCC). Summary of the invention

[0004] One of the purposes of the present invention is to provide a method for quickly determining a dynamic 3D point cloud coding mode based on a lightweight neural network, which can reduce the inter-frame coding time of the dynamic 3D point cloud without substantially affecting the coding quality.

[0005] In order to solve the above technical problems, the present invention adopts the following technical solution: a method for quickly determining a dynamic 3D point cloud coding mode based on a lightweight neural network, which comprises the following steps:

[0006] Step 1, data collection: extract data, the samples will be used for training T1 and T2 networks, and the extracted features include: maximum two-partition prediction distortion variance, maximum four-partition prediction distortion variance, coding block flag, coding block depth, quantization parameter, real-time coding order, coding unit type, front layer mode and merge / skip mode coding flag.

[0007] Step 2, data processing: First, the maximum two-partition prediction distortion variance and the maximum four-partition prediction distortion variance in the proposed features are normalized to distinguish between geometric and texture image encoding; then, the above two features are truncated and assigned to boundary values ​​to limit them to the range of [0,1]; then, data cleaning is performed based on the maximum two-partition prediction distortion variance and the maximum four-partition prediction distortion variance of the samples; finally, the quantization parameters are normalized to the maximum and minimum values.

[0008] Step 3, parameter training: For the CU samples of the T1 network in the training set, as the input of the lightweight neural network T1, a fully connected layer is used as a classifier to determine whether the merge / skip mode is the best mode according to the sample input; for the CU samples of the T2 network in the training set, as the input of the lightweight neural network T2, a fully connected layer is used as a classifier to determine whether the merge / skip mode or the inter 2N×2N mode is the best mode according to the sample input; finally, a fast mode decision is made based on the classification results.

[0009] Step 4, model deployment: After the merge / skip mode decision is completed, if the T1 network determines that the merge / skip mode is the best mode for the current coding CU, all subsequent mode decision processes are skipped and the division stage is directly entered, otherwise the subsequent mode decision process is continued; after the inter2N×2N mode decision is completed, if the T2 network determines that the merge / skip mode or the inter2N×2N mode is the best mode for the current coding CU, all subsequent mode decision processes are skipped and the division stage is directly entered, otherwise the subsequent mode decision process is continued.

[0010] Preferably, in step 1, when calculating the maximum two-partition prediction distortion variance or the maximum four-partition prediction distortion variance, the block is divided into two or four parts horizontally or vertically, and finally the value of the maximum prediction distortion transform variance in the sub-block is extracted as the feature.

[0011] More preferably, in step 1, the features of the T1 and T2 networks should be extracted separately. The features extracted by the T1 network are the maximum two-partition prediction distortion variance, the maximum four-partition prediction distortion variance, the coding block flag, the coding block depth, the quantization parameter, the real-time coding order, the coding unit type, and the front layer mode. The feature merge / skip mode coding flag is only used in the T2 network.

[0012] Furthermore, the prediction distortion transform is calculated based on a prediction distortion transform matrix, and the prediction distortion transform matrix is ​​defined as follows:

[0013]

[0014] in It is the pixel value matrix of the coding mode prediction for the current coding CU with a width of w and a height of h. is the original pixel value matrix of the current encoding CU, It is the prediction distortion transformation matrix with a width of w and a height of h obtained based on the prediction distortion. is the transform coefficient, where α has different values ​​depending on the category of the current coded image:

[0015]

[0016] According to the matrix According to the horizontal or vertical two-partition or four-partition method, the variance of the prediction distortion transform of different sub-blocks is calculated. The variance value of the four sub-blocks under the two-partition is the largest, which is the maximum two-partition prediction distortion variance BDV. The calculation of the maximum four-partition prediction distortion variance QDV is similar, and is defined as follows:

[0017]

[0018]

[0019] Preferably, in step 2, the normalization method of the maximum two-partition prediction distortion variance and the maximum four-partition prediction distortion variance is defined as follows:

[0020]

[0021]

[0022]

[0023] BDV and QDV represent the maximum two-partition prediction distortion variance and the maximum four-partition prediction distortion variance, respectively. i It represents the maximum value of the prediction distortion transform variance in the sub-block divided by horizontal or vertical two-partition or four-partition method. The normalization coefficient β has different fixed values ​​according to the different encoded geometry and texture maps.

[0024] More preferably, in step 2, the data cleaning method is: for the samples of the T1 network, if BDV or QDV<0.2 is satisfied and the final coding mode is non-merge / skip mode, then delete it; if BDV or QDV>0.8 is satisfied and the final coding mode is merge / skip mode, then delete it; for the samples of the T2 network, if BDV or QDV<0.1 is satisfied and the final coding mode is other than merge / skip mode and inter 2N×2N mode, then delete it; if BDV or QDV>0.9 is satisfied and the final coding mode is merge / skip mode or inter 2N×2N mode, then delete it.

[0025] More preferably, in step 2, the maximum and minimum values ​​of the quantization parameter QP are normalized as follows:

[0026]

[0027] Furthermore, for the algorithm flow deployed in the codec, the T1 and T2 networks have their own usage conditions. The usage condition of the T1 network is that the current coding CU does not belong to a placeholder image and is not an I frame, and the usage condition of the T2 network is that the current coding CU does not belong to a placeholder image, is not an I frame, and the T1 network does not make a fast mode decision.

[0028] In addition, the present invention also provides a system for quickly determining a dynamic 3D point cloud coding mode based on a lightweight neural network, which comprises:

[0029] Data collection module: used to extract data. The samples will be used for training T1 and T2 networks. The extracted features include: maximum two-partition prediction distortion variance, maximum four-partition prediction distortion variance, coding block flag, coding block depth, quantization parameter, real-time coding order, coding unit type, front layer mode and merge / skip mode coding flag.

[0030] Data processing module: used to normalize the maximum two-partition prediction distortion variance and the maximum four-partition prediction distortion variance in the proposed features to distinguish between geometric and texture image encoding; then, the above two features are truncated and assigned to boundary values ​​to limit them to the range of [0,1]; then, data cleaning is performed based on the maximum two-partition prediction distortion variance and the maximum four-partition prediction distortion variance of the samples; finally, the quantization parameters are normalized to the maximum and minimum values.

[0031] Parameter training module: used to use the CU samples of the T1 network in the training set as the input of the lightweight neural network T1, using a fully connected layer as a classifier to determine whether the merge / skip mode is the best mode based on the sample input; and used to use the CU samples of the T2 network in the training set as the input of the lightweight neural network T2, using a fully connected layer as a classifier to determine whether the merge / skip mode or the inter 2N×2N mode is the best mode based on the sample input; finally, a fast mode decision is made based on the classification results.

[0032] Model deployment module: after the merge / skip mode decision is completed, if the T1 network determines that the merge / skip mode is the best mode for the current coding CU, then all subsequent mode decision processes are skipped and the partitioning stage is directly entered, otherwise, the subsequent mode decision process is continued; after the inter2N×2N mode decision is completed, if the T2 network determines that the merge / skip mode or the inter 2N×2N mode is the best mode for the current coding CU, then all subsequent mode decision processes are skipped and the partitioning stage is directly entered, otherwise, the subsequent mode decision process is continued.

[0033] The present invention utilizes the maximum two-partition prediction distortion variance, the maximum four-partition prediction distortion variance, the coding block flag, the coding block depth, the quality parameter, the real-time coding order, the coding unit type, the front layer mode and the merge / skip mode flag added separately by T2 as the input features of the neural network, and skips part of the mode decision process without bringing too much additional computational complexity through the lightweight neural network to achieve the purpose of optimizing the encoding time. At the same time, the present invention also pre-processes the sample set to improve the lightweight neural network's ability to fit the data, and adds the RD cost of the coding mode to determine whether the coding mode is within an acceptable range during the training of the neural network, thereby assisting the training of the neural network to be compatible with the coding mode with higher coding efficiency. Therefore, while substantially not reducing the coding quality, the computational complexity of the dynamic 3D point cloud inter-frame coding can be effectively reduced, and the coding speed can be improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] Figure 1 It is the extraction model of the neural network training data in the present invention;

[0035] Figure 2 It is a flowchart of fast decision making of inter-frame coding mode for dynamic 3D point cloud in the present invention;

[0036] Figure 3 This is an example of a method of dividing a CU coding block into two or four. DETAILED DESCRIPTION

[0037] In order to facilitate the understanding of those skilled in the art, the present invention is further described below in conjunction with embodiments and drawings. The contents mentioned in the implementation modes are not intended to limit the present invention.

[0038] A method for quickly determining a dynamic 3D point cloud coding mode based on a lightweight neural network comprises the following steps:

[0039] Step 1, data collection: Figure 1 The feature extraction model shown is used for data extraction, and the samples will be used for training T1 and T2 networks. The extracted features include: maximum two-partition prediction distortion variance, maximum four-partition prediction distortion variance, coding block flag, coding block depth, quantization parameter, real-time coding order, coding unit type, front layer mode and merge / skip mode coding flag.

[0040] Further, when calculating the maximum two-partition prediction distortion variance or the maximum four-partition prediction distortion variance, the following is used: Figure 3 The horizontal or vertical two-partition or four-partition method is shown, and finally the value with the largest prediction distortion transform variance in the sub-block is extracted as the feature.

[0041] Furthermore, the features of T1 and T2 networks should be extracted separately. The features extracted by T1 network are the first eight, namely, maximum two-partition prediction distortion variance, maximum four-partition prediction distortion variance, coding block flag, coding block depth, quantization parameter, real-time coding order, coding unit type, and front layer mode. The feature merge / skip mode coding flag is only used in T2 network.

[0042] Furthermore, the prediction distortion transform is calculated based on a prediction distortion transform matrix, and the prediction distortion transform matrix is ​​defined as follows:

[0043]

[0044] in It is the pixel value matrix of the coding mode prediction for the current coding CU with a width of w and a height of h. is the original pixel value matrix of the current encoding CU, It is the prediction distortion transformation matrix with a width of w and a height of h obtained based on the prediction distortion. is the transformation coefficient, where α has different values ​​depending on the category of the current coded image.

[0045]

[0046] According to the matrix according to Figure 3The division method shown can calculate the variance of the prediction distortion transform of different sub-blocks. The largest variance value among the four sub-blocks under the two-partition is the maximum two-partition prediction distortion variance BDV. The calculation of the maximum four-partition prediction distortion variance QDV is similar, and has the following definition:

[0047]

[0048]

[0049] Step 2, data processing: First, the maximum two-partition prediction distortion variance and the maximum four-partition prediction distortion variance in the proposed features are normalized to distinguish between geometric map and texture map encoding; then, in order to limit the above two features to the range of [0,1], they are truncated and assigned to boundary values; then, data cleaning is performed based on the maximum two-partition prediction distortion variance and the maximum four-partition prediction distortion variance of the samples; finally, the quantization parameters are normalized to the maximum and minimum values;

[0050] Furthermore, the normalization method of the maximum two-partition prediction distortion variance and the maximum four-partition prediction distortion variance is defined as follows:

[0051]

[0052]

[0053]

[0054] BDV and QDV represent the maximum two-partition prediction distortion variance and the maximum four-partition prediction distortion variance, respectively. i Indicates Figure 3 The value with the largest variance of the predicted distortion transform in the divided sub-block shown in , and the normalization coefficient β has different fixed values ​​according to the different encoded geometry and texture maps.

[0055] Furthermore, the data cleaning method is that for the samples of the T1 network, if BDV or QDV<0.2 is satisfied and the final coding mode is not the merge / skip mode, it will be deleted; if BDV or QDV>0.8 is satisfied and the final coding mode is the merge / skip mode, it will be deleted; for the samples of the T2 network, if BDV or QDV<0.1 is satisfied and the final coding mode is other than the merge / skip mode and the inter 2N×2N mode, it will be deleted; if BDV or QDV>0.9 is satisfied and the final coding mode is the merge / skip mode or the inter 2N×2N mode, it will be deleted.

[0056] Furthermore, the normalization definition of the maximum and minimum values ​​of the quantization parameter QP is as follows:

[0057]

[0058] Step 3, parameter training: For the CU samples of the T1 network in the training set, as the input of the lightweight neural network T1, a fully connected layer is used as a classifier to determine whether the merge / skip mode is the best mode according to the sample input; for the CU samples of the T2 network in the training set, as the input of the lightweight neural network T2, a fully connected layer is used as a classifier to determine whether the merge / skip mode or the inter 2N×2N mode is the best mode according to the sample input; finally, a fast mode decision is made based on the classification results.

[0059] Step 4, model deployment: Figure 2 As shown in the mode decision flowchart which is different from the original scheme, after the merge / skip mode decision is completed, if the T1 network determines that the merge / skip mode is the best mode for the current coding CU, all subsequent mode decision processes are skipped and the division stage is directly entered, otherwise the subsequent mode decision process is continued; after the inter 2N×2N mode decision is completed, if the T2 network determines that the merge / skip mode or the inter 2N×2N mode is the best mode for the current coding CU, all subsequent mode decision processes are skipped and the division stage is directly entered, otherwise the subsequent mode decision process is continued.

[0060] Furthermore, for the algorithm flow deployed in the codec, the T1 and T2 networks have their own usage conditions. The usage condition of the T1 network is that the current coding CU does not belong to a placeholder image and is not an I frame, and the usage condition of the T2 network is that the current coding CU does not belong to a placeholder image, is not an I frame, and the T1 network does not make a fast mode decision.

[0061] The difference variance between the brightness pixel value of the CU and the brightness pixel value predicted by the mode reflects to a large extent whether the current coding mode is the optimal mode. If the difference variance between the brightness pixel value of the current CU and the reconstructed brightness pixel value is large, the current CU needs to be further divided. Therefore, the present invention uses the difference variance between the brightness pixel value of the current CU and the brightness pixel value predicted by the mode as an important input feature for the network. In addition, compared with the previous heuristic method, using a lightweight neural network as a self-learning classification standard is closer to the actual situation, and can effectively reduce the inter-frame coding time of V-PCC+HEVC while basically keeping the coding quality unchanged.

[0062] For the extraction model of the neural network training data provided in this embodiment, Figure 1As shown in the figure, the transparent rectangular box represents the original processing step; the dark rectangular box represents the feature extraction step; the solid arrow represents the CU information direction; the hollow arrow represents the feature transmission direction; the three-dimensional square represents the network trained with T1 features, which is only turned on when extracting T2 features. If the network is turned off, only route ① is valid; route ① represents T1 network output>0.5; route ② represents T1 network output≤0.5. The principle of this extraction model is as follows:

[0063] Step 1, use merge / skip mode prediction for the current coding block;

[0064] Step 2, calculate the maximum two-partition prediction distortion variance, the maximum four-partition prediction distortion variance, the coding block flag, the coding block depth, the quantization parameter, the real-time coding order, the coding unit type and the front layer mode of the current coding block, and the obtained features are recorded as T1 features;

[0065] Step 3: If it is necessary to extract T2 features, input T1 features into the T1 network, and determine whether a fast mode decision is needed based on the network prediction results. If so, proceed to Step 8, otherwise proceed to Step 4. If it is not necessary to extract T2 features, proceed directly to Step 4.

[0066] Step 4, use the inter 2N×2N mode prediction for the current coding block;

[0067] Step 5, calculate the maximum two-partition prediction distortion variance, the maximum four-partition prediction distortion variance, the coding block flag, the coding block depth, the quantization parameter, the real-time coding order, the coding unit type, the front layer mode and the merge / skip mode coding flag of the current coding block, and the obtained features are recorded as T2 features;

[0068] Step 6, predict the current coding block using other coding modes;

[0069] Step 7, marking the saved T1 or T2 feature set according to the results of pattern prediction;

[0070] Step 8, proceed with the subsequent encoding process.

[0071] The extraction of T1 and T2 features is performed twice, and only one model feature is extracted each time. When extracting T2 features, the coding unit samples that require fast mode decision for T1 network judgment will not be included in the feature set.

[0072] The method marked in Step 7 is:

[0073] If the T1 feature is currently extracted, the coding unit sample that finally selects the merge / skip mode as the best coding mode is marked as 0, and the CU sample that selects other modes as the best coding mode is marked as 1;

[0074] If the T2 feature is currently extracted, the coding unit samples that finally select the merge / skip mode or the inter 2N×2N mode as the best coding mode are marked as 0, and the coding unit samples that select other modes as the best coding mode are marked as 1.

[0075] The flowchart of the fast decision of the inter-frame coding mode for dynamic 3D point cloud in this embodiment is as follows: Figure 2 As shown in the figure, the solid arrows represent the direction of information presentation of the coding unit; the hollow arrows represent the direction of information presentation of the coding unit placeholder; the dotted arrows represent the mode skipping direction; the rectangular box represents the mode decision process; and the three-dimensional square represents the neural network module. The principle of the process is as follows:

[0076] Step 1, use merge / skip mode to predict the current coding unit;

[0077] Step 2: According to the mode prediction result of the current coding block and the corresponding placeholder information in the placeholder map, the input features of the T1 network are obtained, and the input features are input into the T1 network to obtain the neural network prediction result. According to the result, it is determined whether the subsequent coding mode needs to be skipped. If not, it goes to Step 3, and if yes, it goes to Step 6;

[0078] Step 3, use the inter 2N×2N mode to predict the current coding unit;

[0079] Step 4: According to the mode prediction result of the current coding block and the corresponding placeholder information in the placeholder map, the input features of the T2 network are obtained, and the input features are input into the T2 network to obtain the neural network prediction result. According to the result, it is determined whether the subsequent coding mode needs to be skipped. If not, it goes to Step 5, and if yes, it goes to Step 6;

[0080] Step 5, use asymmetric and intra-frame mode prediction coding for the current coding unit;

[0081] Step 6: further divide the current coding unit into coding units.

[0082] In addition, for Figure 3The CU coding block in the 4-partition method is divided into two or four parts. The width and height of a current coding block may be any of the four cases of 64×64, 32×32, 16×16, and 8×8. It is uniformly expressed as 2N×2N. In order to reduce the overall variance of the prediction distortion matrix to describe the error caused by the current coding mode to predict the coding unit, the entire 2N×2N block is divided into two 2N×N sub-areas, two N×2N sub-areas, and four Sub-areas, divided into four Sub-region. Calculate the maximum variance of each sub-region respectively.

[0083] The following simulation experiments are conducted to verify the encoding performance of the method for quickly determining the dynamic 3D point cloud encoding mode based on a lightweight neural network proposed in this embodiment.

[0084] In order to evaluate the feasibility and effectiveness of the above method, the dynamic 3D point cloud coding reference software TMC2-v15.0 and HEVC reference software HM16.18+SCM8.8 were used as the test platform for independent execution. The test sequence includes 5 different test sequences with a resolution of 1024×1024×1024 provided by 8iVSLF: queen, loot, redandblack, soldier, longdress. The coding quantization parameter combination (QPs) is set to ([32, 42], [28, 37], [24, 32], [20, 27], [16, 22]), and the encoding configuration is RA (Random Access). The point-to-point error (D1) and point-to-plane error (D2) are used to evaluate the geometric image distortion, the mean square error (d) for different attributes (Luma, Chroma Cb, Chroma Cr) is used to evaluate the texture image distortion, and ΔT is used to measure the time saving. The mathematical definition of the above measure is:

[0085]

[0086]

[0087]

[0088]

[0089] in represents the encoding time of the original test model of the i-th group QP of a test sequence, Indicates the encoding time after the method provided in this embodiment is applied to the original test model.

[0090] Table 1: Performance comparison results of this embodiment and TMC2v15.0+HM16.20+SCM8.8 (unit: %)

[0091]

[0092] From the comparison results of encoding time saving and encoding bit rate increase in Table 1, it can be seen that the method provided in this embodiment saves the encoding time of each test sequence by 55.2%, 53.8%, 38.4%, 52.1% and 36.8% on average, and the total time saving is 47.3% on average. The mean square error used to measure the distortion of geometric graph coding increases by -0.5%, -0.5% and 0.1%, -0.2% on average, and the mean square error used to measure the distortion of attribute graph coding increases by 0.9%, -1.3%, 0.5% and 0.1%, -1.3%, -0.3% on average.

[0093] The above experimental results show that the method of the present invention can effectively produce good time optimization effects for different point cloud sequences, and the degradation of video encoding quality is within an acceptable range. In summary, the method of the present invention can effectively achieve a balance between encoding time saving and bit rate increase when the encoding quality degradation is within the range acceptable to the human eye.

[0094] In recent years, in view of the outstanding performance of neural networks in the fields of computer vision and video coding compression, neural networks for point cloud video coding are in urgent need of research. In order to meet the demand for time saving, a lightweight neural network is used to quickly determine the process of coding mode decision-making, and on the basis of saving lengthy coding time, the extra time consumption brought by the algorithm is minimized, and according to the characteristics of the algorithm, the reduction in video coding quality is within an acceptable level. The present invention improves and innovates the algorithm on V-PCC. In V-PCC, the point cloud is initially divided into 3D patches, and then these 3D patches are projected onto a two-dimensional (2D) plane and packaged into geometric and attribute videos, and then the blanks in the geometric and attribute videos are filled to maintain spatial continuity and improve video compression efficiency. Finally, high-efficiency video coding (HEVC) is used to compress the geometric and attribute videos. The coding calculation complexity is effectively reduced while ensuring that the coding quality is almost unchanged.

Claims

1. A method for quickly determining the coding mode of dynamic 3D point clouds based on a lightweight neural network, characterized in that: The following steps are involved: Step 1, data collection: extract data, the samples will be used for training T1 and T2 networks, the extracted features include: maximum two-partition prediction distortion variance, maximum four-partition prediction distortion variance, coding block flag, coding block depth, quantization parameter, real-time coding order, coding unit type, front layer mode and merge / skip mode coding flag; Step 2, data processing: First, the maximum two-partition prediction distortion variance and the maximum four-partition prediction distortion variance in the proposed features are normalized to distinguish between geometric map and texture map encoding; then, the two features of the maximum two-partition prediction distortion variance and the maximum four-partition prediction distortion variance are truncated and assigned to boundary values ​​to limit them to the range of [0,1]; then, data cleaning is performed based on the maximum two-partition prediction distortion variance and the maximum four-partition prediction distortion variance of the samples; finally, the quantization parameters are normalized to the maximum and minimum values; Step 3, parameter training: For the CU samples of the T1 network in the training set, as the input of the lightweight neural network T1, a fully connected layer is used as a classifier to determine whether the merge / skip mode is the best mode according to the sample input; for the CU samples of the T2 network in the training set, as the input of the lightweight neural network T2, a fully connected layer is used as a classifier to determine whether the merge / skip mode or the inter 2N×2N mode is the best mode according to the sample input; finally, a fast mode decision is made based on the classification results; Step 4, model deployment: After the merge / skip mode decision is completed, if the T1 network determines that the merge / skip mode is the best mode for the current coding CU, then all subsequent mode decision processes are skipped and the partitioning stage is directly entered, otherwise the subsequent mode decision process is continued; After the inter2N×2N mode decision is completed, if the T2 network determines that the merge / skip mode or the inter 2N×2N mode is the best mode for the current coding CU, then all subsequent mode decision processes are skipped and the partitioning stage is directly entered, otherwise the subsequent mode decision process is continued; In step 1, when calculating the maximum two-partition prediction distortion variance or the maximum four-partition prediction distortion variance, the block is divided into two or four parts horizontally or vertically, and finally the value with the maximum prediction distortion transform variance in the sub-block is extracted as the feature; In step 2, the normalization method of the maximum two-partition prediction distortion variance and the maximum four-partition prediction distortion variance is defined as follows: Where BDV and QDV represent the maximum two-partition prediction distortion variance and the maximum four-partition prediction distortion variance, respectively, and the normalization coefficient β has different fixed values ​​according to the different encoded geometry and texture maps; In step 2, the data cleaning method is as follows: for samples of the T1 network, if BDV or QDV < 0.2 and the final coding mode is not the merge / skip mode, delete them; if BDV or QDV > 0.8 and the final coding mode is the merge / skip mode, delete them; for samples of the T2 network, if BDV or QDV < 0.1 and the final coding mode is other than the merge / skip mode and the inter 2N×2N mode, delete them; if BDV or QDV > 0.9 and the final coding mode is the merge / skip mode or the inter 2N×2N mode, delete them; In step 2, the maximum and minimum values ​​of the quantization parameter QP are normalized as follows:

2. The method for quickly determining a dynamic 3D point cloud coding mode based on a lightweight neural network according to claim 1, characterized in that: In step 1, the features of T1 and T2 networks should be extracted separately. The features extracted by T1 network are maximum two-partition prediction distortion variance, maximum four-partition prediction distortion variance, coding block flag, coding block depth, quantization parameter, real-time coding order, coding unit type, and front layer mode. The feature merge / skip mode coding flag is only used in T2 network.

3. The method for quickly determining a dynamic 3D point cloud coding mode based on a lightweight neural network according to claim 1, characterized in that: The prediction distortion transform is calculated based on the prediction distortion transform matrix, which is defined as follows: in It is the pixel value matrix of the coding mode prediction for the current coding CU with a width of w and a height of h. is the original pixel value matrix of the current encoding CU, It is the prediction distortion transformation matrix with a width of w and a height of h obtained based on the prediction distortion. is the transform coefficient, where α has different values ​​depending on the category of the current coded image: According to the matrix According to the horizontal or vertical two-partition or four-partition method, the variance of the prediction distortion transform of different sub-blocks is calculated. The variance value of the four sub-blocks under the two-partition is the largest, which is the maximum two-partition prediction distortion variance BDV. The calculation of the maximum four-partition prediction distortion variance QDV is similar, and is defined as follows: Var i Indicates the value with the maximum prediction distortion transform variance among the sub-blocks divided by two or four horizontal or vertical divisions.

4. The method for quickly determining a dynamic 3D point cloud coding mode based on a lightweight neural network according to claim 1, characterized in that: For the algorithm flow deployed in the codec, the T1 and T2 networks have their own usage conditions. The usage condition of the T1 network is that the current coding CU does not belong to a placeholder image and is not an I frame. The usage condition of the T2 network is that the current coding CU does not belong to a placeholder image, is not an I frame, and the T1 network does not make a fast mode decision.

5. A system for quickly determining the coding mode of dynamic 3D point cloud based on lightweight neural network, characterized in that: include: Data collection module: used to extract data. The samples will be used for training T1 and T2 networks. The extracted features include: maximum two-partition prediction distortion variance, maximum four-partition prediction distortion variance, coding block flag, coding block depth, quantization parameter, real-time coding order, coding unit type, front layer mode and merge / skip mode coding flag. When calculating the maximum two-partition prediction distortion variance or the maximum four-partition prediction distortion variance, the horizontal or vertical two-partition or four-partition method is used for division. Finally, the value with the largest prediction distortion transform variance in the sub-block is extracted as this feature; Data processing module: used to normalize the maximum two-partition prediction distortion variance and the maximum four-partition prediction distortion variance in the proposed features to distinguish between geometric and texture image encoding; then, the two features of the maximum two-partition prediction distortion variance and the maximum four-partition prediction distortion variance are truncated and assigned to boundary values ​​to limit them to the range of [0,1]; then, data cleaning is performed based on the maximum two-partition prediction distortion variance and the maximum four-partition prediction distortion variance of the samples; finally, the quantization parameters are normalized to the maximum and minimum values; The normalization method for the maximum two-partition prediction distortion variance and the maximum four-partition prediction distortion variance is defined as follows: Where BDV and QDV represent the maximum two-partition prediction distortion variance and the maximum four-partition prediction distortion variance, respectively, and the normalization coefficient β has different fixed values ​​according to the different encoded geometry and texture maps; The data cleaning method is as follows: for samples of the T1 network, if BDV or QDV < 0.2 and the final coding mode is not the merge / skip mode, delete them; if BDV or QDV > 0.8 and the final coding mode is the merge / skip mode, delete them; for samples of the T2 network, if BDV or QDV < 0.1 and the final coding mode is other than the merge / skip mode and the inter2N×2N mode, delete them; if BDV or QDV > 0.9 and the final coding mode is the merge / skip mode or the inter 2N×2N mode, delete them; The definition of the normalization of the maximum and minimum values ​​of the quantization parameter QP is as follows: Parameter training module: used to use the CU samples of the T1 network in the training set as the input of the lightweight neural network T1, and use a fully connected layer as a classifier to determine whether the merge / skip mode is the best mode based on the sample input; and used to use the CU samples of the T2 network in the training set as the input of the lightweight neural network T2, and use a fully connected layer as a classifier to determine whether the merge / skip mode or the inter 2N×2N mode is the best mode based on the sample input; finally, a fast mode decision is made based on the classification results; Model deployment module: after the merge / skip mode decision is completed, if the T1 network determines that the merge / skip mode is the best mode for the current coding CU, then all subsequent mode decision processes are skipped and the partitioning stage is directly entered, otherwise, the subsequent mode decision process is continued; after the inter2N×2N mode decision is completed, if the T2 network determines that the merge / skip mode or the inter 2N×2N mode is the best mode for the current coding CU, then all subsequent mode decision processes are skipped and the partitioning stage is directly entered, otherwise, the subsequent mode decision process is continued.

Images