A dynamic point cloud fast encoding method based on LSTM network prediction

Abstract

The application belongs to the technical field of coding and relates to a dynamic point cloud fast coding method based on LSTM network prediction, which comprises the following steps: obtaining a point cloud and performing preprocessing, inputting the preprocessed point cloud into a trained multi-level prediction division network to obtain a sub-coding unit after point cloud division, and coding the sub-coding unit into a bit stream; the training process of the multi-level prediction division network comprises the following steps: obtaining a dynamic point cloud sequence and performing preprocessing; inputting the preprocessed dynamic point cloud sequence into the multi-level prediction division network to obtain a sub-coding unit after division and a prediction occupation result; updating network parameters according to a loss function value calculated based on the prediction occupation result until a trained network is obtained; the LSTM network of each level is used to predict the occupation result of each level of a CU, and whether division is performed is determined according to the occupation result; meanwhile, the LSTM network of each level is used to perform division prediction of different levels on different CUs in parallel, so that the coding time is saved.

Description

Technical Field

[0001] This invention belongs to the field of coding technology and relates to a fast coding method for dynamic point clouds based on LSTM network prediction. Background Technology

[0002] In recent years, with the rapid development of 3D scanning technology, people's demands for information acquisition have shifted from two-dimensional images and videos to a pursuit of more realistic representations of the world. To achieve a global travel experience without leaving home, 3D scanning technology extracts point sets from the surface of objects, generating data collectively known as point clouds. Each point in a point cloud contains not only spatial coordinate information (x, y, z) but also additional attributes such as color, density, and normal vectors. These point cloud sequences, as the core form of 3D video, provide users with a highly immersive experience, promoting the rapid development of technologies such as virtual reality (VR), augmented reality (AR), real-time 3D conferencing systems, and 3D free-form sports event replay. However, point cloud data is massive in volume, making the selection of appropriate compression algorithms crucial for storing and transmitting this data.

[0003] Depending on the application scenario and acquisition method, point clouds can be divided into three types: static point clouds, dynamic point clouds, and dynamically acquired point clouds. Static point clouds refer to situations where both the object and the acquisition device remain stationary; dynamic point clouds occur when the object moves while the acquisition device remains stationary; and dynamically acquired point clouds refer to situations where the acquisition device itself is in motion. Regarding point cloud data compression, international and domestic standards organizations such as MPEG (Moving Picture Experts Group), AVS (Audio Video Coding Standard), and JPEG (Joint Photographic Experts Group) are conducting algorithm research and developing relevant standards.

[0004] For compression coding of dynamic point clouds, the coding standard was approved and established by the Moving Picture Experts Group (MPEG) in 2020, namely Video-based Point Cloud Compression (V-PCC), and a standard compression encoder, TMC2, was provided. Its basic principle is to project 3D dynamic point clouds onto a two-dimensional image to obtain a video sequence, and then use mature video codecs such as H.265 / HEVC (High Efficiency Video Coding) to compress the geometric information of these dynamic point cloud video sequences. In recent years, research on point cloud sequence compression algorithms has been gradually increasing. Multiple research institutions and organizations, including MPEG, are actively developing efficient dynamic point cloud (DPC) compression technologies and promoting related standardization efforts.

[0005] like Figure 1 As shown, the H.265 / HEVC encoding process is as follows: Video frames are first divided into 64×64 pixel coding units (LCUs). When encoding LCUs, HEVC tries various encoding methods, recording the RD cost of each method and selecting the optimal encoding mode along with other encoding information. Each LCU can be recursively divided into multiple sub-blocks using a quadtree partitioning method until a predetermined minimum block size (e.g., 8×8) is reached. The process proceeds in reverse from the minimum depth. At this point, the sum of the RD costs of each 8×8 sub-CU is compared with the RD cost of its parent layer (16×16 parent block CU). If the sum of the RD costs of the sub-CUs is less than the RD cost of the parent CU, the sub-CUs need to be further divided; otherwise, no further division is performed. The HEVC partitioning process is as follows: Figure 1 As shown; it is worth noting that the partitioning process here uses recursive partitioning and checks the RD cost of each CU. In a 64×64 LCU, 85 possible CU partitioning patterns need to be checked and pre-encoded, which consumes the largest proportion of the encoding time.

[0006] In summary, H.265 / HEVC encoders for encoding dynamic point cloud video sequences typically use brute-force RDO search to determine the partitioning pattern of each CU, which consumes a significant amount of time in CU partitioning, resulting in a significant decrease in inter-frame predictive coding efficiency. Summary of the Invention

[0007] To address the aforementioned problems in the prior art, this invention employs a fast dynamic point cloud encoding method based on LSTM network prediction, comprising: acquiring the current point cloud and preprocessing it; inputting the preprocessed current point cloud into a trained multi-level prediction partitioning network to obtain sub-encoding units after the current point cloud's encoding unit CU is partitioned; and encoding the partitioned sub-encoding units into a bit stream.

[0008] The training process of a multi-level prediction partitioning network includes:

[0009] S1. Obtain the dynamic point cloud sequence and preprocess the dynamic point cloud sequence to obtain the preprocessed dynamic point cloud sequence. Each frame in the preprocessed dynamic point cloud sequence includes multiple coding units (CUs). Each CU includes pixels, occupancy map and pixel residual information.

[0010] S2. Input each CU of each frame of the preprocessed dynamic point cloud sequence into the multi-level prediction partitioning network for partitioning to obtain the sub-coding units after partitioning each CU of each frame and the prediction occupancy result at each level.

[0011] S3. Calculate the loss function value based on the prediction occupancy result of each CU in each frame at each level, update the model parameters based on the loss function value, and obtain the trained multi-level prediction partitioning network when the loss function value is minimized.

[0012] Preprocessing of dynamic point cloud sequences includes:

[0013] S11. Project the dynamic point cloud sequence onto a two-dimensional plane to obtain a video frame sequence of the point cloud; each frame in the video frame sequence of the point cloud includes multiple coding units (CUs).

[0014] S12. Process the video frame sequence to the point cloud to obtain the pixels and occupancy map of each CU in each frame;

[0015] S13. Perform intra-frame coding and inter-frame coding on the video frame sequence to the point cloud to obtain the intra-frame pixels and inter-frame pixels of each CU in each frame.

[0016] S14. Calculate the pixel residual information of each CU in each frame based on intra-frame pixels and inter-frame pixels;

[0017] S15. Combine the pixels, occupancy map and pixel residual information corresponding to each CU in each frame to obtain the preprocessed dynamic point cloud sequence.

[0018] Intra-frame coding and inter-frame coding of a video frame sequence of point clouds include:

[0019] For each CU in each frame of the video frame sequence of point cloud, the content of the CU is predicted by using the neighboring pixels of the CU within the frame, and the intra-frame pixels of each CU in each frame are obtained; the motion vectors of the CU and the CUs at the same position in the adjacent frames are calculated, and the content of the CU is predicted based on the motion vectors, and the inter-frame pixels of each CU in each frame are obtained.

[0020] The pixel residual information of the CU in each frame is the difference between intra-frame pixels and inter-frame pixels.

[0021] The multi-level prediction partitioning network includes a CNN network and multiple layers of LSTM networks. The process by which the multi-level prediction partitioning network processes each CU of each frame of the preprocessed dynamic point cloud sequence sequentially includes:

[0022] S21. The coding unit CU of frame t t,n Inputting into a CNN network yields the encoding unit CU. t,n The fusion feature; where n is the index of the coding unit;

[0023] S22, The coding unit CU t,n The fused features are input into the first-layer LSTM network to obtain the coding unit CU. t,n First-level prediction occupancy result y t,n Based on the first-level predicted occupancy result y t,n For the coding unit CU t,n The process is divided to obtain the first-level sub-coding units. and its fusion characteristics;

[0024] S23, convert the first-level sub-coding unit The fused features are input into the second-layer LSTM network to obtain each sub-coding unit. Second-level prediction occupancy results Based on the second-level predicted occupancy results Pair coding unit The process is divided to obtain the second-level sub-coding units. and its fusion characteristics;

[0025] S24. The sub-coding unit of the previous level. The fused features are input into the LSTM network of the current layer to obtain each sub-coding unit. Current level predicted occupancy results Based on the current level, the predicted occupancy result Pair coding unit Divide the code to obtain the sub-coding units of the current level. Among them, the set of indices of sub-coding units il This is the index of the sub-coding unit at level l;

[0026] S25. Repeat step S24 until the coding unit CU for frame t is completed. t,n The processing.

[0027] Each level of the LSTM network consists of: LSTM units and fully connected layers; the current level of the LSTM network corresponds to the sub-coding units of the previous level. The fusion features are processed including:

[0028] The sub-coding unit of the previous level The fused features are input into the LSTM unit to obtain the sub-coding unit. Sequence characteristics;

[0029] Obtain the QP value and the corresponding sub-coding unit of level l-1 in the previous frame t-1. Level l prediction occupancy results The sub-coding unit of the previous level Fusion features and predicted occupancy results The QP value is concatenated to obtain the sub-coding unit of the next higher level. The splicing characteristics; where QP is the quantization parameter;

[0030] The sub-coding unit of the previous level The concatenated features are input into the fully connected layer to obtain the sub-coding units of the next level. The predicted occupancy result for the current level l.

[0031] Based on the current level, the predicted occupancy result Pair coding unit The partitioning process includes: predicting the occupancy result based on the current level. Obtain the dividing mark If the dividing mark If the value is 0, then the sub-coding unit ends. Otherwise, process the sub-coding unit. Divide the code to obtain the sub-coding units of the current level.

[0032] The loss function LCE is:

[0033]

[0034] Where N represents the total number of CUs in each frame, T represents the number of frames, and L represents the number of layers. This represents the sub-coding unit at level l of the nth coding unit (CU) of frame t. express Predict the probability that it is the true label k.

[0035] Weight of category k Where C represents the number of categories. Let be the prediction error rate for category k after smoothing.

[0036] Prediction error rate of smoothed category k Among them, e k Let β be the prediction error rate for class k in the historical training batches, and e be the smoothing coefficient. k,current This represents the prediction error rate for class k in the current training batch.

[0037] Beneficial effects:

[0038] 1. This invention utilizes LSTM networks at various levels to predict the occupancy results of each CU at each level. Based on the occupancy results, it determines whether the CU should be partitioned. For unoccupied CUs, the partitioning is terminated early. Simultaneously, the LSTM networks at various levels can perform different level partitioning predictions for different CUs in parallel, saving a significant amount of CU partitioning time and thus improving overall coding efficiency. 2. Intra-frame pixels can provide occupancy information of adjacent CUs of the current CU, while inter-frame pixels can provide occupancy information of CUs at the same position in adjacent frames. Therefore, this invention obtains the pixel residual information of the CUs in the corresponding frame by calculating the difference between intra-frame pixels and inter-frame pixels, constructing richer contextual information for CUs to learn prediction and improve prediction accuracy. 3. Adjacent frames are highly correlated, and the difference in occupancy information is not significant. Therefore, this invention uses the predicted occupancy information of the CUs in the previous frame to provide better information for the prediction of the current CU, improving prediction accuracy and accelerating CU prediction speed. 4. This invention assigns weights to different categories in the loss function based on the prediction error rate, enhancing the model's learning ability for minority categories and hard-to-classify samples, allowing the model to optimize various types of samples more evenly. Attached Figure Description

[0039] Figure 1 This is a schematic diagram of the HEVC partitioning process provided in an embodiment of the present invention;

[0040] Figure 2 A schematic diagram of a multi-layered LSTM network architecture provided in an embodiment of the present invention;

[0041] Figure 3 This is a flowchart of a fast encoding method for dynamic point clouds based on LSTM network prediction, provided for an embodiment of the present invention. Detailed Implementation

[0042] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0043] like Figure 2 , Figure 3 As shown, the present invention employs a dynamic point cloud fast encoding method based on LSTM network prediction, comprising: acquiring the current point cloud and preprocessing it; inputting the preprocessed current point cloud into a trained multi-level prediction partitioning network to obtain the sub-encoding units after CU partitioning of the current point cloud; and encoding the partitioned sub-encoding units into a bit stream using the entropy encoding module of HEVC.

[0044] The training process of a multi-level prediction partitioning network includes:

[0045] S1. Obtain the dynamic point cloud sequence and preprocess the dynamic point cloud sequence to obtain the preprocessed dynamic point cloud sequence. Each frame in the preprocessed dynamic point cloud sequence includes multiple coding units (CUs). Each CU includes pixels, occupancy map and pixel residual information.

[0046] The dynamic point cloud sequence is projected onto a two-dimensional plane using the TMC2 encoder to obtain a video frame sequence of the point cloud; each frame in the video frame sequence of the point cloud includes multiple coding units (CUs).

[0047] The TMC2 encoder functions getYUV() and getOccupy() are used to process the video frame sequence of the point cloud to obtain the pixels and occupancy map of each CU in each frame.

[0048] The TMC2 encoder is used to perform intra-frame and inter-frame coding on the video frame sequence of the point cloud to obtain the intra-frame and inter-frame pixels of each CU in each frame.

[0049] Intra-frame coding and inter-frame coding are two basic compression techniques used to reduce spatial redundancy (i.e., repetitive information within the same frame) and temporal redundancy (i.e., similarity between adjacent frames) in video data. These two coding methods are used in various coding standards, such as H.264 / AVC and HEVC / H.265.

[0050] The main steps of intra-frame coding include: for each CU in each frame of the video frame sequence of point cloud, the content of the CU is predicted by using the neighboring pixels of the CU in the frame, so as to obtain the intra-frame pixels of each CU in each frame.

[0051] The main process of inter-frame coding is as follows: For each CU in each frame of the video frame sequence of point cloud, calculate the motion vector of the CU and the CU at the same position in the adjacent frame, predict the content of the CU based on the motion vector, and obtain the inter-frame pixels of each CU in each frame. This step is called motion estimation and compensation.

[0052] Calculate the difference between intra-frame pixels and inter-frame pixels to obtain the pixel residual information for each CU in each frame;

[0053] The pixels, occupancy map, and pixel residual information corresponding to each CU in each frame are combined to obtain the preprocessed dynamic point cloud sequence.

[0054] S2. Input the preprocessed dynamic point cloud sequence into a multi-level prediction partitioning network for partitioning, and obtain the sub-coding units after CU partitioning of each frame and the prediction occupancy result at each level.

[0055] Since the video frame sequence of dynamic point cloud projection is a temporal sequence, a traditional LSTM network framework is used. To further reduce the partitioning time of CUs, this network model is applied to L levels of CUs to predict the occupancy results of each level of different CUs in parallel. This prediction determines whether each level of CU should be partitioned, resulting in a partitioning flag on each CU. If the flag is 1, partitioning occurs; if it is 0, partitioning does not occur. Based on this process, the partitioning prediction of CUs at each level can be processed in parallel to save time and improve coding efficiency. The multi-level prediction partitioning network includes a CNN network and an L-level LSTM network. Each level of the LSTM network includes LSTM units and fully connected layers.

[0056] For each CU in each frame of the preprocessed dynamic point cloud sequence, the processing procedure of the multi-level prediction partitioning network includes:

[0057] S21. The encoding unit CU of frame t of the preprocessed dynamic point cloud sequence. t,n Inputting into a CNN network yields the encoding unit CU. t,n The fusion characteristics;

[0058] The current coding unit (CU) of the current frame in the preprocessed dynamic point cloud sequence of the CNN network. t,n Feature extraction includes: extracting the current coding unit (CU) of the current frame from the preprocessed dynamic point cloud sequence. t,n The pixel values, occupancy map, and pixel residual information are input into the CNN network to obtain the fused feature f. t,n f t,n Let be the fusion feature of the nth CU in the tth frame.

[0059] S22, The coding unit CU t,n The fused features are input into the first-layer LSTM network to obtain the coding unit CU. t,n First-level prediction occupancy result y t,n Based on the first-level predicted occupancy result y t,n Get the dividing flag t,n If the dividing flag t,n If the value is 0, no partitioning is performed; otherwise, the coding unit CU is partitioned. t,n The process is divided to obtain the first-level sub-coding units. Where i1 is the index of the sub-coding unit of the first level;

[0060] S23, convert the first-level sub-coding unit The fused features are input into the second-layer LSTM network to obtain each sub-coding unit. Second-level prediction occupancy results Based on the second-level predicted occupancy results Obtain the dividing mark If the dividing mark If it is 0, then no sub-encoding unit is specified. Divide into sub-coding units; otherwise, divide into sub-coding units. The process is divided to obtain the second-level sub-coding units. Where i2 is the index of the sub-coding unit at the second level;

[0061] S24. Separate the sub-coding units of the previous level. The fused features are input into the LSTM network of the current layer to obtain each sub-coding unit. The predicted occupancy result of the current level Based on the current level, the predicted occupancy result Obtain the dividing mark If the dividing mark If it is 0, then no sub-encoding unit is specified. Perform the division and end the sub-coding unit. Otherwise, process the sub-coding unit. Divide the code to obtain the sub-coding units of the current level. Among them, the set of indices of sub-coding units i l This is the index of the sub-coding unit at level l;

[0062] Each level of the LSTM network consists of: LSTM units and fully connected layers; the current level of the LSTM network corresponds to the sub-coding units of the previous level. The fusion features are processed including:

[0063] The sub-coding unit of the previous level The fused features are input into the LSTM unit to obtain the sub-coding unit. Sequence characteristics;

[0064] Obtain the QP value and the corresponding sub-coding unit of level l-1 in the previous frame t-1. Predicted occupancy results at level l That is, the coding unit of the previous frame is divided into Previous CU Partition (PCP), and the sub-coding units of the previous level are divided into... Fusion features and predicted occupancy results The QP value is concatenated to obtain the sub-coding unit of the next higher level. splicing characteristics;

[0065] The sub-coding unit of the previous level The concatenated features are input into the fully connected layer to obtain the sub-coding units of the next level. The predicted occupancy result of the current level l; where QP is the quantization parameter.

[0066] The quantization parameter QP value is a built-in parameter in the TMC2 encoder, typically 42 or 27, and can be directly called during encoding.

[0067] S25. Repeat step S24 until the coding unit CU for frame t is completed. t,n The processing.

[0068] In steps S22, S23, and S24, the LSTM network of the current layer processes the sub-coding units of the previous layer. While processing is underway, the remaining LSTM network layers can process the coding units of the layer above them in parallel. The fusion features are predicted, and the occupancy is divided according to the predicted occupancy results. That is, the LSTM prediction network at each level can perform its prediction on the level above it in parallel. Prediction and segmentation significantly reduce processing time.

[0069] Generally, the number of levels L is 3, and the encoding unit CU t,n If the size is 64×64, then the sub-encoding unit The size is 32×32, and the sub-encoding unit is... The size is 16×16, and the sub-coding unit The dimensions are 8×8.

[0070] S3. Calculate the loss function value based on the prediction occupancy result of each CU in each frame at each level, update the model parameters based on the loss function value, and obtain the trained multi-level prediction partitioning network when the loss function value is minimized.

[0071] Since the purpose of this invention is to predict the occupancy of CU coding units, with labels of 0 or 1, we consider using the Cross-Entropy Loss (LCE), a common objective function in binary classification tasks, to measure the difference between the model's predicted distribution and the true distribution. The calculation formula is as follows:

[0072]

[0073] Where N represents the total number of CUs in each frame, T represents the number of frames, and L represents the number of layers. express The predicted occupancy result, i.e. Predict the probability that it is the true label k.

[0074] However, the sensitivity of the original cross-entropy to outliers and imbalanced data may cause the model to favor the class with a larger number of samples, thus affecting model performance. Therefore, dynamic weights w are introduced. k We use weighted cross-entropy to balance the impact of different classes on the loss, therefore the improved loss function is:

[0075]

[0076] Among them, w k The weights are for category k. Compared to the original cross-entropy, the weighted cross-entropy enhances the model's ability to learn minority classes and hard-to-classify samples by assigning weights to different categories, allowing the model to optimize samples of all categories more evenly.

[0077] Weights based on class error rate Where C is the number of categories, e k e is the prediction error rate of class k in the historical training batches. k,current Let be the prediction error rate for class k in the current training batch. The prediction error rate is the ratio of the number of prediction errors in the occupancy results of CUs labeled class k to the number of CUs labeled class k. β is the smoothing coefficient. Let be the prediction error rate for category k after smoothing.

[0078] Through the construction of the above network model, the coding time saving effect after predicting CU partitioning in this invention is considerable. As shown in Table 1, compared with the standard encoder, this invention can save more than 60% of the time in CU partitioning. In the D1 (point-to-point quality) and D2 (point-to-area quality) data, it can be seen that in geometric information compression coding, the BD total rate increases by a maximum of only 0.2%, while in luminance, the maximum increase is only 0.66%. However, the coding time is saved by more than 60% in various dataset sequences.

[0079] Table 1. Results of sequence encoding for various datasets

[0080]

[0081]

[0082] Among them, basketball, dancer, queen, loot, redandblack, soldier, and longdress are standard datasets in the V-PCC encoder, containing point sets of various objects. Geom.BD-TotGeomRate is the total geometric bandwidth, End-to-End BD-AttrRate is the total attribute bandwidth, Geom.BD-TotalRate is the total geometric BD rate, End-to-End BD-TotalRate is the total attribute BD rate, Enc Self is the time-saving parameter, D1 is the point-to-point quality, D2 is the point-to-area quality, Luma is the luminance, Chroma Cb is the blue color difference component, and Chroma Cr is the red color difference component.

[0083] The above-described embodiments further illustrate the purpose, technical solution, and advantages of the present invention. It should be understood that the above-described embodiments are merely preferred embodiments of the present invention and are not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made to the present invention within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A fast encoding method for dynamic point clouds based on LSTM network prediction, characterized in that, include: The current point cloud is acquired and preprocessed. The preprocessed point cloud is then input into a trained multi-level prediction partitioning network to obtain the sub-coding units after the current point cloud's coding unit (CU) is partitioned. The partitioned sub-coding units are then encoded into a bit stream. The training process of a multi-level prediction partitioning network includes: S1. Obtain the dynamic point cloud sequence and preprocess the dynamic point cloud sequence to obtain the preprocessed dynamic point cloud sequence. Each frame in the preprocessed dynamic point cloud sequence includes multiple coding units (CUs). Each CU includes pixels, occupancy map and pixel residual information. S2. Input each CU of each frame of the preprocessed dynamic point cloud sequence into the multi-level prediction partitioning network for partitioning to obtain the sub-coding units after partitioning each CU of each frame and the prediction occupancy result at each level. The multi-level prediction partitioning network includes a CNN network and multiple layers of LSTM networks. The process by which the multi-level prediction partitioning network processes each CU of each frame of the preprocessed dynamic point cloud sequence sequentially includes: S21. The coding unit of frame t Inputting into a CNN network yields encoding units. The fusion feature; where n is the index of the coding unit; S22, encoding unit The fused features are input into the first-layer LSTM network to obtain the coding unit. First-level prediction occupancy results Based on the first-level predicted occupancy results For coding unit The process is divided to obtain the first-level sub-coding units. and its fusion characteristics; S23, convert the first-level sub-coding unit The fused features are input into the second-layer LSTM network to obtain each sub-coding unit. Second-level prediction occupancy results Based on the second-level predicted occupancy results Pair coding unit The process is divided to obtain the second-level sub-coding units. and its fusion characteristics; S24. The sub-coding unit of the previous level. The fused features are input into the LSTM network of the current layer to obtain each sub-coding unit. Current level predicted occupancy results Predict occupancy results based on the current level Pair coding unit Divide the code to obtain the sub-coding units of the current level. ; where, is the set of indices of sub-coding units. , This is the index of the sub-coding unit at level l; S25. Repeat step S24 until the coding unit for frame t is finished. The processing; Based on the current level, the predicted occupancy result Pair coding unit The partitioning process includes: predicting the occupancy result based on the current level. Obtain the dividing mark If the dividing marker If the value is 0, then the sub-coding unit ends. Otherwise, process the sub-coding unit. Divide the code to obtain the sub-coding units of the current level. ; S3. Calculate the loss function value based on the prediction occupancy result of each CU in each frame at each level, update the model parameters based on the loss function value, and obtain the trained multi-level prediction partitioning network when the loss function value is minimized.

2. The method for fast encoding of dynamic point clouds based on LSTM network prediction according to claim 1, characterized in that, Preprocessing of dynamic point cloud sequences includes: S11. Project the dynamic point cloud sequence onto a two-dimensional plane to obtain a video frame sequence of the point cloud; each frame in the video frame sequence of the point cloud includes multiple coding units (CUs). S12. Process the video frame sequence to the point cloud to obtain the pixels and occupancy map of each CU in each frame; S13. Perform intra-frame coding and inter-frame coding on the video frame sequence to the point cloud to obtain the intra-frame pixels and inter-frame pixels of each CU in each frame. S14. Calculate the pixel residual information of each CU in each frame based on intra-frame pixels and inter-frame pixels; S15. Combine the pixels, occupancy map and pixel residual information corresponding to each CU in each frame to obtain the preprocessed dynamic point cloud sequence.

3. The method for fast encoding of dynamic point clouds based on LSTM network prediction according to claim 2, characterized in that, Intra-frame coding and inter-frame coding of a video frame sequence of point clouds include: For each CU in each frame of the video frame sequence of point cloud, the content of the CU is predicted by using the neighboring pixels of the CU within the frame, and the intra-frame pixels of each CU in each frame are obtained; the motion vectors of the CU and the CUs at the same position in the adjacent frames are calculated, and the content of the CU is predicted based on the motion vectors, and the inter-frame pixels of each CU in each frame are obtained.

4. The method for fast encoding of dynamic point clouds based on LSTM network prediction according to claim 2, characterized in that, The pixel residual information of the CU in each frame is the difference between intra-frame pixels and inter-frame pixels.

5. The method for fast encoding of dynamic point clouds based on LSTM network prediction according to claim 1, characterized in that, Each level of the LSTM network consists of: LSTM units and fully connected layers; the current level of the LSTM network corresponds to the sub-coding units of the previous level. The processing includes: The sub-coding unit of the previous level Inputting into an LSTM unit yields a sub-coding unit. ; Obtain the QP value and the corresponding sub-coding unit of level l-1 in the previous frame t-1. Level l prediction occupancy results The sub-coding unit of the previous level Compared with the predicted occupancy results The QP value is concatenated to obtain the sub-coding unit of the next higher level. The splicing characteristics; where QP is the quantization parameter; The sub-coding unit of the previous level The concatenated features are input into the fully connected layer to obtain the sub-coding units of the next level. The predicted occupancy result for the current level l.

6. The method for fast encoding of dynamic point clouds based on LSTM network prediction according to claim 1, characterized in that, loss function for: Where N represents the total number of CUs in each frame, T represents the number of frames, and L represents the number of layers. express Predict the probability that it is the true label k; Weight of category k Where C represents the number of categories. The prediction error rate for category k after smoothing; Prediction error rate of smoothed category k ;in, The prediction error rate for class k in the historical training batches. For smoothing coefficients, This represents the prediction error rate for class k in the current training batch.

Images