A self-supervised point cloud completion method and device based on scene flow prior guidance

Abstract

The present invention relates to the field of computer vision technology, and in particular to a method, device and computer-readable storage medium for self-supervised point cloud completion based on scene flow prior guidance. The method comprises: obtaining a point cloud sequence containing residual point clouds collected by a radar, and using the point cloud sequences of the previous frame and the next frame to obtain the forward scene flow and the reverse scene flow of the current frame as potential supplementary point clouds of the current frame; removing points repeated with the point cloud sequence of the current frame in the potential supplementary point cloud of the current frame, and combining them with the point cloud sequence of the current frame; dividing the combined point cloud sequence into blocks to obtain a point cloud proxy, and generating visible marks, mask marks and position embeddings; and generating a reconstructed point cloud using a transformer model. The present invention effectively utilizes the point cloud information of the previous and next frames to fill in the missing point cloud of the current frame, ensuring the stability of point cloud completion in the later stage, and has strong generalization ability.

Description

Technical Field

[0001] The present invention relates to the field of computer vision technology, and in particular to a method, device and computer-readable storage medium for self-supervised point cloud completion based on scene flow prior guidance. Background Art

[0002] With the development of sensor technology, the emergence of 3D sensor devices such as solid-state laser radar and RGB-D cameras has made it easier to obtain 3D point clouds, which has greatly promoted the development of 3D computer vision. As an important way to express 3D information, point clouds have the advantages of small storage space and simple and clear data structure. However, in the process of 3D sensor acquisition equipment working, the equipment is easily interfered by multiple factors, such as occlusion between imaging objects, signal transmission interference between devices, and point cloud blur caused by the movement of the device itself. These factors cause the point clouds in practical applications to be sparse and incomplete, which greatly limits the performance of downstream tasks such as detection, segmentation, and tracking. Therefore, it is necessary to complete and enhance the data of the collected original point cloud in practical applications.

[0003] In recent years, research in the field of point cloud completion has made great progress. Some early works completed point clouds through voxelization and 3D convolution. PointCNN achieves permutation invariance through X-conv transformation, so that the classic convolution operation can be performed on the features of the point cloud after X transformation. GRNet introduces 3D grids as intermediate representations to normalize unordered point clouds, thereby explicitly preserving the structure and background of the point cloud. PointConv extends the dynamic filter to a new convolution operation, namely PointConv, which can build a deep convolutional network for point clouds. With the emergence of PointNet and PointNet++, direct processing of point clouds has become the mainstream method in the field of 3D analysis. By learning the structural information between point cloud blocks and the global structure, such models can accurately complete missing point clouds.

[0004] Scene flow estimation is a typical emerging research idea for analyzing point cloud sequences. For a given point cloud pair, how to accurately restore the motion relationship between the two frames is a task of concern to scene flow estimation. FlowNet3D++ incorporates geometric constraints based on the distance from the point to the plane and the angle alignment between the vectors in the flow field into FlowNet3D, which significantly improves the accuracy performance of the network. With the further development of sensor technology, dense point clouds have become the mainstream in the industry, increasing the difficulty of obtaining labeled data sets. Some point cloud scene flow estimation works revolve around unsupervised or self-supervised methods. SLIM introduces self-supervised training into point cloud scene flow estimation for the first time. It divides points into two categories, "moving" and "stationary", based on self-supervised training, and proposes a new scene flow estimation architecture on this basis, which enables it to handle an order of magnitude more points than previous training.

[0005] However, the existing point cloud reconstruction methods based on self-supervision require the model to learn a large number of manually annotated samples during training and cannot be flexibly transferred between different scenarios. Also, due to the discontinuity of the data set, the model can only learn one sample at a time. When applied in real scenarios, it does not fully utilize the previous and next frame information of the sequence data, resulting in poor point cloud reconstruction results. Summary of the invention

[0006] To this end, the technical problem to be solved by the present invention is to overcome the problem that the point cloud reconstruction method in the prior art does not fully utilize the previous and next frame information of the sequence data.

[0007] In order to solve the above technical problems, the present invention provides a self-supervised point cloud completion method based on scene flow prior guidance, comprising:

[0008] S1. Obtain the point cloud sequence containing residual point clouds collected by the radar. For each frame of the point cloud sequence P t , using the previous frame point cloud sequence P t-1 Estimation of P t-1 To P t The forward scene flow uses the next frame point cloud sequence P t+1 Estimation of P t+1 To P t The reverse scene flow combines the forward scene flow and the reverse scene flow into a point cloud sequence P SF , as P t The potential supplementary point cloud of t-1 , P t and P t+1 They are the point cloud sequences of the t-1th frame, the tth frame, and the t+1th frame respectively;

[0009] S2, point cloud sequence P SF Preprocessing is performed to obtain the point cloud sequence P tThe average point cloud distance between the point clouds is used as the threshold. For the point cloud sequence P t For each point cloud in, delete the point cloud sequence P SF After the point cloud whose distance to the point cloud is less than the threshold is found, the point cloud sequence P SF The remaining point clouds and point cloud sequence P t The point cloud combination in is used as the preprocessed point cloud sequence P SF ;

[0010] S3. Obtain the preprocessed point cloud sequence P SF As the point cloud proxy center, each point cloud proxy center is combined with its surrounding adjacent point clouds to form each point cloud proxy; according to the point cloud proxy, a global position embedding, a visible mark and its corresponding position embedding and a mask mark are generated;

[0011] S4. Input the global position embedding, visible markers and their corresponding position embedding and mask markers into a standard transformer model, and output the point cloud sequence reconstructed at the tth frame.

[0012] In one embodiment of the present invention, in S1, the implicit regularizer based on the neural network uses the point cloud sequence P of the previous frame t-1 Estimation of P t-1 To P t The forward scene flow of the neural network based implicit regularizer uses the point cloud sequence P of the next frame t+1 Estimation of P t+1 To P t Reverse scene flow.

[0013] In one embodiment of the present invention, a scene flow is generated based on an implicit regularizer of a neural network, comprising:

[0014] After weight initialization of the fully connected neural network, the real point cloud sequence is input into the fully connected neural network;

[0015] The chamfer distance is used as the objective function of the fully connected neural network, and the Adam optimizer is used to iteratively optimize the objective function, and the point cloud with the smallest chamfer distance with the point cloud in the real point cloud sequence is output as the scene flow;

[0016] Among them, the formula of the objective function of the fully connected neural network is:

[0017]

[0018] Among them, S1 is the point cloud set of the scene flow, x is any point cloud in S1, |S1| represents the number of point clouds in S1, S2 is the point cloud set of the real point cloud sequence, y is any point cloud in S2, |S2| represents the number of point clouds in S2, and CD is the chamfer distance between S1 and S2.

[0019] In one embodiment of the present invention, the number of layers of the fully connected neural network is 8, the number of hidden units is 128, and the activation function is Relu.

[0020] In one embodiment of the present invention, in S3, the farthest point sampling method is used to obtain the preprocessed point cloud sequence P SF Several down-sampling points are used as point cloud proxy centers, and the K nearest neighbor algorithm is used to obtain the k nearest neighbor point clouds near each point cloud proxy center. Each point cloud proxy center is combined with its corresponding k nearest neighbor point clouds to form each point cloud proxy.

[0021] In one embodiment of the present invention, in S3, generating visible marks, mask marks, and position embeddings according to the point cloud proxy includes:

[0022] Input all point cloud proxy centers into the fully connected network to generate the position embeddings of all point cloud proxy centers as the global position embeddings;

[0023] Determine whether each point cloud proxy center is a point cloud in the point cloud sequence P_t. If so, input all point clouds in the point cloud proxy corresponding to the point cloud proxy center into mini-pointnet for feature extraction, output features as visible marks, and embed the position corresponding to the point cloud proxy center as the position embedding of the visible mark;

[0024] A vector following a normal distribution is randomly initialized and its dimension is expanded to be the same as the visible mark as the mask mark.

[0025] In one embodiment of the present invention, in S4, the visible mark, the mask mark and the position are embedded into the standard transformer model, and the point cloud sequence reconstructed after the t-th frame is output, including:

[0026] The standard transformer model uses a symmetric encoder-decoder architecture to embed visible tokens and their corresponding positions into the input encoder and output the encoded visible tokens;

[0027] The encoded visible markers are concatenated with the mask markers respectively, and input into the decoder together with the global position embedding, and the reconstructed point cloud sequence of the tth frame is output.

[0028] In one embodiment of the present invention, when training the standard transformer model, the L2 chamfer distance is used as the loss function of the standard transformer model, and the formula is:

[0029]

[0030] Where P pre is the reconstructed point cloud set, a is P pre Any point cloud in |P pre | indicates P pre The number of point clouds in P gt is the input point cloud set, b is P gt Any point cloud in |P gt | indicates P gt The number of point clouds in .

[0031] The present invention also provides a self-supervised point cloud completion device based on scene flow prior guidance, comprising:

[0032] The scene flow estimation module is used to obtain the point cloud sequence containing residual point clouds collected by the radar. For each frame of the point cloud sequence P t , using the previous frame point cloud sequence P t-1 Estimation of P t-1 To P t The forward scene flow uses the next frame point cloud sequence P t+1 Estimation of P t+1 To P t The reverse scene flow combines the forward scene flow and the reverse scene flow into a point cloud sequence P SF , as P t The potential supplementary point cloud of t-1 , P t and P t+1 They are the point cloud sequences of the t-1th frame, the tth frame, and the t+1th frame respectively;

[0033] Preprocessing module, used to process the point cloud sequence P SF Preprocessing is performed to obtain the point cloud sequence P t The average point cloud distance between the point clouds is used as the threshold. For the point cloud sequence P t For each point cloud in, delete the point cloud sequence P SF After the point cloud whose distance to the point cloud is less than the threshold is found, the point cloud sequence P SF The remaining point clouds and point cloud sequence P t The point cloud combination in is used as the preprocessed point cloud sequence P SF ;

[0034] Point cloud embedding module, used to obtain the preprocessed point cloud sequence P SFAs the point cloud proxy center, each point cloud proxy center is combined with its surrounding adjacent point clouds to form each point cloud proxy; according to the point cloud proxy, a global position embedding, a visible mark and its corresponding position embedding and a mask mark are generated;

[0035] The point cloud reconstruction module is used to input the global position embedding, visible markers and their corresponding position embedding and mask markers into a standard transformer model, and output a point cloud sequence after reconstructing the t-th frame.

[0036] The present invention also provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the above-mentioned self-supervised point cloud completion method based on scene flow prior guidance are implemented.

[0037] The above technical solution of the present invention has the following advantages compared with the prior art:

[0038] The self-supervised point cloud completion method based on scene flow prior guidance described in the present invention generates potential supplementary point clouds of the current frame by scene flow estimation, using the information of the point clouds of the previous and subsequent frames as prior guidance. Through preprocessing, the point clouds that are repeated with the real point clouds are deleted from the potential supplementary point clouds, thereby improving the computational efficiency. The global position embedding, visible marks and their corresponding position embedding and mask marks of the current frame are generated according to the preprocessed point cloud sequence, and the point cloud of the current frame is reconstructed. The present invention effectively utilizes the point cloud information of the previous and subsequent frames when completing the missing point cloud, thereby ensuring the stability of the point cloud completion.

[0039] The present invention utilizes a self-supervised model and adopts an advanced transformer model as a reconstruction model. No manual labels are required during training, and the reconstruction model can be flexibly migrated between different application scenarios. Compared with existing point cloud completion methods, it has stronger generalization capabilities. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] In order to make the content of the present invention more clearly understood, the present invention is further described in detail below according to specific embodiments of the present invention in conjunction with the accompanying drawings, wherein

[0041] Figure 1 It is a flow chart of a self-supervised point cloud completion method based on scene flow prior guidance of the present invention;

[0042] Figure 2 This is a visualization effect diagram of point cloud sequence data collected by a part of radar provided in an embodiment of the present invention;

[0043] Figure 3 is a visualization effect diagram of the forward scene flow and the reverse scene flow provided by the embodiment of the present invention; wherein Figure 3a in the figure is a visualization diagram of the forward scene flow. Figure 3 b in the figure is a visualization diagram of the reverse scene flow;

[0044] Figure 4 is a comparison diagram of the visualization effect of the original point cloud, the partially occluded point cloud and the point cloud completed by the method of the present invention provided by the embodiment of the present invention; wherein Figure 4 a in the figure is the visualization effect diagram of the original point cloud. Figure 4 b in the figure is a visualization of the partially occluded point cloud. Figure 4 c in the figure is a visualization effect diagram of completing the point cloud using the method of the present invention;

[0045] Figure 5 It is a workflow diagram of a self-supervised point cloud completion device based on scene flow prior guidance of the present invention. DETAILED DESCRIPTION

[0046] The present invention is further described below in conjunction with the accompanying drawings and specific embodiments so that those skilled in the art can better understand the present invention and implement it, but the embodiments are not intended to limit the present invention.

[0047] Embodiment 1

[0048] Reference Figure 1 As shown, the present invention provides a self-supervised point cloud completion method based on scene flow prior guidance, comprising:

[0049] S1. Obtain the point cloud sequence containing residual point clouds collected by the radar. For the visualization effect of the point cloud sequence data collected by some radars, refer to Figure 2 As shown. For each frame point cloud sequence P t , using the previous frame point cloud sequence P t-1 Estimation of P t-1 To P t The forward scene flow uses the next frame point cloud sequence P t+1 Estimation of P t+1 To P t The reverse scene flow. t-1 , P t and P t+1 They are the point cloud sequences of the t-1th frame, the tth frame and the t+1th frame respectively.

[0050] In this embodiment, the scene flow estimation model used to estimate the forward scene flow and the reverse scene flow adopts a runtime optimization strategy, and estimates the scene flow based on an implicit regularizer of a neural network. The specific steps include:

[0051] Construct a fully connected neural network with 8 layers, 128 hidden units, and Relu activation function, and initialize the weights of each layer of the fully connected network;

[0052] Inputting a real point cloud sequence into the fully connected neural network;

[0053] The chamfer distance is used as the objective function of the fully connected neural network:

[0054]

[0055] Wherein, S1 is the point cloud set of the scene flow, x is any point cloud in S1, |S1| represents the number of point clouds in S1, S2 is the point cloud set of the real point cloud sequence, y is any point cloud in S2, |S2| represents the number of point clouds in S2, and CD is the chamfer distance between S1 and S2. The objective function is used to measure the distance between the point cloud in the scene flow and the real point cloud.

[0056] In this embodiment, the Adam optimizer is used to iteratively optimize the objective function. The learning rate during optimization is set to 8e-4, and the number of optimization rounds is set to 1000. The output is the same as the point cloud sequence P. t-1 The point cloud with the smallest chamfer distance in the midpoint cloud is taken as the scene flow.

[0057] The forward scene flow and reverse scene flow obtained are as follows Figure 3 As shown, Figure 3 a in the figure is a visualization diagram of the forward scene flow. Figure 3 The b in FIG is a visualization diagram of the reverse scene flow.

[0058] Combine the forward scene flow and the reverse scene flow into a point cloud sequence P SF , as P t Potential supplementary point cloud.

[0059] S2, point cloud sequence P SF Preprocessing is performed to obtain the point cloud sequence P t The average point cloud distance between the point clouds is used as the threshold. For the point cloud sequence P t For each point cloud in, delete the point cloud sequence P SF After the point cloud whose distance to the point cloud is less than the threshold is found, the point cloud sequence P SF The remaining point clouds and point cloud sequence P t The point cloud combination in is used as the preprocessed point cloud sequence P SF Through preprocessing, the potential supplementary point clouds are deleted from the point cloud sequence P t The repeated point clouds in the point cloud reduce the amount of calculation and improve the calculation efficiency.

[0060] S3, using the farthest point sampling method to obtain the preprocessed point cloud sequence P SFSeveral down-sampled points are used as the point cloud proxy center. Then the K nearest neighbor algorithm is used to obtain the k nearest neighbor point clouds near each point cloud proxy center, and each point cloud proxy center is combined with its corresponding k nearest neighbor point clouds to form each point cloud proxy.

[0061] Generate global positional embeddings, visible tokens and their corresponding positional embeddings and mask tokens according to the point cloud proxy, the specific steps include:

[0062] Input all point cloud proxy centers into the fully connected network to generate the position embeddings of all point cloud proxy centers as the global position embeddings;

[0063] Determine whether each point cloud proxy center is a point cloud sequence P t If so, all point clouds in the point cloud proxy corresponding to the point cloud proxy center are input into mini-pointnet for feature extraction, the output features are used as visible marks, and the position embedding corresponding to the point cloud proxy center is used as the position embedding of the visible mark;

[0064] A vector following a normal distribution is randomly initialized and its dimension is expanded to be the same as the visible mark as the mask mark.

[0065] S4. Input the global position embedding, the visible mark and its corresponding position embedding, and the mask mark into the standard transformer model. The standard transformer model adopts a symmetric encoder-decoder architecture, inputs the visible mark and its corresponding position embedding into the encoder, and outputs the encoded visible mark; concatenates the encoded visible mark with the mask mark respectively, inputs the global position embedding into the decoder together, and outputs the point cloud sequence reconstructed at the tth frame.

[0066] In the transformer model, each transformer block has 384 layers and 6 attention heads, and the encoder and decoder have 12 and 4 transformer blocks respectively.

[0067] When training the transformer model, a public data set such as Shapenet is used. In this embodiment, the data set used to train the transformer model is a self-owned pedestrian point cloud data set. During the training process, the obtained point cloud proxy is first subjected to a random mask operation, and then used to generate visible marks, mask marks, and position embeddings. The L2 chamfer distance is used as the loss function of the standard transformer model, and the AdamW optimizer is used to optimize the loss function to minimize and converge. The formula of the loss function is:

[0068]

[0069] Where P pre is the reconstructed point cloud set, a is P pre Any point cloud in |P pre | indicates P pre The number of point clouds in P gt is the input point cloud set, b is P gt Any point cloud in |P gt | indicates P gt The number of point clouds in .

[0070] Figure 4 The visualization effects of the original point cloud, the partially occluded point cloud, and the point cloud completed by the method of the present invention are shown. Figure 4 a in the figure is the visualization effect diagram of the original point cloud. Figure 4 b in the figure is a visualization of the partially occluded point cloud. Figure 4 c in the figure is a visualization effect diagram of completing the point cloud using the method of the present invention.

[0071] In order to evaluate the effectiveness of the present invention, Fidelity Distance and Consistency are used as evaluation indicators of point cloud completion effect. The results are shown in Table 1.

[0072] Table 1. Point cloud completion accuracy evaluation table of the present invention

[0073] FidelityDistance Consistency Own dataset 0.01765 0.01097

[0074] Fidelity Distance represents the chamfer distance between the input point cloud and the output point cloud, which is used to measure the model's ability to maintain the input. Consistency represents the chamfer distance between different outputs of the same object, which is used to measure the consistency of the model output when the input data changes. From the results in Table 1, it can be seen that the method proposed in the present invention can effectively maintain the characteristics of the input point cloud, and the output remains consistent when the input data changes.

[0075] In summary, the present invention can effectively utilize the information of the previous and next frame point clouds to fill in the missing point clouds, and adopts the transformer model, which has strong generalization ability in different application scenarios.

[0076] Embodiment 2

[0077] Reference Figure 5 As shown, this embodiment provides a self-supervised point cloud completion device based on scene flow prior guidance, including a scene flow estimation module, a preprocessing module, a point cloud embedding module and a point cloud reconstruction module.

[0078] The scene flow estimation module is used to obtain a point cloud sequence containing residual point clouds collected by the radar. For each frame of the point cloud sequence P t , using the previous frame point cloud sequence P t-1 Estimation of P t-1 To P t The forward scene flow uses the next frame point cloud sequence P t+1 Estimation of P t+1 To P t The reverse scene flow combines the forward scene flow and the reverse scene flow into a point cloud sequence P SF , as P t Potential supplementary point cloud.

[0079] The preprocessing module is used to process the point cloud sequence P SF Preprocessing is performed to obtain the point cloud sequence P t The average point cloud distance between the point clouds is used as the threshold. For the point cloud sequence P t For each point cloud in the , delete the point cloud sequence P in its neighborhood whose distance is less than the threshold SF The point cloud in the SF Point cloud and point cloud sequence P in t The point cloud combination in is used as the preprocessed point cloud sequence P SF .

[0080] The point cloud embedding module is used to obtain the preprocessed point cloud sequence P SF As the point cloud proxy center, each point cloud proxy center is combined with its surrounding point clouds to form a point cloud proxy; a visible mark, a mask mark and a position embedding are generated according to the point cloud proxy

[0081] The point cloud reconstruction module is used to embed the visible markers, mask markers and positions into the input standard transformer model and output the point cloud sequence after the t-th frame reconstruction.

[0082] Embodiment 3

[0083] This embodiment provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the above-mentioned self-supervised point cloud completion method based on scene flow prior guidance are implemented.

[0084] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented in one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that contain computer-usable program code.

[0085] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0086] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture including an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0087] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0088] Obviously, the above embodiments are merely examples for clear explanation and are not intended to limit the implementation methods. For those skilled in the art, other different forms of changes or modifications can be made based on the above description. It is not necessary and impossible to list all the implementation methods here. The obvious changes or modifications derived from these are still within the protection scope of the invention.

Claims

1. A self-supervised point cloud completion method based on scene flow prior guidance, characterized in that: include: S1. Obtain the point cloud sequence containing residual point clouds collected by the radar. For each frame of the point cloud sequence , an implicit regularizer based on a neural network, using its previous frame point cloud sequence estimate arrive The forward scene flow, the implicit regularizer based on the neural network, uses the point cloud sequence of the next frame estimate arrive The reverse scene flow combines the forward scene flow and the reverse scene flow into a point cloud sequence , as of potential supplementary point clouds; , and Respectively Frame, Frame and Point cloud sequence; Among them, the implicit regularizer based on neural network generates scene flow, including: After weight initialization of the fully connected neural network, the real point cloud sequence is input into the fully connected neural network; The chamfer distance is used as the objective function of the fully connected neural network, and the Adam optimizer is used to iteratively optimize the objective function, and the point cloud with the smallest chamfer distance with the point cloud in the real point cloud sequence is output as the scene flow; Among them, the formula of the objective function of the fully connected neural network is: ； in, is the point cloud set of the scene flow, for Any point cloud, express The number of point clouds in is the point cloud set of the real point cloud sequence, for Any point cloud, express The number of point clouds in for and Chamfer distance; S2. Point cloud sequence Preprocessing is performed to obtain point cloud sequences The average point cloud distance between the midpoint clouds is used as the threshold. For each point cloud in, delete the point cloud sequence After the point cloud whose distance to the point cloud is less than the threshold is selected, the point cloud sequence The remaining point clouds and point cloud sequences in The point cloud combination in is used as the preprocessed point cloud sequence ; S3. Obtain the preprocessed point cloud sequence As the point cloud proxy center, each point cloud proxy center is combined with its surrounding adjacent point clouds to form each point cloud proxy; according to the point cloud proxy, a global position embedding, a visible mark and its corresponding position embedding and a mask mark are generated; S4, input the global position embedding, visible mark and its corresponding position embedding and mask mark into the standard transformer model, and output the Point cloud sequence after frame reconstruction.

2. The method for self-supervised point cloud completion based on scene flow prior guidance according to claim 1, characterized in that: The number of layers of the fully connected neural network is 8, the number of hidden units is 128, and the activation function is Relu.

3. The method for self-supervised point cloud completion based on scene flow prior guidance according to claim 1, characterized in that: In S3, the farthest point sampling method is used to obtain the preprocessed point cloud sequence Several down-sampling points are used as point cloud proxy centers, and the K nearest neighbor algorithm is used to obtain the k nearest neighbor point clouds near each point cloud proxy center. Each point cloud proxy center is combined with its corresponding k nearest neighbor point clouds to form each point cloud proxy.

4. The method for self-supervised point cloud completion based on scene flow prior guidance according to claim 1, characterized in that: In S3, generating visible markers, mask markers, and position embeddings according to the point cloud proxy includes: Input all point cloud proxy centers into the fully connected network to generate the position embeddings of all point cloud proxy centers as the global position embeddings; Determine whether each point cloud proxy center is a point cloud sequence If so, all point clouds in the point cloud proxy corresponding to the point cloud proxy center are input into mini-pointnet for feature extraction, the output features are used as visible markers, and the position embedding corresponding to the point cloud proxy center is used as the position embedding of the visible marker; A vector following a normal distribution is randomly initialized and its dimension is expanded to be the same as the visible mark as the mask mark.

5. The method for self-supervised point cloud completion based on scene flow prior guidance according to claim 4, characterized in that: In S4, the visible mark, mask mark and position are embedded into the standard transformer model, and the output is The point cloud sequence after frame reconstruction includes: The standard transformer model uses a symmetric encoder-decoder architecture to embed visible tokens and their corresponding positions into the input encoder and output the encoded visible tokens; The encoded visible markers are concatenated with the mask markers and input into the decoder together with the global position embedding to output the first Point cloud sequence after frame reconstruction.

6. The method for self-supervised point cloud completion based on scene flow prior guidance according to claim 5, characterized in that: When training the standard transformer model, the L2 chamfer distance is used as the loss function of the standard transformer model, and the formula is: ； in To reconstruct a point cloud set, for Any point cloud, express The number of point clouds in is the input point cloud set, for Any point cloud, express The number of point clouds in .

7. A self-supervised point cloud completion device based on scene flow prior guidance, characterized in that: include: The scene flow estimation module is used to obtain the point cloud sequence containing residual point clouds collected by the radar. For each frame of the point cloud sequence , an implicit regularizer based on a neural network, using its previous frame point cloud sequence estimate arrive The forward scene flow, the implicit regularizer based on the neural network, uses the point cloud sequence of the next frame estimate arrive The reverse scene flow combines the forward scene flow and the reverse scene flow into a point cloud sequence , as of potential supplementary point clouds; , and Respectively Frame, Frame and Point cloud sequence; Among them, the implicit regularizer based on neural network generates scene flow, including: After weight initialization of the fully connected neural network, the real point cloud sequence is input into the fully connected neural network; The chamfer distance is used as the objective function of the fully connected neural network, and the Adam optimizer is used to iteratively optimize the objective function, and the point cloud with the smallest chamfer distance with the point cloud in the real point cloud sequence is output as the scene flow; Among them, the formula of the objective function of the fully connected neural network is: ； in, is the point cloud set of the scene flow, for Any point cloud, express The number of point clouds in is the point cloud set of the real point cloud sequence, for Any point cloud, express The number of point clouds in for and Chamfer distance; pre-processing module for point cloud sequence Preprocessing is performed to obtain point cloud sequences The average point cloud distance between the midpoint clouds is used as the threshold. For each point cloud in, delete the point cloud sequence After the point cloud whose distance to the point cloud is less than the threshold is selected, the point cloud sequence The remaining point clouds and point cloud sequences in The point cloud combination in is used as the preprocessed point cloud sequence ; Point cloud embedding module, used to obtain preprocessed point cloud sequence As the point cloud proxy center, each point cloud proxy center is combined with its surrounding adjacent point clouds to form each point cloud proxy; according to the point cloud proxy, a global position embedding, a visible mark and its corresponding position embedding and a mask mark are generated; The point cloud reconstruction module is used to input the global position embedding, visible markers and their corresponding position embeddings and mask markers into the standard transformer model and output the Point cloud sequence after frame reconstruction.

8. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by the processor, the steps of the self-supervised point cloud completion method based on scene flow prior guidance as described in any one of claims 1 to 6 are implemented.

Images