Point cloud restoration processing method, device and computer equipment

Abstract

The present application provides a point cloud restoration processing method, device and computer equipment, which belongs to the field of point cloud restoration technology. The method includes: obtaining training samples, and obtaining a complete target point cloud based on the sample splicing point cloud; inputting the sample original point cloud into the initial restoration model to obtain the corresponding bird's-eye view features, attribute information and first feature descriptor, and outputting the predicted target point cloud; obtaining a first loss value based on the attribute information and the true value of the target; obtaining a second loss value based on the first feature descriptor and the second feature descriptors under multiple other perspectives; obtaining a third loss value based on the predicted target point cloud and the complete target point cloud; iteratively correcting the initial restoration model based on the loss value to obtain a target restoration model; inputting the point cloud to be restored into the target restoration model to obtain a restored point cloud. It can achieve the effect of restoring the true state of the target point cloud under different perspectives, so as to reduce the difference in point cloud features when the sensor illuminates the same target under different perspectives.

Description

Technical Field

[0001] The present application relates to the field of point cloud restoration technology, and more specifically, to a point cloud restoration processing method, apparatus, and computer equipment. Background Art

[0002] Object detection, a key area of computer vision research, can accurately identify the category and location of specific objects in a given image. Due to its superior performance in computer vision and digital image processing, object detection is widely used in fields such as facial recognition, smart transportation, industrial inspection, security monitoring, and medical image analysis. In the field of smart transportation, object detection is often used in vehicles to monitor vehicle operation in real time, enabling object recognition and obstacle avoidance.

[0003] Currently, most target detection solutions use a series of sensors such as images, lidar, and millimeter waves to perform real-time target detection on the vehicle side. They often use an occupancy grid or bird's eye view (BEV) perspective to perform real-time target detection and segmentation around the vehicle body.

[0004] However, when performing target detection in roadside perception scenarios, due to the significant difference between the deployment scheme of roadside perception equipment and the vehicle-mounted deployment scheme, the features generated by different sensors when illuminating the same target may be different, or the target illuminated by the sensor may be incomplete, which may lead to the inability to accurately detect the target. Specifically, in roadside perception scenarios, the camera and lidar are deployed at a long distance or diagonally to fill the blind spots of different sensors, but this deployment scheme will cause the point cloud target illuminated by the lidar to be different from the observation angle of the point cloud target in the image, which will lead to significant differences in the point cloud features. In addition, even if the vehicle-mounted and roadside perception equipment are deployed at the same angle, some sensors may be blocked by large vehicles, resulting in differences in point cloud features. Therefore, it is necessary to solve the problems of differences in features generated by different sensors when illuminating the same target and incomplete targets illuminated by sensors in roadside perception scenarios. Summary of the Invention

[0005] The purpose of this application is to provide a point cloud restoration processing method, device and computer equipment, which can restore the true state of the target point cloud under different perspectives, so as to reduce the difference in point cloud features when the sensor illuminates the same target under different perspectives.

[0006] The embodiment of the present application is implemented as follows:

[0007] In a first aspect of an embodiment of the present application, a point cloud restoration processing method is provided, the method comprising:

[0008] Acquire training samples, including: original point clouds of samples with missing targets collected from the first perspective, and spliced point clouds of samples with complete targets, perform target detection on the spliced point clouds of the samples, and obtain a complete target point cloud;

[0009] The sample original point cloud is input into the initial restoration model, and the backbone network in the initial restoration model processes it to obtain the bird's-eye view features of the sample original point cloud. The detection head in the initial restoration model outputs the attribute information of the sample original point cloud based on the bird's-eye view features. The encoder in the initial restoration model outputs the first feature descriptor of the sample original point cloud based on the bird's-eye view features, and the decoder in the initial restoration model outputs the predicted target point cloud based on the first feature descriptor.

[0010] A first loss value is calculated based on the attribute information of the sample original point cloud and the true value of the target; a second loss value is calculated based on the first feature descriptor and the second feature descriptors under multiple other perspectives; a third loss value is calculated based on the predicted target point cloud and the complete target point cloud; the initial restoration model is iteratively corrected based on the first loss value, the second loss value, and the third loss value, and the initial restoration model at the end of the iteration is used as the target restoration model;

[0011] The point cloud to be restored is input into the target restoration model to obtain the restored point cloud.

[0012] As a possible implementation method, the first loss value is calculated based on the attribute information of the sample original point cloud and the true value of the target, including:

[0013] The center heat map loss is calculated based on the center heat map of the sample original point cloud and the center heat map truth value of the target;

[0014] The subvoxel position loss is calculated based on the subvoxel position of the sample original point cloud and the subvoxel ground truth of the target;

[0015] The heading angle loss is calculated based on the heading angle of the sample original point cloud and the true value of the heading angle of the target;

[0016] The ground height loss is calculated based on the ground height value of the sample original point cloud and the ground height true value of the target;

[0017] A first loss value is determined according to the center heat map loss, the subvoxel position loss, the heading angle loss, and the ground height loss.

[0018] As a possible implementation method, based on the first feature descriptor and the second feature descriptors under multiple other perspectives, the method further includes:

[0019] Search for relevant point clouds from other perspectives through a preset search algorithm;

[0020] The feature descriptors of the relevant point clouds under each other perspective are used as the second feature descriptors under each other perspective.

[0021] As a possible implementation method, a second loss value is calculated based on the first feature descriptor and a plurality of second feature descriptors under other perspectives, including:

[0022] Calculating the similarity between the first feature descriptor and each second feature descriptor;

[0023] A second loss value is determined according to the similarity between the first feature descriptor and each second feature descriptor.

[0024] As a possible implementation method, the third loss value is calculated based on the predicted target point cloud and the complete target point cloud, including:

[0025] Performing point cloud segmentation on the predicted target point cloud and the complete target point cloud respectively to obtain a plurality of first feature points corresponding to the predicted target point cloud and a plurality of second feature points corresponding to the complete target point cloud;

[0026] Determining the confidence and bias of each first feature point based on each second feature point;

[0027] A third loss value is determined according to the confidence and bias of each first feature point.

[0028] As a possible implementation method, the sample original point cloud is obtained through the following steps:

[0029] Input multiple raw point cloud data within a preset time period into the background filtering and clustering module;

[0030] The background filtering and clustering module determines the point cloud type of each original point cloud data according to the number of frames in each voxel space. The point cloud types include background point cloud, scintillation point cloud and foreground point cloud.

[0031] The background filtering and clustering module performs background filtering on multiple original point cloud data according to the point cloud type of each original point cloud data, and performs clustering operation on the filtered point cloud data to obtain the sample original point cloud.

[0032] As a possible implementation method, the background filtering and clustering module determines the point cloud type of each raw point cloud data according to the number of frames of each raw point cloud data in each voxel space, including:

[0033] If the number of frames of the original point cloud data in the target voxel space is greater than a first preset threshold, the target voxel space is a background voxel, and the point cloud type of the original point cloud data contained in the background voxel is all background point cloud;

[0034] If the number of frames of the original point cloud data in the target voxel space is greater than a second preset threshold, and the number of consecutive blank frames in the target voxel space is less than a third preset threshold, then the target voxel space is a scintillation voxel, and the point cloud type of the original point cloud data contained in the scintillation voxel is all scintillation point cloud;

[0035] If the number of frames of the original point cloud data in the target voxel space is less than the second preset threshold, and the number of consecutive blank frames in the target voxel space is greater than the third preset threshold, then the target voxel space is a foreground voxel, and the point cloud type of the original point cloud data contained in the foreground voxel is all foreground point cloud.

[0036] As a possible implementation method, background filtering is performed on multiple original point cloud data according to the point cloud type of each original point cloud data, and clustering operation is performed on the filtered point cloud data to obtain sample original point clouds, including:

[0037] Filtering out background point clouds and scintillation point clouds from a plurality of original point cloud data to obtain a plurality of foreground point cloud data;

[0038] Clustering operation is performed on multiple foreground point cloud data to obtain sample original point clouds.

[0039] A second aspect of an embodiment of the present application provides a point cloud restoration processing device, the device comprising:

[0040] An acquisition module is used to acquire training samples, including: original point clouds of samples with missing targets collected from the first perspective, and spliced point clouds of samples with complete targets, and perform target detection on the spliced point clouds of samples to obtain complete target point clouds;

[0041] A restoration module is configured to input the sample original point cloud into the initial restoration model, and the backbone network in the initial restoration model processes the sample original point cloud to obtain a bird's-eye view feature of the sample original point cloud. The detection head in the initial restoration model outputs attribute information of the sample original point cloud based on the bird's-eye view feature. The encoder in the initial restoration model outputs a first feature descriptor of the sample original point cloud based on the bird's-eye view feature. The decoder in the initial restoration model outputs a predicted target point cloud based on the first feature descriptor.

[0042] A correction module is configured to calculate a first loss value based on the attribute information of the sample original point cloud and the true value of the target; calculate a second loss value based on the first feature descriptor and the second feature descriptors under multiple other perspectives; calculate a third loss value based on the predicted target point cloud and the complete target point cloud; iteratively correct the initial restoration model based on the first loss value, the second loss value, and the third loss value, and use the initial restoration model at the end of the iteration as the target restoration model;

[0043] The restoration module is also used to input the point cloud to be restored into the target restoration model to obtain the restored point cloud.

[0044] In a third aspect of an embodiment of the present application, a computer device is provided, which includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the computer program is executed by the processor, the point cloud restoration processing method described in the first aspect is implemented.

[0045] In a fourth aspect of an embodiment of the present application, a computer-readable storage medium is provided, which stores a computer program. When the computer program is executed by a processor, the point cloud restoration processing method described in the first aspect is implemented.

[0046] The beneficial effects of the embodiments of the present application include:

[0047] The embodiment of the present application provides a point cloud restoration processing method, which obtains a sample original point cloud with target missing collected by a roadside device from a first perspective, and obtains a sample spliced point cloud obtained by splicing the point cloud data collected by roadside devices from various perspectives, performs target detection on the sample spliced point cloud, and obtains a complete target point cloud; the sample original point cloud is input into an initial restoration model, and the background filtering and clustering module in the initial restoration model preferentially performs background filtering on the sample original point cloud to obtain a valid sample original point cloud, and clusters the filtered valid sample original point cloud through the clustering algorithm in the background filtering and clustering module to obtain the original target corresponding to the sample original point cloud; the backbone network extracts the bird's-eye view features of the sample original point cloud according to the original target output by the background filtering and clustering module, and performs target detection on the bird's-eye view features through the detection head A detection is performed to obtain the attribute information of the sample original point cloud, and the bird's-eye view features are encoded by an encoder to obtain the first feature descriptor of the sample original point cloud; the first feature descriptor is decoded by a decoder to output a predicted target point cloud; a first loss value is calculated based on the attribute information of the sample original point cloud and the true attribute information of the target, and a second loss value is calculated based on the first feature descriptor corresponding to the sample original point cloud and the second feature descriptor corresponding to the point cloud data of the target at other perspectives other than the first perspective, and a third loss value is obtained based on the predicted target point cloud output by the initial restoration model and the complete target point cloud. The initial restoration model is iteratively corrected based on the first loss value, the second loss value, and the third loss value to obtain a target restoration model; the point cloud to be restored is input into the target restoration model to obtain a restored point cloud. Among them, the predicted target point cloud output by the target restoration model is infinitely close to the complete target point cloud, and the attribute information of the target extracted by the target restoration model is close to the true attribute information of the target, and the first feature descriptor extracted by the target restoration model is close to the point cloud features of the target at other perspectives. In this way, the true state of the target point cloud under different perspectives can be restored, thereby reducing the difference in point cloud features when the sensor illuminates the same target under different perspectives. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following is a brief introduction to the drawings required for use in the embodiments. It should be understood that the following drawings only show certain embodiments of the present application and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other relevant drawings can be obtained based on these drawings without creative work.

[0049] Figure 1 A system architecture diagram of a point cloud restoration model provided in an embodiment of the present application;

[0050] Figure 2 A flowchart of the first point cloud restoration processing method provided in an embodiment of the present application;

[0051] Figure 3 A flowchart of the second point cloud restoration processing method provided in an embodiment of the present application;

[0052] Figure 4 A flowchart of a third point cloud restoration processing method provided in an embodiment of the present application;

[0053] Figure 5 A flowchart of a fourth point cloud restoration processing method provided in an embodiment of the present application;

[0054] Figure 6 A flowchart of a fifth point cloud restoration processing method provided in an embodiment of the present application;

[0055] Figure 7 A flowchart of a sixth point cloud restoration processing method provided in an embodiment of the present application;

[0056] Figure 8 A flowchart of a seventh point cloud restoration processing method provided in an embodiment of the present application;

[0057] Figure 9 A flowchart of an eighth point cloud restoration processing method provided in an embodiment of the present application;

[0058] Figure 10 A schematic diagram of the filtering effect of a background filtering and clustering module provided in an embodiment of the present application;

[0059] Figure 11 A schematic structural diagram of a point cloud restoration processing device provided in an embodiment of the present application;

[0060] Figure 12 A schematic diagram of the structure of a computer device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0061] To make the objectives, technical solutions, and advantages of the embodiments of the present application more clear, the technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the accompanying drawings of the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Generally, the components of the embodiments of the present application described and shown in the drawings herein can be arranged and designed in various different configurations.

[0062] Therefore, the following detailed description of the embodiments of the present application provided in the accompanying drawings is not intended to limit the scope of the present application for protection, but merely represents selected embodiments of the present application. All other embodiments obtained by persons of ordinary skill in the art based on the embodiments in the present application without creative work are within the scope of protection of the present application.

[0063] Currently, most object detection fusion solutions utilize a combination of sensors, including imaging, lidar, and millimeter-wave sensors, for on-board object detection. These often utilize on-board sensors, occupancy grids, or BEV perspectives to detect and segment objects around the vehicle. However, when performing object detection in roadside sensing scenarios, significant differences between the deployment of roadside sensing equipment and on-board systems can lead to discrepancies in the features generated when different sensors illuminate the same target, or incomplete sensor illumination of the target, resulting in inaccurate object detection. Specifically, in roadside sensing scenarios, cameras and lidar are deployed at long distances or diagonally to fill in the blind spots of the different sensors. However, such deployments result in different observation angles between the point cloud objects illuminated by the lidar and those in the image, leading to significant discrepancies in point cloud features. Furthermore, even when the on-board and roadside sensing equipment are deployed at the same angle, some sensors may be obscured by large vehicles, resulting in discrepancies in point cloud features.

[0064] To this end, an embodiment of the present application provides a point cloud restoration processing method, which obtains training samples and performs target detection on the sample spliced point cloud in the training samples to obtain a complete target point cloud; the sample original point cloud in the training samples is input into the initial restoration model to obtain the attribute information, bird's-eye view features and first feature descriptors of the sample original point cloud, and the decoder in the initial restoration model decodes the first feature descriptor to obtain a predicted target point cloud; based on the attribute information, bird's-eye view features, first feature descriptors, second feature descriptors of the target at other perspectives, target true values and the complete target point cloud, the initial restoration model is iteratively corrected to obtain a target restoration model; the point cloud to be restored is input into the target restoration model to obtain a restored point cloud. In this way, the true state of the target point cloud under different perspectives can be restored to reduce the difference in point cloud features when the sensor illuminates the same target under different perspectives.

[0065] The point cloud restoration processing method provided in the embodiment of the present application is explained in detail below with reference to the accompanying drawings.

[0066] Figure 1 This is a system architecture diagram of a point cloud restoration model provided by this application, see Figure 1 The point cloud restoration model provided by the embodiment of the present application includes: a background filtering and clustering module, a backbone network, a detection head, an encoder and a decoder, wherein the background filtering and clustering module is used to filter out invalid point clouds in the input original point cloud and the spliced point cloud, and cluster the filtered original point cloud through a clustering algorithm to obtain a point cloud target, and cluster the filtered spliced point cloud through a clustering algorithm to obtain a complete point cloud target; the bird's-eye view features of the point cloud data are extracted through the backbone network; the central heat map, category, heading angle, height and other attribute information of the point cloud target are detected by the detection head; the feature descriptor corresponding to the point cloud target is obtained through the encoder; the feature descriptor is decoded by the decoder to obtain a predicted target point cloud. In addition, based on the first loss value between the attribute information of the original point cloud and the target true value, the second loss value between the feature descriptor of the point cloud target and the feature descriptor of the target at other perspectives, and the third loss value between the predicted target point cloud output by the point cloud restoration model and the complete point cloud target, the point cloud restoration model is iteratively corrected so that the predicted target point cloud finally output by the point cloud restoration model is completely consistent with the target complete point cloud.

[0067] Figure 2 This is a flowchart of a point cloud restoration processing method provided by this application, see Figure 2 This method can be applied to computer equipment, cloud network servers, third servers, etc., and this application does not make specific limitations on this. Figure 2 , an embodiment of the present application provides a point cloud restoration processing method, comprising:

[0068] S201 , obtaining training samples, including: original point clouds of samples with missing targets collected from a first perspective, and spliced point clouds of samples with complete targets, performing target detection on the spliced point clouds of samples to obtain complete target point clouds.

[0069] Optionally, the training sample refers to sample data for training the point cloud restoration model, and the training sample data is obtained through roadside equipment. The roadside equipment can be implemented by remote sensing equipment such as radar cameras. Radar cameras are often installed on mounting poles of road infrastructure. This application does not make specific limitations on this.

[0070] Optionally, the target refers to an object captured by the roadside equipment on the road at the current moment. The target can be a car, pedestrian, bicycle, green belt, etc., and this application does not make specific limitations on this.

[0071] Optionally, the first viewing angle refers to any viewing angle of the detection target. The first viewing angle may be a left viewing angle, a right viewing angle, a front viewing angle, a rear viewing angle, etc. This application does not make any specific limitation on this.

[0072] Optionally, the acquired training sample data includes: a sample original point cloud with a missing target collected from the first perspective and a sample spliced point cloud with a complete target. The sample original point cloud is point cloud data collected at a certain moment by a roadside device deployed at the first perspective. The sample original point cloud is affected by the deployment of the roadside device and is unable to collect the complete point cloud of the target, resulting in the collected point cloud data being relatively one-sided. The original point cloud data from the first perspective cannot represent the complete target, that is, the sample original point cloud collected from the first perspective only contains the point cloud features of the target at the first perspective. The sample spliced point cloud refers to the splicing result of point cloud data collected at the same moment by roadside devices deployed at various perspectives of the target. A complete target point cloud can be obtained by splicing the point clouds of the roadside devices at various perspectives. The complete target point cloud can represent the complete target and contains all the point cloud features of the target. It is worth noting that the point cloud features corresponding to the point cloud data of the target collected at the same moment by roadside devices deployed at different perspectives around the target are different.

[0073] It is worth noting that the original point cloud of the sample collected from the first perspective in the training sample and the sample spliced point cloud obtained by stitching the point clouds collected from each perspective of the target are both for the same target.

[0074] S202. Input the sample original point cloud into the initial restoration model, and the backbone network in the initial restoration model processes it to obtain the bird's-eye view features of the sample original point cloud. The detection head in the initial restoration model outputs the attribute information of the sample original point cloud according to the bird's-eye view features. The encoder in the initial restoration model outputs the first feature descriptor of the sample original point cloud according to the bird's-eye view features, and the decoder in the initial restoration model outputs the predicted target point cloud according to the first feature descriptor.

[0075] Optionally, the initial restoration model is a point cloud restoration model pre-built by the user. This model needs to be trained with a large amount of training sample data to converge before a final, applicable restoration model can be obtained. The initial restoration model has the same structure as the point cloud restoration model described above, including: a background filtering and clustering module, a backbone network, a detection head, an encoder, and a decoder.

[0076] Optionally, after the sample original point cloud is input into the initial restoration model, the background filtering and clustering module in the initial restoration model preferentially performs background filtering on the sample original point cloud to obtain a valid sample original point cloud, and clusters the filtered valid sample original point cloud through the clustering algorithm in the background filtering and clustering module to obtain the original target corresponding to the sample original point cloud; the backbone network extracts the bird's-eye view features of the sample original point cloud according to the original target output by the background filtering and clustering module, and detects the bird's-eye view features through the detection head to obtain the attribute information of the sample original point cloud, and at the same time encodes the bird's-eye view features through the encoder to obtain the first feature descriptor of the sample original point cloud; the first feature descriptor is decoded through the decoder to output the predicted target point cloud.

[0077] Optionally, the bird's-eye view feature refers to the feature of the original target from a bird's-eye view. The bird's-eye view refers to a bird's-eye view based on the God's perspective formed by uniformly projecting images from multiple perspectives into the same bird's-eye view space, that is, the view obtained when looking down at the original target from a high altitude; the attribute information refers to the central heat map, heading angle, type and height above the ground of the original target; the first feature descriptor is used to describe the features of the original target at the first perspective; the predicted target point cloud is the complete point cloud data of the target restored by the initial restoration model based on the point cloud data of the target at the first perspective, that is, the original point cloud data at any perspective is input into the initial restoration model to obtain the complete point cloud data of the target.

[0078] It is worth noting that the predicted target point cloud is not necessarily exactly the same as the real complete point cloud of the target. The power restoration model needs to be further corrected to make the predicted target point cloud as close as possible to the real complete point cloud of the target.

[0079] S203. Calculate a first loss value based on the attribute information of the sample original point cloud and the true value of the target; calculate a second loss value based on the first feature descriptor and the second feature descriptors under multiple other perspectives; calculate a third loss value based on the predicted target point cloud and the complete target point cloud; iteratively correct the initial restoration model based on the first loss value, the second loss value, and the third loss value, and use the initial restoration model at the end of the iteration as the target restoration model.

[0080] Optionally, the true value of the target refers to the attribute characteristics of the target in a real state or an ideal state; the first loss value is used to describe the difference between the attribute information of the original target and the true attribute information of the target; other perspectives refer to perspectives other than the first perspective, that is, other perspectives are perspectives different from the first perspective; the second feature descriptor refers to the point cloud features of other perspectives other than the first perspective; the second loss value is used to describe the difference between the first feature descriptor of the sample original point cloud under the first perspective and the second feature descriptor of the point cloud data of the same target under other perspectives; the complete target point cloud is obtained after splicing and clustering analysis of the point cloud data of the target at various perspectives; the third loss value is used to describe the difference between the predicted target point cloud output by the initial restoration model based on the input sample original point cloud under the first perspective and the complete target point cloud.

[0081] It is worth noting that the first perspective and other perspectives are for the same target. The first feature descriptor extracted from the sample original point cloud under the first perspective and the second feature descriptor extracted from the point cloud under other perspectives are both for the same target, that is, the first feature descriptor is the point cloud feature extracted from the sample original point cloud obtained by the roadside equipment at the first perspective of the target, and the second feature descriptor is the point cloud feature extracted from the point cloud obtained by the roadside equipment at other perspectives of the target other than the first perspective.

[0082] Optionally, the initial restoration model is continuously iteratively corrected according to the first loss value, the second loss value and the third loss value to obtain a target restoration model, the predicted target point cloud output by the target restoration model is infinitely close to the complete target point cloud, and the attribute information of the target extracted by the target restoration model is close to the real attribute information of the target, and the first feature descriptor extracted by the target restoration model is close to the point cloud features of the target under other perspectives.

[0083] S204: Input the point cloud to be restored into the target restoration model to obtain the restored point cloud.

[0084] Alternatively, the point cloud to be restored can be point cloud data collected by a roadside device at any viewing angle of the target. By inputting the point cloud to be restored into the target restoration model, a complete point cloud of the target can be obtained. The restored point cloud output by the target restoration model is the complete point cloud of the target.

[0085] It is worth noting that by inputting the point cloud data collected by the roadside equipment at any perspective of the target into the target restoration model, the complete point cloud data of the target can be obtained.

[0086] In an embodiment of the present application, a sample original point cloud with missing targets collected by a roadside device from a first perspective is obtained, and a sample spliced point cloud is obtained by splicing point cloud data collected by roadside devices from various perspectives. Target detection is performed on the sample spliced point cloud to obtain a complete target point cloud. The sample original point cloud is input into an initial restoration model, and the background filtering and clustering module in the initial restoration model preferentially performs background filtering on the sample original point cloud to obtain a valid sample original point cloud, and the filtered valid sample original point cloud is clustered by the clustering algorithm in the background filtering and clustering module to obtain the original target corresponding to the sample original point cloud. The backbone network extracts the bird's-eye view features of the sample original point cloud according to the original target output by the background filtering and clustering module, and detects the bird's-eye view features via the detection head. The attribute information of the sample original point cloud is obtained, and the bird's-eye view features are encoded through an encoder to obtain a first feature descriptor of the sample original point cloud; the first feature descriptor is decoded through a decoder to output a predicted target point cloud; a first loss value is calculated based on the attribute information of the sample original point cloud and the true attribute information of the target, and a second loss value is calculated based on the first feature descriptor corresponding to the sample original point cloud and the second feature descriptor corresponding to the point cloud data of the target at other perspectives other than the first perspective, and a third loss value is obtained based on the predicted target point cloud output by the initial restoration model and the complete target point cloud. The initial restoration model is iteratively corrected based on the first loss value, the second loss value, and the third loss value to obtain a target restoration model; the point cloud to be restored is input into the target restoration model to obtain a restored point cloud. Among them, the predicted target point cloud output by the target restoration model is infinitely close to the complete target point cloud, and the attribute information of the target extracted by the target restoration model is close to the true attribute information of the target, and the first feature descriptor extracted by the target restoration model is close to the point cloud features of the target at other perspectives. In this way, the true state of the target point cloud under different perspectives can be restored, thereby reducing the difference in point cloud features when the sensor illuminates the same target under different perspectives.

[0087] In one possible implementation, see Figure 3 The operation of step S202 may specifically be:

[0088] S301. Calculate the center heat map loss based on the center heat map of the sample original point cloud and the center heat map true value of the target.

[0089] Optionally, the central heat map refers to high values, low values, clustering models and outliers in the sample original point cloud, and color coding is used to represent the relative values of the data of the sample original point cloud; the true value of the central heat map refers to the central heat map of the complete point cloud of the target under ideal conditions.

[0090] Optionally, the center heatmap loss is used to indicate the gap between the center heatmap of the sample original point cloud and the true value of the center heatmap of the target. The center heatmap loss calculation process is as follows: for the K categories of the point cloud data, heatmaps of K channels are generated respectively, and the target center in the heatmap is projected into a bird's-eye view to generate a two-dimensional Gaussian function, and the key points are distributed to the feature map through Gaussian filtering.

[0091] Optionally, the key points of the heat map can be projected onto the bird's-eye view according to the following formula (1):

[0092]

[0093] in, and is the center coordinate of the projected bird's-eye view, (x, y) refers to the center coordinate of the central heat map of the sample original point cloud, Y xyc Heatmap loss used to represent the original point cloud of the sample.

[0094] S302 : Calculate the subvoxel position loss according to the subvoxel position of the sample original point cloud and the subvoxel true value of the target.

[0095] Optionally, a voxel is the smallest unit in a three-dimensional space, a sub-voxel indicates that the voxel space is further subdivided into multiple smaller voxel spaces, the sub-voxel position refers to the position of the sample original point cloud in each sub-voxel space in the three-dimensional space, and the sub-voxel true value refers to the position of the target's complete point cloud data in each sub-voxel space in the three-dimensional space.

[0096] Optionally, the subvoxel position loss is used to indicate the difference between the position of the original sample point cloud in each subvoxel space in the three-dimensional space and the position of the target complete point cloud data in each subvoxel space in the three-dimensional space. 2 ,Subvoxel position optimization reduces the errors of voxelization and backbone network.

[0097] S303: Calculate the heading angle loss based on the heading angle of the sample original point cloud and the true value of the heading angle of the target.

[0098] Optionally, the heading angle is used to indicate the direction or orientation of the original target obtained by clustering the sample original point cloud, the true value of the heading angle is used to indicate the direction or orientation of the target represented by the complete point cloud of the target, and the heading angle loss indicates the difference between the direction of the target represented by the sample original point cloud and the direction of the target represented by the complete target point cloud. For example, the heading angle (sin(α), cos(α))∈R 2 The heading angle can predict the direction of the target. In order to avoid excessive loss when the direction is opposite to the true direction of the target, the sine or cosine value of the true heading angle is used as the regression target.

[0099] S304: Calculate the ground height loss based on the ground height value of the sample original point cloud and the ground height true value of the target.

[0100] Optionally, the ground height value refers to the height between the original sample point cloud and the ground, the ground height truth value refers to the height between the complete point cloud data of the target and the ground, and the ground high speed loss refers to the difference between the height value between the original sample point cloud and the ground and the height value between the complete target point cloud and the ground. For example, the ground height hg∈R, the 3D size s∈R 3 ,The ground height hg helps to locate objects in the height range and make up for the missing height information when projecting from a bird's-eye view.

[0101] S305 : Determine a first loss value according to the center heat map loss, the sub-voxel position loss, the heading angle loss, and the ground height loss.

[0102] Optionally, based on the center heat map loss, sub-voxel position loss, heading angle loss and ground height loss, the first loss value of the attribute information of the sample original point cloud can be calculated, wherein the position optimization is combined with the ground height and heat points to predict the specific spatial position of the target.

[0103] In one possible implementation, see Figure 4 , according to the first feature descriptor and the second feature descriptors under multiple other perspectives, further comprising:

[0104] S401 , searching for relevant point clouds at other viewing angles using a preset search algorithm.

[0105] Optionally, the preset search algorithm refers to an algorithm used to search for point clouds of the same target from perspectives other than the first perspective. The preset search algorithm may be a center distance algorithm, a Hungarian algorithm, or other algorithms, and this application does not make any specific limitations on this.

[0106] Optionally, the related point cloud refers to point cloud data collected by roadside equipment at other perspectives of the same target besides the first perspective.

[0107] S402 : Using the feature descriptors of the relevant point clouds under each other viewing angle as the second feature descriptors under each other viewing angle.

[0108] Optionally, feature descriptors extracted from relevant point clouds of other perspectives other than the first perspective are used as second feature descriptors of other perspectives.

[0109] In one possible implementation, see Figure 5 The operation of step S202 may specifically be:

[0110] S501: Calculate the similarity between the first feature descriptor and each second feature descriptor.

[0111] Optionally, the similarity refers to the degree of correlation between the first feature descriptor and the second feature descriptor. The similarity can be represented by Euclidean distance or cosine distance, which is not specifically limited in this application.

[0112] S502: Determine a second loss value according to the similarity between the first feature descriptor and each second feature descriptor.

[0113] Optionally, a second loss value between the first feature descriptor and the second feature descriptor is determined according to a correlation degree between the first feature descriptor and the second feature descriptor.

[0114] In one possible implementation, see Figure 6 The operation of step S202 may specifically be:

[0115] S601 , performing point cloud segmentation on the predicted target point cloud and the complete target point cloud respectively to obtain a plurality of first feature points corresponding to the predicted target point cloud and a plurality of second feature points corresponding to the complete target point cloud.

[0116] Optionally, the predicted target point cloud and the complete target point cloud output by the initial restoration model are segmented by a segmentation head, so as to obtain multiple first feature points of the predicted target point cloud in the bird's-eye view and multiple second feature points of the complete target point cloud in the bird's-eye view.

[0117] S602: Determine the confidence and bias of each first feature point based on each second feature point.

[0118] Optionally, the confidence and bias of multiple first feature points of the sample original point cloud in the bird's-eye view at the first perspective are predicted based on the multiple second feature points of the complete target point cloud in the bird's-eye view. The confidence refers to whether there is a complete target point cloud in the grid where the first feature point is located, and the bias refers to the offset between the first feature point and the center position of the grid where it is located.

[0119] S603: Determine a third loss value according to the confidence and bias of each first feature point.

[0120] Optionally, a third loss value between the predicted target point cloud and the complete target point cloud is determined based on the confidence and bias of the first feature point of the predicted target point cloud.

[0121] In one possible implementation, see Figure 7 In step S201, the sample original point cloud is obtained by the following steps:

[0122] S701: Input a plurality of original point cloud data within a preset time period into a background filtering and clustering module in an initial restoration model.

[0123] Optionally, the preset period is a time period set by the user, and the preset period may be 30 seconds, 3 minutes, etc., which is not specifically limited in this application.

[0124] Optionally, a plurality of original point cloud data collected from the first perspective within a preset time period are input into a background filtering and clustering module in the initial restoration model.

[0125] S702 , the background filtering and clustering module determines the point cloud type of each original point cloud data according to the number of frames of each original point cloud data in each voxel space. The point cloud types include: background point cloud, scintillation point cloud, and foreground point cloud.

[0126] Optionally, the background filtering and clustering module determines the point cloud type of each raw point cloud data based on the number of frames in voxel space. Point cloud types include background point cloud, scintillation point cloud, and foreground point cloud. Background point cloud refers to the point cloud of the background image in the image captured by the radar camera, scintillation point cloud refers to the point cloud generated by the laser flash of the radar camera, and foreground point cloud refers to the point cloud of the target's motion in the image captured by the radar camera.

[0127] S703 , the background filtering and clustering module performs background filtering on the plurality of original point cloud data according to the point cloud type of each original point cloud data, and performs clustering operation on the filtered point cloud data to obtain a sample original point cloud.

[0128] Optionally, the background filtering and clustering module performs background filtering on the original point cloud data according to the point cloud type of the original point cloud data to obtain valid point cloud data, and after clustering the valid point cloud data, sample original point cloud data can be obtained.

[0129] In one possible implementation, see Figure 8 The operation of step S702 may specifically be:

[0130] S801: If the number of frames of the original point cloud data in the target voxel space is greater than a first preset threshold, the target voxel space is a background voxel, and the point cloud type of the original point cloud data contained in the background voxel is all background point cloud.

[0131] Optionally, the target voxel space refers to any voxel space in the three-dimensional space, and the first preset threshold is a background point cloud threshold pre-set by the user. When the number of frames of the original point cloud data falling in the target voxel space is greater than the first preset threshold, the voxel space is defaulted to the background voxel, and all point cloud data falling in the background voxel thereafter are defaulted to the background point cloud.

[0132] S802. If the number of frames of the original point cloud data in the target voxel space is greater than a second preset threshold, and the number of consecutive blank frames in the target voxel space is less than a third preset threshold, then the target voxel space is a scintillation voxel, and the point cloud type of the original point cloud data contained in the scintillation voxel is all scintillation point cloud.

[0133] Optionally, the second preset threshold is a scintillation point cloud threshold preset by the user, and the third preset threshold is a blank frame threshold preset by the user. When the number of frames of the original point cloud data falling in the target voxel space is greater than the second preset threshold, and the number of consecutive blank frames in the target voxel space is less than the third preset threshold, the voxel space is defaulted to a scintillation voxel, and all point cloud data falling in the scintillation voxel thereafter are defaulted to a scintillation point cloud.

[0134] S803. If the number of frames of the original point cloud data in the target voxel space is less than the second preset threshold, and the number of consecutive blank frames in the target voxel space is greater than the third preset threshold, then the target voxel space is a foreground voxel, and the point cloud type of the original point cloud data contained in the foreground voxel is all foreground point cloud.

[0135] Optionally, when the number of frames of the original point cloud data falling in the target voxel space is less than a second preset threshold, and the number of consecutive blank frames in the target voxel space is greater than a third preset threshold, the voxel space is defaulted to a foreground voxel, and all subsequent point cloud data falling in the foreground voxel are defaulted to a foreground point cloud.

[0136] In one possible implementation, see Figure 9 The operation of step S703 may specifically be:

[0137] S901: Filter out background point clouds and scintillation point clouds from a plurality of original point cloud data to obtain a plurality of foreground point cloud data.

[0138] Optionally, after filtering out the background point cloud and the glint point cloud in the input original point cloud data, a point cloud data set containing only the foreground point cloud data can be obtained.

[0139] S902: Perform clustering operation on multiple foreground point cloud data to obtain sample original point clouds.

[0140] Optionally, after performing a clustering operation on the foreground point cloud data, the sample original point cloud can be obtained.

[0141] Figure 10 This is a schematic diagram of the filtering effect of a background filtering and clustering module provided in this application, see Figure 10 The point cloud in the background point cloud is relatively dense, the flickering point cloud has a certain regularity, and there are many continuous blank frames in the foreground point cloud. The specific functions of the background filtering and clustering modules will not be described in detail.

[0142] The following describes the apparatus, device, and computer-readable storage medium used to execute the point cloud restoration processing method provided in this application. The specific implementation process and technical effects are described above and will not be repeated below.

[0143] Figure 11 This is a schematic diagram of the structure of a point cloud restoration processing device provided by an embodiment of the present application, see Figure 11 , the device comprises:

[0144] An acquisition module 1101 is configured to acquire training samples, wherein the training samples include: original point clouds of samples with missing targets collected at a first viewing angle, and spliced point clouds of samples with complete targets, and perform target detection on the spliced point clouds of the samples to obtain a complete target point cloud;

[0145] Restoration module 1102 is configured to input the sample original point cloud into an initial restoration model, obtain bird's-eye view features of the sample original point cloud through processing by the backbone network in the initial restoration model, output attribute information of the sample original point cloud based on the bird's-eye view features by the detection head in the initial restoration model, output a first feature descriptor of the sample original point cloud based on the bird's-eye view features by the encoder in the initial restoration model, and output a predicted target point cloud based on the first feature descriptor by the decoder in the initial restoration model;

[0146] The correction module 1103 is configured to calculate a first loss value based on the attribute information of the sample original point cloud and the true value of the target; calculate a second loss value based on the first feature descriptor and second feature descriptors under multiple other perspectives; calculate a third loss value based on the predicted target point cloud and the complete target point cloud; iteratively correct the initial restoration model based on the first loss value, the second loss value, and the third loss value, and use the initial restoration model at the end of the iteration as the target restoration model;

[0147] The restoration module 1102 is further configured to input the point cloud to be restored into the target restoration model to obtain a restored point cloud.

[0148] In one possible implementation, the correction module 1103 is specifically configured to:

[0149] The center heat map loss is calculated based on the center heat map of the sample original point cloud and the center heat map truth value of the target;

[0150] The subvoxel position loss is calculated based on the subvoxel position of the sample original point cloud and the subvoxel ground truth of the target;

[0151] The heading angle loss is calculated based on the heading angle of the sample original point cloud and the true value of the heading angle of the target;

[0152] The ground height loss is calculated based on the ground height value of the sample original point cloud and the ground height true value of the target;

[0153] A first loss value is determined according to the center heat map loss, the subvoxel position loss, the heading angle loss, and the ground height loss.

[0154] In a possible implementation, the correction module 1103 is further configured to:

[0155] Search for relevant point clouds from other perspectives through a preset search algorithm;

[0156] The feature descriptors of the relevant point clouds under each other perspective are used as the second feature descriptors under each other perspective.

[0157] In a possible implementation, the correction module 1103 may further be used to:

[0158] Calculating the similarity between the first feature descriptor and each second feature descriptor;

[0159] A second loss value is determined according to the similarity between the first feature descriptor and each second feature descriptor.

[0160] In a possible implementation, the correction module 1103 may further be used to:

[0161] Performing point cloud segmentation on the predicted target point cloud and the complete target point cloud respectively to obtain a plurality of first feature points corresponding to the predicted target point cloud and a plurality of second feature points corresponding to the complete target point cloud;

[0162] Determining the confidence and bias of each first feature point based on each second feature point;

[0163] A third loss value is determined according to the confidence and bias of each first feature point.

[0164] In one possible implementation, the acquisition module 1101 is specifically configured to:

[0165] Input multiple raw point cloud data within a preset time period into the background filtering and clustering module;

[0166] The background filtering and clustering module determines the point cloud type of each original point cloud data according to the number of frames in each voxel space. The point cloud types include background point cloud, scintillation point cloud and foreground point cloud.

[0167] The background filtering and clustering module performs background filtering on multiple original point cloud data according to the point cloud type of each original point cloud data, and performs clustering operation on the filtered point cloud data to obtain the sample original point cloud.

[0168] In a possible implementation, the acquisition module 1101 is further configured to:

[0169] If the number of frames of the original point cloud data in the target voxel space is greater than a first preset threshold, the target voxel space is a background voxel, and the point cloud type of the original point cloud data contained in the background voxel is all background point cloud;

[0170] If the number of frames of the original point cloud data in the target voxel space is greater than a second preset threshold, and the number of consecutive blank frames in the target voxel space is less than a third preset threshold, then the target voxel space is a scintillation voxel, and the point cloud type of the original point cloud data contained in the scintillation voxel is all scintillation point cloud;

[0171] If the number of frames of the original point cloud data in the target voxel space is less than the second preset threshold, and the number of consecutive blank frames in the target voxel space is greater than the third preset threshold, then the target voxel space is a foreground voxel, and the point cloud type of the original point cloud data contained in the foreground voxel is all foreground point cloud.

[0172] In a possible implementation, the acquisition module 1101 may further be used to:

[0173] Filtering out background point clouds and scintillation point clouds from a plurality of original point cloud data to obtain a plurality of foreground point cloud data;

[0174] Clustering operation is performed on multiple foreground point cloud data to obtain sample original point clouds.

[0175] The above-mentioned device is used to execute the method provided in the above-mentioned embodiment. Its implementation principle and technical effect are similar and will not be repeated here.

[0176] The above modules can be one or more integrated circuits configured to implement the above methods, such as one or more application-specific integrated circuits (ASICs), one or more microprocessors, or one or more field programmable gate arrays (FPGAs). For another example, when a module is implemented by scheduling program code through a processing element, the processing element can be a general-purpose processor, such as a central processing unit (CPU) or other processor that can call program code. For another example, these modules can be integrated together and implemented in the form of a system-on-a-chip (SOC).

[0177] Figure 12 This is a schematic diagram of the structure of a computer device provided in an embodiment of the present application. Figure 12The computer device includes: a memory 1201 and a processor 1202. The memory 1201 stores a computer program that can be run on the processor 1202. When the processor 302 executes the computer program, the steps in any of the above method embodiments are implemented.

[0178] An embodiment of the present application further provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, the steps in the above-mentioned method embodiments can be implemented.

[0179] Optionally, the present application also provides a program product, such as a computer-readable storage medium, comprising a program, which is used to execute any of the above-mentioned point cloud restoration processing method embodiments when executed by a processor.

[0180] In the several embodiments provided by the present invention, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are merely illustrative. For example, the division of units is merely a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the mutual coupling or direct coupling or communication connection shown or discussed can be through some interface, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0181] Units described as separate components may or may not be physically separate, and components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected to achieve the purpose of this embodiment according to actual needs.

[0182] In addition, the functional units in various embodiments of the present invention may be integrated into a single processing unit, each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or hardware plus software functional units.

[0183] The above-mentioned integrated unit implemented in the form of a software functional unit can be stored in a computer-readable storage medium. The above-mentioned software functional unit is stored in a storage medium and includes a number of instructions for causing a computer device (which can be a personal computer, server, or network device, etc.) or a processor (English: processor) to perform some steps of the methods of various embodiments of the present invention. The aforementioned storage medium includes: a USB flash drive, a mobile hard disk, a read-only memory (English: Read-Only Memory, abbreviated: ROM), a random access memory (English: Random Access Memory, abbreviated: RAM), a magnetic disk or an optical disk, and other media that can store program code.

[0184] The above are only specific embodiments of the present application, but the scope of protection of this application is not limited thereto. Any changes or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in this application should be included in the scope of protection of this application. Therefore, the scope of protection of this application should be based on the scope of protection of the claims.

[0185] The above description is merely a preferred embodiment of the present application and is not intended to limit the present application. Various modifications and variations are possible for those skilled in the art. Any modifications, equivalent substitutions, or improvements made within the spirit and principles of the present application shall be included within the scope of protection of the present application.

Claims

1. A point cloud restoration processing method, characterized in that: The method comprises: Acquire training samples, the training samples including: original point clouds of samples with missing targets collected at a first perspective, and spliced point clouds of samples with complete targets, perform target detection on the spliced point clouds of samples to obtain a complete target point cloud; Inputting the sample original point cloud into an initial restoration model, the backbone network in the initial restoration model processes the sample original point cloud to obtain a bird's-eye view feature of the sample original point cloud, the detection head in the initial restoration model outputs attribute information of the sample original point cloud based on the bird's-eye view feature, the encoder in the initial restoration model outputs a first feature descriptor of the sample original point cloud based on the bird's-eye view feature, and the decoder in the initial restoration model outputs a predicted target point cloud based on the first feature descriptor; Calculate a first loss value based on the attribute information of the sample original point cloud and the true value of the target; calculate a second loss value based on the first feature descriptor and second feature descriptors under multiple other perspectives; calculate a third loss value based on the predicted target point cloud and the complete target point cloud; iteratively correct the initial restoration model based on the first loss value, the second loss value, and the third loss value, and use the initial restoration model at the end of the iteration as the target restoration model; The point cloud to be restored is input into the target restoration model to obtain the restored point cloud.

2. The point cloud restoration processing method according to claim 1, characterized in that: The calculating a first loss value according to the attribute information of the sample original point cloud and the true value of the target includes: Calculating a central heatmap loss based on the central heatmap of the sample original point cloud and the true value of the central heatmap of the target; Calculating a subvoxel position loss according to the subvoxel position of the sample original point cloud and the subvoxel true value of the target; Calculating a heading angle loss according to the heading angle of the sample original point cloud and the true value of the heading angle of the target; Calculating a ground height loss based on the ground height value of the sample original point cloud and the ground height true value of the target; The first loss value is determined according to the central heat map loss, the sub-voxel position loss, the heading angle loss, and the ground height loss.

3. The point cloud restoration processing method according to claim 1, characterized in that: Before the method is performed based on the first feature descriptor and the second feature descriptors under multiple other perspectives, the method further includes: Searching for relevant point clouds under each of the other perspectives using a preset search algorithm; The feature descriptors of the relevant point clouds under each other perspective are used as the second feature descriptors under each other perspective.

4. The point cloud restoration processing method according to claim 1, characterized in that: The calculating a second loss value according to the first feature descriptor and a plurality of second feature descriptors under other perspectives includes: Calculating the similarity between the first feature descriptor and each second feature descriptor; The second loss value is determined according to the similarity between the first feature descriptor and each second feature descriptor.

5. The point cloud restoration processing method according to claim 1, characterized in that: The calculating a third loss value according to the predicted target point cloud and the complete target point cloud includes: Performing point cloud segmentation on the predicted target point cloud and the complete target point cloud respectively to obtain a plurality of first feature points corresponding to the predicted target point cloud and a plurality of second feature points corresponding to the complete target point cloud; Determining the confidence and bias of each of the first feature points based on each of the second feature points; The third loss value is determined according to the confidence and bias of each of the first feature points.

6. The point cloud restoration processing method according to any one of claims 1 to 5, characterized in that: The sample original point cloud is obtained through the following steps: Inputting multiple original point cloud data within a preset time period into the background filtering and clustering module in the initial restoration model; The background filtering and clustering module determines the point cloud type of each of the original point cloud data according to the number of frames of each of the original point cloud data in each voxel space, wherein the point cloud types include: background point cloud, scintillation point cloud, and foreground point cloud; The background filtering and clustering module performs background filtering on the plurality of original point cloud data according to the point cloud type of each original point cloud data, and performs clustering operation on the filtered point cloud data to obtain the sample original point cloud.

7. The point cloud restoration processing method according to claim 6, characterized in that: The background filtering and clustering module determines the point cloud type of each of the original point cloud data according to the number of frames of each of the original point cloud data in each voxel space, including: If the number of frames of the original point cloud data in the target voxel space is greater than a first preset threshold, the target voxel space is a background voxel, and the point cloud type of the original point cloud data contained in the background voxel is all background point cloud; If the number of frames of the original point cloud data in the target voxel space is greater than a second preset threshold, and the number of consecutive blank frames in the target voxel space is less than a third preset threshold, then the target voxel space is a scintillation voxel, and the point cloud type of the original point cloud data contained in the scintillation voxel is all scintillation point cloud; If the number of frames of the original point cloud data in the target voxel space is less than a second preset threshold, and the number of consecutive blank frames in the target voxel space is greater than a third preset threshold, then the target voxel space is a foreground voxel, and the point cloud type of the original point cloud data contained in the foreground voxel is all foreground point cloud.

8. The point cloud restoration processing method according to claim 6, characterized in that: The performing background filtering on the plurality of original point cloud data according to the point cloud type of each original point cloud data, and performing clustering operation on the filtered point cloud data to obtain the sample original point cloud, includes: Filtering out background point clouds and scintillation point clouds from the plurality of original point cloud data to obtain a plurality of foreground point cloud data; A clustering operation is performed on the plurality of foreground point cloud data to obtain the sample original point cloud.

9. A point cloud restoration processing device, characterized in that: The device comprises: an acquisition module, configured to acquire training samples, wherein the training samples include: original point clouds of samples with missing targets acquired at a first perspective, and spliced point clouds of samples with complete targets, and perform target detection on the spliced point clouds of samples to obtain a complete target point cloud; a restoration module, configured to input the sample original point cloud into an initial restoration model, obtain bird's-eye view features of the sample original point cloud by processing the backbone network in the initial restoration model, output attribute information of the sample original point cloud based on the bird's-eye view features by the detection head in the initial restoration model, output a first feature descriptor of the sample original point cloud based on the bird's-eye view features by the encoder in the initial restoration model, and output a predicted target point cloud based on the first feature descriptor by the decoder in the initial restoration model; a correction module, configured to calculate a first loss value based on the attribute information of the sample original point cloud and the true value of the target; calculate a second loss value based on the first feature descriptor and second feature descriptors under multiple other perspectives; calculate a third loss value based on the predicted target point cloud and the complete target point cloud; iteratively correct the initial restoration model based on the first loss value, the second loss value, and the third loss value, and use the initial restoration model at the end of the iteration as the target restoration model; The restoration module is further used to input the point cloud to be restored into the target restoration model to obtain the restored point cloud.

10. A computer device, characterized in that: include: A memory and a processor, wherein the memory stores a computer program that can be run on the processor, and when the processor executes the computer program, the steps of the method described in any one of claims 1 to 8 are implemented.

Images