An intelligent automobile laser radar point cloud anomaly detection method based on deep learning

Abstract

This invention relates to a method for anomaly detection in point clouds of intelligent vehicle LiDAR based on deep learning. The invention addresses the problems of data annotation and real-time performance in dense anomaly detection of point clouds using existing deep learning models. The method involves inputting the point cloud to be tested into a trained deep learning model to obtain a reconstructed noise-free point cloud and anomaly categories. When the anomaly category is "no anomaly," the anomaly detection result is "no anomaly." When the anomaly category is an unprocessable anomaly, the point cloud is considered an anomaly. When the anomaly category is a processable anomaly, the absolute value of the relative distance difference is calculated. If the value is greater than a threshold, the k-th point in the point cloud is considered an anomaly; otherwise, the k-th point is not identified as an anomaly. This process continues until all points in the point cloud have been identified as anomalies. This invention is applicable to the field of anomaly detection in intelligent vehicle LiDAR point clouds.

Description

Technical Field

[0001] This invention relates to a deep learning-based method for anomaly detection of point clouds in intelligent automotive LiDAR systems. Background Technology

[0002] Establishing comprehensive safety standards to ensure safety during driving is crucial for intelligent vehicles, especially autonomous vehicles. In intelligent vehicle systems, perception sensors such as cameras and LiDAR capture environmental information, providing a vital foundation for system planning and decision-making. Therefore, ensuring the normal operation of perception sensors or troubleshooting faulty sensors is also essential for achieving intelligent vehicle safety. Currently, LiDAR plays a key role in advanced autonomous vehicles. However, in actual operation, complex environmental factors can cause various interferences to LiDAR. According to the detection principle of LiDAR, particles in the air under adverse weather conditions (rain, snow, fog, etc.) can affect the scattering or reflection of light, leading to unwanted noise from the LiDAR. It can also cause contamination of the radar panel, causing more serious malfunctions. Depending on the internal structure of the LiDAR, mechanical failures or external signal interference can lead to abnormal point cloud results. These noisy anomalies can severely affect the performance of perception algorithms, ultimately having a catastrophic impact on the entire system. Previous literature has yielded significant results in LiDAR fault detection. Some works have established mathematical models of laser reflection or scattering under adverse weather conditions, theoretically providing some results on LiDAR performance degradation. Some works start directly from the final point cloud results, using improved clustering algorithms to treat poorly clustered points as noise or outliers. A recent work proposed a universal anomaly detection algorithm that determines whether a result is abnormal by calculating spatial correlation and analyzing point cloud intensity anomalies. This method can detect point cloud anomalies caused by factors such as abnormal weather and signal interference. Considering the complexity of factors causing anomalies in real-world situations, making it difficult to build accurate models, some works use deep learning algorithms for fault detection and classification. While deep learning models can handle very complex problems, applying them to point cloud anomaly detection presents two challenges. The first is the data labeling problem. Deep learning models rely on labeled data for training, and generally, the more training data, the better the generalization. However, labeling each point in anomaly points is extremely difficult, and building such a large dataset requires enormous human resources. The second is the computational complexity of deep learning models, while point cloud anomaly detection algorithms, as a preprocessing step, need to run at high speed and frequency; ensuring real-time performance is also a challenge. Due to these issues, most existing deep learning-based LiDAR fault detection algorithms remain at the global classification level. Summary of the Invention

[0003] The purpose of this invention is to address the data annotation problem in existing methods for dense anomaly detection of point clouds using deep learning models, and the problem of ensuring real-time performance while achieving dense anomaly detection. Therefore, this invention proposes a deep learning-based method for anomaly detection of point clouds in intelligent automotive LiDAR systems.

[0004] The specific process of a deep learning-based intelligent vehicle LiDAR point cloud anomaly detection method is as follows:

[0005] Step 1: Build a deep learning model;

[0006] Step 2: Input the simulated abnormal point clouds into the deep learning model in batches to obtain the reconstructed noise-free point clouds and abnormal categories;

[0007] The multi-task loss function is calculated based on the reconstructed noise-free point cloud and anomaly categories until the deep learning model converges, thus obtaining the trained deep learning model.

[0008] Step 3: Input the point cloud to be tested into the trained deep learning model to obtain the reconstructed noise-free point cloud and anomaly categories;

[0009] When the anomaly category of the point cloud to be tested is no anomaly, the anomaly detection result of the point cloud to be tested is no anomaly.

[0010] When the anomaly category of the point cloud to be tested is an unprocessable anomaly, the point cloud to be tested is an anomalous point cloud.

[0011] When the anomaly category of the point cloud under test is a processable anomaly, the absolute value of the relative distance difference is calculated based on the distance between the k-th point in the reconstructed noise-free point cloud and the distance between the k-th point in the input point cloud under test. When the absolute value of the relative distance difference is greater than a threshold, the k-th point in the point cloud under test is an anomaly. When the absolute value of the relative distance difference is less than or equal to the threshold, the k-th point in the point cloud under test is not identified as an anomaly. This process continues until all points in the point cloud under test are determined to be anomalies.

[0012] The point cloud is obtained from simulated vehicle-mounted lidar sensors.

[0013] The beneficial effects of this invention are as follows:

[0014] This invention transforms the point cloud anomaly detection problem into a point cloud denoising problem. While achieving anomaly detection in dense point clouds, it can also use simulation data to alleviate the difficulty of annotating abnormal point cloud data.

[0015] This invention decouples local point cloud features from global point cloud features. A lightweight deep learning module is used to achieve real-time extraction of local point cloud features. Furthermore, this invention divides the point cloud into multiple windows, inputting the point cloud data within each window into the local point cloud feature extraction module, thus reducing the receptive field requirements of the local point cloud feature extraction module.

[0016] The multi-task loss function proposed in this invention adds an anomaly type classification loss function and a point cloud quality alignment loss function to the point cloud reconstruction loss function. The anomaly type classification loss function can supervise the global feature extraction network, and the anomaly type classification results also provide important evidence for the final point cloud anomaly detection results. This ensures reasonable anomaly detection results even when the point cloud has almost no anomalies or the anomalies are very severe.

[0017] Furthermore, conventional point cloud reconstruction loss functions only consider the reconstruction quality of a single point cloud, but there is often a certain correlation between adjacent point clouds. This invention further proposes a point cloud quality alignment loss function to ensure the local consistency of point cloud reconstruction results. Attached Figure Description

[0018] Figure 1 This is a flowchart of the present invention;

[0019] Figure 2 This is an overall block diagram of the deep learning algorithm proposed in this invention during training.

[0020] Figure 3 This is an overall block diagram of the deep learning algorithm inference proposed in this invention. Detailed Implementation

[0021] Specific Implementation Method 1: The specific process of this implementation method for anomaly detection of intelligent vehicle LiDAR point cloud based on deep learning is as follows:

[0022] This invention provides a lightweight deep learning algorithm for dense lidar point cloud anomaly detection, and a novel multi-task loss function to supervise the proposed deep learning algorithm.

[0023] In this invention, point cloud anomaly detection is transformed into a point cloud denoising problem, so that simulation data can be directly used for training, thus alleviating the data labeling problem.

[0024] The deep learning model proposed in this invention mainly consists of three parts: a very lightweight deep learning model for processing local point clouds, called the local point cloud feature extraction module; a lightweight deep learning model for extracting global point cloud features and classifying anomaly types, called the global point cloud feature extraction network; and a very lightweight deep learning model for fusing the features extracted by the local point cloud feature extraction module and the features extracted by the global point cloud feature extraction network to reconstruct a noise-free local point cloud, called the fusion decoding module.

[0025] Step 1: Build a deep learning model;

[0026] Step 2: Input the labeled simulated anomalous point clouds (which may include non-annomous point clouds) into the deep learning model in batches to obtain the reconstructed noise-free point clouds and anomaly categories.

[0027] The multi-task loss function is calculated based on the reconstructed noise-free point cloud and anomaly categories until the deep learning model converges, thus obtaining the trained deep learning model.

[0028] The gradient is calculated based on the multi-task loss function and the backpropagation algorithm. The parameters of the deep learning model are continuously adjusted through a suitable optimizer and step size to obtain a well-trained deep learning model.

[0029] Continuously adjust hyperparameters such as batch size, learning rate, and number of iterations during the training process to maximize the anomaly detection accuracy of the trained algorithm model on the validation set.

[0030] Step 3: Input the point cloud to be tested into the trained deep learning model to obtain the reconstructed noise-free point cloud and anomaly categories;

[0031] When the anomaly category of the point cloud to be tested is no anomaly, it indicates that the current point cloud is less contaminated, and the anomaly detection result of the point cloud to be tested is no anomaly.

[0032] When the anomaly category of the point cloud to be tested is an unprocessable anomaly, it indicates that the current point cloud is severely contaminated, and the point cloud to be tested is an anomalous point cloud.

[0033] When the anomaly category of the point cloud under test is a processable anomaly, the absolute value of the relative distance difference is calculated based on the distance between the kth point in the reconstructed noise-free point cloud and the distance between the kth point in the input point cloud under test. When the absolute value of the relative distance difference is greater than the threshold, the kth point in the point cloud under test is an anomaly. When the absolute value of the relative distance difference is less than or equal to the threshold, the kth point in the point cloud under test is not identified as an anomaly. This process continues until all points in the point cloud under test are determined to be anomalies.

[0034] The point cloud is point cloud data acquired by a simulated vehicle-mounted LiDAR sensor, such as road edge lines, lane lines, traffic signs, and various types of obstacles;

[0035] Each point in a point cloud contains three-dimensional coordinates.

[0036] Specific Implementation Method Two: This implementation method differs from Specific Implementation Method One in that: in step one, a deep learning model is built; the specific process is as follows:

[0037] The deep learning model includes a local point cloud feature extraction module, a global point cloud feature extraction module, and a fusion decoding module;

[0038] The local point cloud feature extraction module includes, in sequence, a first 3×3 convolutional layer, a first 1×1 convolutional layer, a second 3×3 convolutional layer, a second 1×1 convolutional layer, a third 3×3 convolutional layer, and a third 1×1 convolutional layer;

[0039] The global point cloud feature extraction module is the encoder of SqueezeSegV2;

[0040] The fusion decoding module includes, in sequence, a first 2×2 deconvolution layer, a second 2×2 deconvolution layer, and a third 2×2 deconvolution layer.

[0041] The other steps and parameters are the same as in Specific Implementation Method 1.

[0042] Specific Implementation Method Three: This implementation method differs from Specific Implementation Method One or Two in that:

[0043] In step two, labeled simulated anomaly point clouds (which may include anomaly-free point clouds) are batch-input into the deep learning model to obtain reconstructed noise-free point clouds and anomaly categories; the specific process is as follows:

[0044] Step 21: Obtain point cloud pairs, which include simulated point clouds without anomalies and simulated point clouds with anomalies; the specific process is as follows:

[0045] Step 211: Define the anomaly categories of the abnormal point clouds as follows:

[0046] No abnormalities;

[0047] Handleable anomalies: point cloud anomalies caused by rain, snow, and fog;

[0048] Unmanageable anomalies: Point cloud anomalies caused by hardware failure (e.g., LiDAR malfunction prevents point cloud acquisition);

[0049] Note: The classification criteria for anomalies can be changed according to the actual situation, but each category should have unique features to ensure that the deep learning model can classify them.

[0050] Step 212: After the anomaly category is determined, the normal point cloud, the point cloud under rainy weather conditions, the point cloud under snowy weather conditions, and the point cloud under foggy weather conditions are obtained based on the Carla autonomous driving simulation platform.

[0051] Point clouds under hardware failure conditions are obtained by randomly removing P% of the point clouds from the normal point cloud.

[0052] 30≤P≤90;

[0053] The simulation platform obtains normal point clouds in the same scene as simulated point clouds without anomalies.

[0054] The simulation platform obtains point clouds under rainy weather conditions, snowy weather conditions, foggy weather conditions, and hardware failure conditions under the same scene as simulated abnormal point clouds.

[0055] The specific process is as follows:

[0056] Build an autonomous driving scenario in the Carla simulation platform and complete the configuration of the vehicle and sensors (LiDAR);

[0057] The Carla simulation platform supports simulations of rainy, snowy, and foggy days, thus enabling the acquisition of normal point clouds and point clouds under three different weather conditions within the same scene.

[0058] Based on point clouds under normal weather conditions, abnormal point clouds under hardware failure conditions are simulated by randomly removing a certain number of point clouds.

[0059] The normal point cloud corresponding to the abnormal point cloud is directly regarded as the label of the deep learning algorithm;

[0060] Step 22: Project the simulated anomaly-free 3D point cloud into a 2D image; divide the 2D image into M windows of the same size; M is a positive integer;

[0061] The 2D images projected from point clouds without anomalies and point clouds with anomalies are the same size. The M windows segmented from point clouds without anomalies and point clouds with anomalies are also the same size and number. The positions of the M windows segmented from point clouds without anomalies and point clouds with anomalies are one-to-one. The first window segmented from point clouds without anomalies is in the same position as the first window segmented from point clouds with anomalies. The second window segmented from point clouds without anomalies is in the same position as the second window segmented from point clouds with anomalies. The Mth window segmented from point clouds without anomalies is in the same position as the Mth window segmented from point clouds with anomalies (the 2D images projected from point clouds without anomalies and point clouds with anomalies are placed overlapping, and the points in the two images are in the same position).

[0062] Steps two and three: Input the simulated anomalous point clouds (which may include point clouds without anomalies) into the deep learning model in batches to obtain the reconstructed noise-free point clouds and anomaly categories; the specific process is as follows:

[0063] Step 231: Project the simulated abnormal 3D point cloud into a 2D image;

[0064] Step 232: Divide the 2D image into M windows of equal size, and use the local point cloud feature extraction module to extract the features of the point cloud in each window in turn; M takes the value of a positive integer;

[0065] Step 233: Input the 2D image (process the entire image at once) into the global point cloud feature extraction module to obtain global point cloud features;

[0066] Steps 2, 3, and 4: Divide the global point cloud features according to the position of each window in the local point cloud features to obtain windows of the global point cloud features that correspond one-to-one with the position of each window in the local point cloud features;

[0067] Steps 2, 3, and 5: Add the point cloud features within each window of the global point cloud features to the point cloud features within the window of the corresponding local point cloud features to obtain the local point cloud features after fusing global information.

[0068] Steps 2, 3, and 6: Input the local point cloud features after fusing global information into the fusion decoding module, and process the point cloud features of each window in turn to obtain the reconstructed noise-free point cloud (point cloud denoising result);

[0069] Steps 2, 3, and 7: Input the global point cloud features sequentially into global average pooling, 1×1 convolution, and softmax to obtain the anomaly categories.

[0070] Other steps and parameters are the same as in specific implementation method one or two.

[0071] Specific Implementation Method Four: This implementation method differs from Specific Implementation Methods One to Three in that: in steps Two to Three, the simulated abnormal 3D point cloud is projected as a 2D image; the specific process is as follows:

[0072] The simulated anomalous 3D point cloud is projected into a 2D image using spherical projection. The transformation formula is as follows:

[0073]

[0074] φ=(1-(arcsin(z×r -1 )+f up )×f -1 )×h

[0075] Where (x, y, z) are the 3D point cloud coordinates, (θ, φ) are the angular coordinates of the projected image, (h, w) are the height and width of the projected image, respectively, and f = f up +f down It is the vertical FOV of the radar, f up It is the field of view above the horizontal direction of the radar, f down It is the field of view below the horizontal direction of the radar. It is the distance of the point cloud;

[0076] The angular coordinates (θ,φ) of each projected image contain four attributes, namely (x,y,z,r);

[0077] The projected image has the shape (h, w, 4), where 4 is the number of input channels.

[0078] The other steps and parameters are the same as those in one of the specific implementation methods one to three.

[0079] Specific Implementation Method Five: This implementation method differs from Specific Implementation Methods One to Four in that: in step two, the multi-task loss function is calculated based on the reconstructed noise-free point cloud and anomaly category; the specific process is as follows:

[0080] The multi-task loss function L consists of point cloud reconstruction loss function L1, anomaly classification loss function L2, and point cloud quality alignment loss function L3;

[0081] L = L1 + L2 + L3

[0082] The point cloud reconstruction loss function L1 is the distance reconstruction loss, which is defined as follows:

[0083]

[0084]

[0085] Where N represents the number of point clouds, and α and λ are two adjustable hyperparameters. r represents the distance to the i-th point within the reconstructed noise-free point cloud window. i This represents the distance to the i-th point within the simulated anomaly-free point cloud window. Middle window and r i The middle window is in the same position. The i-th point within the middle window and r i (The i-th point within the middle window is at the same position); d i Indicates an intermediate variable; i represents the i-th point;

[0086] The aforementioned distance reconstruction loss function is scale invariant, which can mitigate the impact of different distance ground truth scales.

[0087] L2 is an anomaly classification loss function, which is the cross-entropy loss function. Specifically:

[0088] The cross-entropy loss function is calculated based on the point cloud anomaly categories predicted by the deep learning model and the point cloud anomaly category labels (which can be obtained when the anomaly point cloud is acquired).

[0089] The purpose of the point cloud quality alignment loss function L3 is to make the spatial autocorrelation of the reconstructed noiseless point cloud as similar as possible to the spatial autocorrelation of the simulated anomaly-free point cloud, |K A -K B |=0, the spatial autocorrelation of the point cloud is obtained by calculating the correlation between a point and the surrounding point cloud;

[0090] Where K AK represents the spatial autocorrelation of all points within each window of the reconstructed noise-free point cloud. B Each window (K) of the simulated anomaly-free point cloud represents a window of the point cloud. B Middle window and K A The middle window is in the same position, such as K A When each window refers to the first window, K B Take K for each window A Spatial autocorrelation of all points within a window (the window whose position is the same as the first window in the middle).

[0091] The other steps and parameters are the same as those in one of the specific implementation methods one to four.

[0092] Specific Implementation Method Six: This implementation method differs from Specific Implementation Methods One to Five in that: the spatial autocorrelation K of all points within each window of the reconstructed noise-free point cloud... A The calculation formula is as follows:

[0093]

[0094] in This represents the distance to the i-th point within the reconstructed noise-free point cloud window. This represents the distance to the j-th point within the reconstructed noise-free point cloud window, where i represents the i-th point and j represents the j-th point. N is the average distance of all point cloud points within the selected window; A w represents the number of noise-free point clouds reconstructed within the selected window. ij These are predefined weight values. It is the sum of all weights.

[0095] The other steps and parameters are the same as those in one of the specific implementation methods one to five.

[0096] Specific Implementation Method Seven: This implementation method differs from Specific Implementation Methods One through Six in that: the spatial autocorrelation K of all points within each window of the simulated anomaly-free point cloud is... B The calculation formula is as follows:

[0097]

[0098] Where r i r represents the distance to the i-th point within the simulated anomaly-free point cloud window. j This represents the distance to the j-th point within the simulated point cloud window without anomalies, where i represents the i-th point and j represents the j-th point. N is the average distance of all point cloud points within the selected window; B w represents the number of anomaly-free point clouds simulated within the selected window. ij These are predefined weight values.

[0099] The N = N A =N B .

[0100] The other steps and parameters are the same as those in one of the specific implementation methods one to six.

[0101] Specific Implementation Method Eight: This implementation method differs from Specific Implementation Methods One through Seven in that the predefined weight value w... ij The definition is as follows:

[0102]

[0103] The other steps and parameters are the same as those in any of the specific implementation methods one to seven.

[0104] Specific Implementation Method Nine: This implementation method differs from Specific Implementation Methods One to Eight in that the stride of the first 3×3 convolutional layer is 2;

[0105] The stride of the second 3×3 convolutional layer is 2;

[0106] The stride of the third 3×3 convolutional layer is 2;

[0107] The stride of the first 2×2 deconvolution layer is 2;

[0108] The stride of the second 2×2 deconvolution layer is 2;

[0109] The stride of the third 2×2 deconvolution layer is 2.

[0110] The other steps and parameters are the same as those in one of the specific implementation methods one to eight.

[0111] Specific Implementation Method Ten: This implementation method differs from Specific Implementation Methods One through Nine in that: in step four...

[0112]

[0113]

[0114] (x,y,z) are the coordinates of a 3D point.

[0115] The other steps and parameters are the same as those in any of the specific implementation methods one to nine.

[0116] To obtain robust anomaly detection results, a simple post-processing algorithm is proposed. When the point cloud anomaly detection result is no anomaly, it indicates that the current point cloud is at a low level of contamination, and the point cloud anomaly detection result is that there are no anomalies.

[0117] When the point cloud anomaly detection result is an unprocessable anomaly, it indicates that the current point cloud is severely contaminated, and the point cloud result is directly given as completely abnormal.

[0118] When the point cloud anomaly detection result is a processable anomaly, the reconstructed noise-free point cloud is compared with the input point cloud, and the point cloud with the larger difference is the anomalous point cloud.

[0119] If point-by-point anomaly detection results are not required in practical applications or the accuracy of point-by-point anomaly detection cannot meet the requirements, the point cloud can be divided into different regions, and the number of abnormal point clouds falling in that region can be used to determine whether it is an abnormal region.

[0120] This method can also be efficiently extended to 4D point cloud scenes, i.e., continuous 3D point cloud sequences over a period of time. Considering that the input point clouds of adjacent frames have a certain continuity in practical applications, and the weather or surrounding environment will not change drastically within a short period of time, the global point cloud feature extraction network in the above deep learning model does not need to process every frame of input, which can further accelerate the inference speed.

[0121] This invention may have other embodiments. Without departing from the spirit and essence of this invention, those skilled in the art can make various corresponding changes and modifications according to this invention, but these corresponding changes and modifications should all fall within the protection scope of the appended claims.

Claims

1. A deep learning-based method for anomaly detection in point clouds of intelligent vehicle LiDAR, characterized in that: The specific process of the method is as follows: Step 1: Build a deep learning model; Step 2: Input the simulated abnormal point clouds into the deep learning model in batches to obtain the reconstructed noise-free point clouds and abnormal categories; The multi-task loss function is calculated based on the reconstructed noise-free point cloud and anomaly categories until the deep learning model converges, thus obtaining the trained deep learning model. Step 3: Input the point cloud to be tested into the trained deep learning model to obtain the reconstructed noise-free point cloud and anomaly categories; When the anomaly category of the point cloud to be tested is no anomaly, the anomaly detection result of the point cloud to be tested is no anomaly. When the anomaly category of the point cloud to be tested is an unprocessable anomaly, the point cloud to be tested is an anomalous point cloud. When the anomaly category of the point cloud under test is a certain processable anomaly, based on the reconstructed noise-free point cloud, the first... The distance between points and the input point cloud to be measured is the distance between the points and the i-th point. The absolute value of the relative distance difference between the n points is calculated. When the absolute value of the relative distance difference is greater than a threshold, the nth point in the point cloud to be measured is considered an indicative point. The nth point is considered an outlier. When the absolute value of the relative distance difference is less than or equal to a threshold, the nth point in the point cloud to be measured is considered an outlier. The process continues until all points in the test point cloud are determined to be abnormal. The point cloud is obtained from simulated vehicle-mounted lidar sensors; The process of building a deep learning model in step one is as follows: The deep learning model includes a local point cloud feature extraction module, a global point cloud feature extraction module, and a fusion decoding module; The local point cloud feature extraction module includes, in sequence, a first... Convolutional layer, first Convolutional layer, second Convolutional layer, second Convolutional layer, third Convolutional layer, third Convolutional layer; The global point cloud feature extraction module is the encoder of SqueezeSegV2; The fusion decoding module includes, in sequence, a first Deconvolution layer, second Deconvolution layer, third Deconvolutional layer; In step two, the simulated abnormal point clouds are input in batches into the deep learning model to obtain the reconstructed noise-free point clouds and abnormal categories; the specific process is as follows: Step 21: Obtain point cloud pairs, which include simulated point clouds without anomalies and simulated point clouds with anomalies; the specific process is as follows: Step 211: Define the anomaly categories of the abnormal point clouds as follows: No abnormalities; Handleable anomalies: point cloud anomalies caused by rain, snow, and fog; Unmanageable anomalies: Point cloud anomalies caused by hardware failure; Step 212: After the anomaly category is determined, the normal point cloud, the point cloud under rainy weather conditions, the point cloud under snowy weather conditions, and the point cloud under foggy weather conditions are obtained based on the Carla autonomous driving simulation platform. By randomly removing normal point clouds % of the point cloud, to obtain the point cloud under hardware failure conditions; ； The simulation platform obtains normal point clouds in the same scene as simulated point clouds without anomalies. The simulation platform obtains point clouds under rainy weather conditions, snowy weather conditions, foggy weather conditions, and hardware failure conditions under the same scene as simulated abnormal point clouds. Step 22: Project the simulated anomaly-free 3D point cloud into a 2D image; divide the 2D image into M windows of the same size; M is a positive integer; Steps two and three: Input the simulated anomaly point clouds into the deep learning model in batches to obtain the reconstructed noise-free point clouds and anomaly categories; the specific process is as follows: Step 231: Project the simulated abnormal 3D point cloud into a 2D image; Step 232: Divide the 2D image into M windows of equal size, and use the local point cloud feature extraction module to extract the features of the point cloud in each window in turn; M takes the value of a positive integer; Step 233: Input the 2D image into the global point cloud feature extraction module to obtain global point cloud features; Steps 2, 3, and 4: Divide the global point cloud features according to the position of each window in the local point cloud features to obtain windows of the global point cloud features that correspond one-to-one with the position of each window in the local point cloud features; Steps 2, 3, and 5: Add the point cloud features within each window of the global point cloud features to the point cloud features within the window of the corresponding local point cloud features to obtain the local point cloud features after fusing global information. Steps 2, 3, and 6: Input the local point cloud features after fusing global information into the fusion decoding module, and process the point cloud features of each window in turn to obtain the reconstructed noise-free point cloud; Steps 2, 3, and 7: Input the global point cloud features sequentially into global average pooling... Convolution and softmax are used to obtain the anomaly category.

2. The method for anomaly detection of point clouds in intelligent vehicle LiDAR based on deep learning according to claim 1, characterized in that: In steps two and three, the simulated abnormal 3D point cloud is projected into a 2D image; the specific process is as follows: The simulated anomalous 3D point cloud is projected into a 2D image using spherical projection. The transformation formula is as follows: in It is 3D point cloud coordinates. These are the angular coordinates of the projected image. These are the height and width of the projected image, respectively. It is the vertical field of view (FOV) of the radar. It is the field of view above the horizontal direction of the radar. It is the field of view below the horizontal direction of the radar. It is the distance of the point cloud; Angular coordinates of each projected image It contains four attributes, namely ; The shape of the projected image is 4 is the number of input channels.

3. The method for anomaly detection of point clouds in intelligent vehicle LiDAR based on deep learning according to claim 2, characterized in that: In step two, a multi-task loss function is calculated based on the reconstructed noise-free point cloud and anomaly categories; the specific process is as follows: Multi-task loss function Including point cloud reconstruction loss function Anomaly category classification loss function Composed of point cloud quality alignment loss function ; The point cloud reconstruction loss function The distance reconstruction loss is defined as follows: in Indicates the number of point clouds, and It's a hyperparameter. The first point cloud in the window representing the reconstructed noise-free point cloud. The distance between points The simulated point cloud window contains the first point cloud element without anomalies. The distance between points; Indicates intermediate variables; Indicates the first One point; The anomaly classification loss function is the cross-entropy loss function, specifically: The cross-entropy loss function is calculated based on the point cloud anomaly categories predicted by the deep learning model and the point cloud anomaly category labels. Point cloud quality alignment loss function The goal is to make the spatial autocorrelation of the reconstructed noise-free point cloud equal to the spatial autocorrelation of the simulated anomaly-free point cloud. ; in This represents the spatial autocorrelation of all points within each window of the reconstructed noise-free point cloud. This represents the spatial autocorrelation of all points within each window of the simulated anomaly-free point cloud.

4. The method for anomaly detection of point clouds in intelligent vehicle LiDAR based on deep learning according to claim 3, characterized in that: Spatial autocorrelation of all points within each window of the reconstructed noise-free point cloud. The calculation formula is as follows: in The first point cloud in the window representing the reconstructed noise-free point cloud. The distance between points The first point cloud in the window representing the reconstructed noise-free point cloud. The distance between points Indicates the first One point, Indicates the first One point; This is the average distance of all point cloud points within the selected window; The number of noise-free point clouds reconstructed within the selected window. These are predefined weight values. It is the sum of all weights.

5. The method for anomaly detection of point clouds in intelligent vehicle LiDAR based on deep learning according to claim 4, characterized in that: The spatial autocorrelation of all points within each window of the simulated anomaly-free point cloud is analyzed. The calculation formula is as follows: ， in The simulated point cloud window contains the first point cloud element without anomalies. The distance between points The simulated point cloud window contains the first point cloud element without anomalies. The distance between points Indicates the first One point, Indicates the first One point; This is the average distance of all point cloud points within the selected window; This represents the number of anomaly-free point clouds simulated within the selected window. These are predefined weight values. It is the sum of all weights.

6. The method for anomaly detection of point clouds in intelligent vehicle LiDAR based on deep learning according to claim 5, characterized in that: The predefined weight value The definition is as follows: 。 7. The method for anomaly detection of point clouds in intelligent vehicle LiDAR based on deep learning according to claim 6, characterized in that: The first The stride of the convolutional layer is 2; The second The stride of the convolutional layer is 2; The third The stride of the convolutional layer is 2; The first The stride of the deconvolution layer is 2; The second The stride of the deconvolution layer is 2; The third The stride of the deconvolution layer is 2.

8. The method for anomaly detection of point clouds in intelligent vehicle LiDAR based on deep learning according to claim 7, characterized in that: In step four Distance between points ; These are 3D point coordinates.

Images