A point cloud anomaly detection method, system and autonomous vehicle

Abstract

The application provides a point cloud anomaly detection method, system and automatic driving vehicle, and the method comprises the following steps: acquiring point cloud data of a vehicle surrounding environment perceived by a laser radar; inputting the point cloud data of the vehicle surrounding environment into a pre-established and trained point cloud anomaly detection network model to output a point cloud anomaly detection result; the point cloud anomaly detection network model is used for classifying and identifying fusion perception data and performing supervised learning training through a preset loss function; the fusion perception data is data fused after a known scene perceived by the laser radar and a pseudo abnormal object generated based on a point cloud deep learning data set; and the application can realize environment perception of the vehicle surrounding environment, accurately classify and identify unknown abnormal objects, improve the generalization performance of point cloud anomaly detection, and further improve the accuracy of point cloud anomaly detection when applied to the automatic driving vehicle.

Description

Technical Field

[0001] This invention relates to the field of autonomous driving technology, and in particular to a point cloud anomaly detection method, system, and autonomous vehicle. Background Technology

[0002] With the development of artificial intelligence and computer vision technologies, and the increasing emphasis on driving safety in autonomous driving, related technologies have become a research hotspot. Autonomous driving is a technology that uses computer devices to control vehicles to drive automatically on roads. Its realization relies on the collaborative efforts of artificial intelligence, computer vision, radar, and positioning components. Due to the complexity of road conditions and the presence of numerous irregular barriers, vehicles, and other unusual objects, identifying these objects in the vehicle's surroundings and planning routes to avoid them is crucial for autonomous driving.

[0003] Autonomous vehicles utilize 3D obstacle detection technology, employing LiDAR to detect abnormal objects in the vehicle's surroundings. LiDAR emits laser beams and constructs laser point clouds based on the reflected beams from object surfaces, thereby identifying abnormal objects in the environment. However, current technologies identify abnormal objects by comparing them to existing categories of abnormal objects, which has limitations and ultimately affects the accuracy of point cloud anomaly detection. Summary of the Invention

[0004] This invention provides a point cloud anomaly detection method, system, and autonomous vehicle to address the limitations and low accuracy of existing point cloud anomaly detection technologies. When applied to autonomous vehicles, this invention enables environmental perception around the vehicle, accurate classification and identification of unknown abnormal objects, and improves the generalization performance of point cloud anomaly detection, thereby enhancing its accuracy.

[0005] This invention provides a point cloud anomaly detection method, comprising: acquiring point cloud data of the vehicle's surrounding environment perceived by lidar; inputting the point cloud data of the vehicle's surrounding environment into a pre-established and trained point cloud anomaly detection network model, and outputting point cloud anomaly detection results; wherein the point cloud anomaly detection network model is used to classify and identify fused sensing data, and is trained through supervised learning using a preset loss function; wherein the fused sensing data is data obtained by fusing known scenes perceived by lidar and pseudo-anomalies generated based on a point cloud deep learning dataset.

[0006] According to a point cloud anomaly detection method provided by the present invention, before inputting the point cloud data of the vehicle's surrounding environment into a pre-established and trained point cloud anomaly detection network model, the method further includes: training the point cloud anomaly detection network model based on the fused perception data and testing the performance of the point cloud anomaly detection network model.

[0007] According to a point cloud anomaly detection method provided by the present invention, the point cloud anomaly detection network model includes: a known sensing scene acquisition module for acquiring a known scene sensed by a lidar; a pseudo-anomaly generation module for generating pseudo-anomaly objects based on a ShapeNet dataset; a fusion module for fusing the known scene sensed by the lidar and the pseudo-anomaly objects generated based on the ShapeNet dataset to obtain fused sensing data; a feature extraction module for extracting features from the fused sensing data; a point cloud classification module for performing point cloud classification on the features of the fused sensing data to obtain the predicted probability of the known category, and constraining it with a preset penalty loss function; and an anomaly detection module for performing anomaly detection on the features of the fused sensing data to obtain the predicted probability of the anomaly category, and constraining it with a point-by-point rejection loss function.

[0008] According to a point cloud anomaly detection method provided by the present invention, the fusion module is specifically used to: adjust the proportion of the pseudo-anomaly object; and set the pseudo-anomaly object with the adjusted proportion in the known scene to obtain the fused perception data.

[0009] According to the point cloud anomaly detection method provided by the present invention, the point-by-point rejection loss function is:

[0010]

[0011] Among them, l abstain Let m be the pointwise rejection loss function, and S be the total number of data scenarios. m Let n be the set of training datasets, c be the number of point clouds in a single scene, and P be the probability of all classes. y P is the predicted probability of the known category. o α represents the predicted probability of the anomaly category, and α is the penalty value.

[0012] According to the point cloud anomaly detection method provided by the present invention, the preset penalty loss function is a point-by-point penalty loss function:

[0013]

[0014] Among them, l penaltyLet I be the pointwise penalty loss function, max(.) be the function that takes the maximum value, I(.) be the indicator function, c be the defined number of classes, and m be the number of classes. in and m out is the hyperparameter of the penalty loss function.

[0015] According to a point cloud anomaly detection method provided by the present invention, after setting the scaled pseudo-anomaly object in the known scene, the method further includes: performing point cloud sparsity matching between the scaled pseudo-anomaly object and the known scene.

[0016] According to a point cloud anomaly detection method provided by the present invention, before adjusting the scale of the pseudo-anomaly object, the method further includes: performing a state transformation on the pseudo-anomaly object in the known scene, wherein the state transformation is at least one of rotation and movement.

[0017] According to the point cloud anomaly detection method provided by the present invention, the preset penalty loss function is a point-by-point dynamic penalty loss function as follows:

[0018]

[0019] Among them, l dynamic penalty Here, `max(.)` is the pointwise dynamic penalty loss function, `I(.)` is the function that takes the maximum value, `I(.)` is the indicator function, and the expression within the parentheses returns 1 if the expression is true and 0 otherwise. `c` is the number of defined classes, and `m`... in m out and m sout β is a hyperparameter of the penalty loss function. in β rout and β sout is a learnable hyperparameter, and is the coefficient of m.

[0020] This invention also provides a point cloud anomaly detection system, comprising: an acquisition module for acquiring point cloud data of the vehicle's surrounding environment perceived by lidar; a detection module for inputting the point cloud data of the vehicle's surrounding environment into a pre-established and trained point cloud anomaly detection network model, and outputting point cloud anomaly detection results; the point cloud anomaly detection network model is used to classify and identify the fused perception data, and to perform supervised learning training through a preset loss function; the fused perception data is data obtained by fusing the known scene perceived by lidar and pseudo-anomaly objects generated based on a point cloud deep learning dataset.

[0021] The present invention also provides an autonomous vehicle, including the point cloud anomaly detection system described above.

[0022] This invention provides a point cloud anomaly detection method, system, and autonomous vehicle. The method includes: acquiring point cloud data of the vehicle's surrounding environment perceived by LiDAR; inputting the point cloud data of the vehicle's surrounding environment into a pre-established and trained point cloud anomaly detection network model, and outputting point cloud anomaly detection results; the point cloud anomaly detection network model is used to classify and identify fused perception data, and is trained through supervised learning using a preset loss function; the fused perception data is data obtained by fusing the known scene perceived by LiDAR and pseudo-anomalies generated based on a point cloud deep learning dataset. When applied to autonomous vehicles, this invention can achieve environmental perception around the vehicle, accurately classify and identify unknown anomalies, improve the generalization performance of point cloud anomaly detection, and thus improve the accuracy of point cloud anomaly detection. Attached Figure Description

[0023] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0024] Figure 1 This is a flowchart illustrating a point cloud anomaly detection method provided by the present invention;

[0025] Figure 2 This is a schematic diagram illustrating the principle of a point cloud anomaly detection method provided by the present invention;

[0026] Figure 3 This is a schematic diagram of the fusion module provided by the present invention;

[0027] Figure 4 This is a graph showing the relationship between the pointwise penalty loss function and the penalty value provided by this invention;

[0028] Figure 5 This is a schematic diagram of the structure of a point cloud anomaly detection system provided by the present invention;

[0029] Figure 6 This is a schematic diagram of the structure of the electronic device provided by the present invention. Detailed Implementation

[0030] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0031] With the development of vehicle technology and related information electronics technologies, intelligent vehicles, such as autonomous vehicles, have begun to emerge and be applied. Intelligent vehicles are endowed with intelligent and connected characteristics, and their application improves the convenience, comfort, safety, and energy efficiency of people's travel. When driving, intelligent vehicles need to perceive their surrounding environment and control their steering and speed based on the perceived road, vehicle position, and obstacle information, thus enabling them to drive safely and reliably on the road. Current technologies using lidar to detect abnormal objects in the vehicle's surrounding environment, which rely on comparison with existing abnormal object categories, have certain limitations, resulting in low accuracy in point cloud anomaly detection.

[0032] Please refer to Figure 1 , Figure 1 This is a flowchart illustrating a point cloud anomaly detection method provided by the present invention.

[0033] To address the technical problems existing in the prior art, the present invention provides a point cloud anomaly detection method, comprising:

[0034] 101: Acquire point cloud data of the vehicle's surrounding environment as perceived by LiDAR;

[0035] Specifically, LiDAR can be installed on the roof of a vehicle or in other locations to perceive the surrounding environment. The point cloud anomaly detection method operates on electronic devices within the vehicle (such as the vehicle's onboard computer or in-vehicle terminal), which can control the LiDAR via wired or wireless connections. While the vehicle (e.g., an autonomous vehicle) is in motion, the onboard computer or in-vehicle terminal can control the LiDAR to collect laser point clouds of the surrounding environment on the road at a preset sampling frequency, thereby obtaining three-dimensional point cloud data of the surrounding environment.

[0036] It should be noted that the aforementioned wireless connection methods may include, but are not limited to, 5G / 6G connections, WiFi (Wireless Fidelity) connections, Bluetooth connections, WiMAX (Worldwide Interoperability for Microwave Access) connections, Zigbee connections, UWB (Ultra Wide Band) connections, and other currently known or future wireless connection methods.

[0037] A lidar system comprises a transmitter that emits a laser beam and a receiver that receives the reflected beam. During operation, the transmitter emits a laser beam, which is reflected from the surface of an object illuminated by the beam. The receiver collects the reflected laser points from the object's surface. Based on information such as the propagation time of the laser points, the three-dimensional coordinates of each point on the object's surface can be accurately calculated. Since a point cloud is the data of points on the object's surface obtained from the laser points, the type of anomaly can be identified based on the corresponding three-dimensional point cloud. Lidar systems can be configured with 16, 32, or 64 lines, with a higher line count indicating greater energy density and higher accuracy. In this embodiment, the German Velodyne-HDLE64 lidar is used. This 64-line lidar consists of 64 laser photodiodes, detects the surrounding environment at a rotation speed of 600 r / min, has a vertical measurement range of 26° to 5°, and achieves a resolution of 5 cm at a distance of 100 m.

[0038] Among the three-dimensional objects in the vehicle's surrounding environment that can reflect the signals emitted by the lidar, all other categories of objects besides the common categories (predefined categories such as pedestrians and vehicles) are the abnormal objects described in this embodiment of the invention, and these objects are the objects of anomaly detection.

[0039] 102: Input the point cloud data of the vehicle's surrounding environment into the pre-established and trained point cloud anomaly detection network model, and output the point cloud anomaly detection results;

[0040] A point cloud anomaly detection network model is used to classify and identify fused sensing data, and is trained through supervised learning using a preset loss function; the fused sensing data is the data obtained by fusing known scenes perceived by LiDAR and pseudo-anomalies generated based on a point cloud deep learning dataset.

[0041] Specifically, pseudo-anomalies generated based on a point cloud deep learning dataset are used as training samples for the point cloud anomaly detection network model. These training samples can be used to train a point cloud anomaly detection network that uses pseudo-anomalies as input to identify the categories of anomalies, thus obtaining the point cloud anomaly detection network model.

[0042] The point cloud anomaly detection network model here can be, but is not limited to, convolutional neural networks (CNNs). A CNN is a multi-layered neural network adept at handling machine learning problems related to images, especially large images. CNNs have successfully reduced the dimensionality of massive image recognition data through a series of methods, ultimately making it trainable. A typical CNN consists of convolutional layers, pooling layers, and fully connected layers. Convolutional and pooling layers work together to form multiple convolutional groups, extracting features layer by layer, and finally completing classification through several fully connected layers. In summary, CNNs simulate feature differentiation through convolution, and reduce the order of magnitude of network parameters through weight sharing and pooling, ultimately performing tasks such as classification using traditional neural networks.

[0043] By using a trained point cloud anomaly detection network model to identify point cloud data of the vehicle's surrounding environment, the category of abnormal objects, such as people, bicycles, and motor vehicles, can be determined.

[0044] The known scenarios can be divided into ordinary urban roads, intersections, highways, and ramps.

[0045] Based on the laser point cloud obtained by the lidar, each point contains its three-dimensional position coordinates and its reflection intensity. The reflection intensity of the lidar refers to the ratio of the echo power of the received laser point to the emission power of the emitted laser beam, that is, the reflectivity of the laser point.

[0046] Based on the above embodiments:

[0047] Please refer to Figure 2 , Figure 2 This is a schematic diagram illustrating the principle of a point cloud anomaly detection method provided by the present invention.

[0048] Please refer to Figure 3 , Figure 3 This is a schematic diagram of the fusion module provided by the present invention.

[0049] As a preferred embodiment, before inputting the point cloud data of the vehicle's surrounding environment into the pre-established and trained point cloud anomaly detection network model, the method further includes: training the point cloud anomaly detection network model based on the fused perception data and testing the performance of the point cloud anomaly detection network model.

[0050] As a preferred embodiment, the point cloud anomaly detection network model includes: a known perception scene acquisition module for acquiring the known scene perceived by LiDAR; a pseudo-anomaly generation module for generating pseudo-anomaly objects based on the ShapeNet dataset; a fusion module for fusing the known scene perceived by LiDAR and the pseudo-anomaly objects generated based on the ShapeNet dataset to obtain fused perception data; a feature extraction module for extracting features from the fused perception data; a point cloud classification module for performing point cloud classification on the features of the fused perception data to obtain the predicted probability of the known category, and constraining it with a preset penalty loss function; and an anomaly detection module for performing anomaly detection on the features of the fused perception data to obtain the predicted probability of the anomaly category, and constraining it with a point-by-point rejection loss function.

[0051] Specifically, considering the lack of semantically rich features in existing LiDAR point cloud anomaly detection—for example, some furniture on a road is clearly an anomaly, yet existing technologies struggle to identify it—capturing the subtle differences between known and anomalous categories in LiDAR point clouds is crucial for point cloud anomaly detection. In this embodiment, the known scene acquisition module acquires the known scene perceived by the LiDAR. The pseudo-anomaly generation module generates pseudo-anomaly objects based on the ShapeNet dataset. Then, the fusion module fuses the known scene perceived by the LiDAR and the pseudo-anomaly objects generated from the ShapeNet dataset to obtain fused perception data. The feature extraction module then extracts features from the fused perception data. Finally, the point cloud classification module and the anomaly detection module perform classification prediction, whereby the point cloud classification module performs point cloud classification on the features of the fused perception data to obtain the predicted probability P of the known category. y The system employs a pre-defined penalty loss function for constraint. The anomaly detection module performs anomaly detection on the features of the fused sensing data, obtaining the predicted probability P of the anomaly category. o And a point-by-point rejection loss function is used for constraint.

[0052] It's important to note that the ShapeNet dataset is a dataset for 3D shape understanding and analysis, used for computer vision tasks. It's a large-scale, multi-class 3D model dataset containing a vast number of 3D models. The ShapeNet dataset is rich in categories, consisting of 220,000 models divided into 3,135 classes, covering various commonly used object categories in daily life, such as chairs, cars, airplanes, and the human body. Each category has a large number of corresponding 3D models. Each 3D model consists of a set of vertices and faces, and includes the model's geometric information and topological structure. In addition to geometric information, the ShapeNet dataset also provides rich descriptive data for the models, such as annotations for the model's label, class, pose, and dimensions.

[0053] The known scenes used in training the point cloud anomaly detection network model can be obtained based on the SemanticKITTI dataset. The SemanticKITTI dataset is a dataset used for semantic scene understanding of LiDAR sequences. The SemanticKITTI dataset consists of 22 sequences, of which sequences 00 to 10 are used as the training set, sequence 08 is used as the validation set, and sequences 11 to 21 are used as the test set.

[0054] Additionally, known scenes can also be obtained based on the nuScenes dataset. The nuScenes dataset is a large-scale dataset providing data from the entire sensor suite of an autonomous vehicle (6 cameras, 1 LiDAR, 5 radars, GPS, IMU). The nuScenes dataset consists of 1000 scenes, each lasting 20 seconds. For object detection and tracking tasks, 23 object categories are annotated across the entire dataset with accurate 3D bounding boxes at 2Hz. Furthermore, object-level attributes such as visibility, motion, and pose are annotated. For LiDAR semantic segmentation tasks, each LiDAR point in the keyframes of the nuScenes dataset is annotated with 32 possible semantic labels. Therefore, the nuScenes dataset contains 1.4 billion annotated points across 40,000 point clouds and 1000 scenes (850 for training and validation, and 150 for testing), improving the generalization ability of computer vision algorithms.

[0055] In a preferred embodiment, the fusion module is specifically used to: adjust the proportion of the pseudo-abnormal object; and set the pseudo-abnormal object with the adjusted proportion in the known scene to obtain fused perception data.

[0056] To improve the generalization performance and training effectiveness of point cloud anomaly detection network models, data augmentation can be performed on the training point cloud data. Specifically, the training point cloud data is scaled relative to the origin to obtain augmented point cloud data. The scaling rate can be set to [0.8, 1.2], but is not limited to this range and can be set according to the actual situation. For example, furniture can be enlarged or reduced and placed on a road in a driving scene to obtain fused perception data.

[0057] Please refer to Figure 4 , Figure 4 The graph showing the relationship between the pointwise penalty loss function and the penalty value provided by this invention.

[0058] As a preferred embodiment, the pointwise rejection loss function is:

[0059]

[0060] Among them, labstain For pointwise rejection loss function, m is the total number of scenarios in the data, and S m Let n be the set of training datasets, c be the number of point clouds in a single scene, and P be the probability of all classes. y P is the predicted probability of a known class. o α represents the predicted probability of the anomaly category, and α is the penalty value.

[0061] Specifically, the predicted values ​​for known categories are obtained from the point cloud classification module. The anomaly detection module obtains the predicted value of the anomaly category. The predicted value for all categories is then... The probability values ​​for all categories are p y ,p o =P where, P y P is the predicted probability of a known class. o Let ψ(.) be the predicted probability of the abnormal category, f(.) be the feature extractor, g(.) be the known category predictor, g(.) be the abnormal category predictor, c be the number of defined categories, such as car, person, bicycle, etc., c+1 be the c+1th category, representing the label value of the abnormal category. In addition, 1 to c are the labels of the normal category.

[0062] It should be noted that: The α value represents the point-by-point penalty that controls whether the tendency to abstain is present. Since this value is defined in terms of points, although point clouds lack the semantically rich features of anomaly detection, they have the potential to capture subtle differences between known and anomalous categories.

[0063] In conjunction with the pointwise rejection loss function, a preset penalty loss function is introduced. As a preferred embodiment, the preset penalty loss function is the pointwise penalty loss function:

[0064]

[0065] Among them, l penalty The loss function is the pointwise penalty function, max(.) is the function that takes the maximum value, I(.) is the indicator function, c is the number of defined classes, and m in and m out is the hyperparameter of the penalty loss function.

[0066] When y i ≠c+1, α i Greater than m in When, max(α) i -m in ,0)=α i -m in When α iLess than m in When, max(α) i -m in ,0)=0, indicating that α has the normal category. i The smaller the better.

[0067] When y i =c+1, α i Greater than m out When, max(m) out -α i ,0)=0; when α i Less than m out When, max(m) out -α i ,0)=m out -α i This indicates that α has an anomalous category. i The bigger the better.

[0068] When setting m in The value is -12, m out When the value is -6, the penalty term is negative.

[0069] In conclusion, Where, λ abstain For l abstain hyperparameters, λ penalty For l penalty Hyperparameters.

[0070] As a preferred embodiment, after setting the scaled pseudo-anomaly object in the known scene, the method further includes: performing point cloud sparsity matching between the scaled pseudo-anomaly object and the known scene.

[0071] To further improve the accuracy of point cloud anomaly detection, this implementation performs point cloud sparsity matching between the scaled-adjusted pseudo-anomalies and the known scene. Specifically, this is achieved by changing the radius of certain scene points in polar coordinates and merging randomly selected ShapeNet objects into the road scene. By replacing the radius, the sampling pattern of the real point cloud is preserved. By merging semantically rich anomalies, the lack of semantically rich features in the point cloud is compensated for, thereby reducing the sparsity difference between the pseudo-anomalies and the known scene's point cloud. This allows the point cloud anomaly detection network to perform point cloud anomaly detection based on rich semantic information, reducing detection errors and achieving higher detection accuracy.

[0072] As a preferred embodiment, before adjusting the scale of the pseudo-anomaly, the method further includes: performing a state transformation on the pseudo-anomaly in a known scene, wherein the state transformation is at least one of rotation and movement.

[0073] To improve the generalization performance and training effectiveness of object detection models, data augmentation can be performed on the training point cloud data. Data augmentation methods include at least one of global rotation and movement. Specifically, the training point cloud data is rotated and moved along a preset direction to obtain augmented point cloud data. This augmented point cloud data is also used as training point cloud data. The rotation angle and movement range of the training point cloud data can be set to [-π / 2, π / 2] and [-1.5m, 1.5m] respectively, but are not limited to these ranges and can be set according to actual conditions. For example, a pseudo-anomaly object can be globally rotated, then moved along the x-axis, then rotated horizontally, its size adjusted, and then placed on the ground. Finally, the pseudo-anomaly object is matched with the known scene using point cloud sparsity matching to merge it into the known scene.

[0074] As a preferred embodiment, the preset penalty loss function is a point-by-point dynamic penalty loss function:

[0075]

[0076] Among them, l dynamic penalty The function is a pointwise dynamic penalty loss function, max(.) is the function that takes the maximum value, I(.) is the indicator function, the parentheses contain a conditional expression that returns 1 if the expression is true, otherwise returns 0, c is the number of defined classes, and m in m out and m sout β is a hyperparameter of the penalty loss function. in β rout and β sout is a learnable hyperparameter, and is the coefficient of m.

[0077] In conclusion, Where, λ dynamic penalty For l dynamic penalty Hyperparameters.

[0078] Table 1 shows the ablation experiments of the pseudo-anomaly generation method on the SemanticKITTI dataset. AUPR represents the area under the precision-recall curve, AUROC represents the area under the receiver operating feature curve, and IoU represents the intersection-over-union ratio. Specifically, AUPR and AUROC measure the performance of anomaly detection, while IoU evaluates the performance of the original classification model. It can be observed that our ShapeNet pseudo-anomaly synthesis method significantly improves the model's ability to identify anomalies.

[0079] Table 1 Ablation Experiment Results

[0080] ShapeNet pseudo-anomaly synthesis AUPR AUROC mIoU × 41.82(1.87↓) 93.04(0.53↑) 57.30(0.17↓) √ 43.69 92.51 57.47

[0081] Please refer to Figure 5 , Figure 5 This is a schematic diagram of the structure of a point cloud anomaly detection system provided by the present invention.

[0082] This invention also provides a point cloud anomaly detection system, comprising: an acquisition module 1 for acquiring point cloud data of the vehicle's surrounding environment perceived by a lidar; a detection module 2 for inputting the point cloud data of the vehicle's surrounding environment into a pre-established and trained point cloud anomaly detection network model and outputting point cloud anomaly detection results; the point cloud anomaly detection network model for classifying and identifying the fused sensing data and performing supervised learning training through a preset loss function; the fused sensing data is the data obtained by fusing the known scene perceived by the lidar with pseudo-anomaly objects generated based on a point cloud deep learning dataset.

[0083] For an introduction to the point cloud anomaly detection system provided by the present invention, please refer to the above method embodiments; the present invention will not be described in detail here.

[0084] The present invention also provides an autonomous vehicle, including the point cloud anomaly detection system described above.

[0085] For an introduction to the autonomous vehicle provided by this invention, please refer to the above method embodiments; the invention itself will not be described in detail here.

[0086] Figure 6 An example is a schematic diagram of the physical structure of an electronic device, such as... Figure 6 As shown, the electronic device may include: a processor 601, a communication interface 602, a memory 603, and a communication bus 604. The processor 601, communication interface 602, and memory 603 communicate with each other via the communication bus 604. The processor 601 can call logical instructions in the memory 603 to execute a point cloud anomaly detection method. This method includes: acquiring point cloud data of the vehicle's surrounding environment perceived by the LiDAR; inputting the point cloud data of the vehicle's surrounding environment into a pre-established and trained point cloud anomaly detection network model, and outputting point cloud anomaly detection results; the point cloud anomaly detection network model is used to classify and identify the fused perception data, and is trained through supervised learning using a preset loss function; the fused perception data is the data obtained by fusing the known scene perceived by the LiDAR and pseudo-anomaly objects generated based on a point cloud deep learning dataset.

[0087] Furthermore, the logical instructions in the aforementioned memory 603 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0088] On the other hand, the present invention also provides a computer program product, which includes a computer program that can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the point cloud anomaly detection method provided by the above methods. The method includes: acquiring point cloud data of the vehicle's surrounding environment perceived by LiDAR; inputting the point cloud data of the vehicle's surrounding environment into a pre-established and trained point cloud anomaly detection network model, and outputting point cloud anomaly detection results; the point cloud anomaly detection network model is used to classify and identify the fused perception data, and is trained through supervised learning using a preset loss function; the fused perception data is the data after fusing the known scene perceived by LiDAR and pseudo-anomalies generated based on a point cloud deep learning dataset.

[0089] In another aspect, the present invention also provides a non-transitory computer-readable storage medium storing a computer program thereon. When executed by a processor, the computer program is implemented to perform the point cloud anomaly detection method provided by the above methods. The method includes: acquiring point cloud data of the vehicle's surrounding environment perceived by lidar; inputting the point cloud data of the vehicle's surrounding environment into a pre-established and trained point cloud anomaly detection network model, and outputting point cloud anomaly detection results; the point cloud anomaly detection network model is used to classify and identify the fused sensing data, and is trained through supervised learning using a preset loss function; the fused sensing data is the data obtained by fusing the known scene perceived by lidar and pseudo-anomalies generated based on a point cloud deep learning dataset.

[0090] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0091] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0092] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A point cloud anomaly detection method, characterized in that, include: Acquire point cloud data of the vehicle's surrounding environment as perceived by lidar; The point cloud data of the vehicle's surrounding environment is input into a pre-established and trained point cloud anomaly detection network model, and the point cloud anomaly detection results are output. The point cloud anomaly detection network model is used to classify and identify fused sensing data, and is trained through supervised learning using a preset loss function. The fused sensing data is the data obtained by fusing the known scene sensed by lidar with the pseudo-anomaly object generated based on the point cloud deep learning dataset; The point cloud anomaly detection network model includes: The known scene acquisition module is used to acquire the known scene sensed by the lidar; The pseudo-anomaly generation module is used to generate pseudo-anomaly objects based on the ShapeNet dataset. The fusion module is used to fuse the known scene sensed by the lidar and the pseudo-anomaly object generated based on the ShapeNet dataset to obtain the fused sensing data. The feature extraction module is used to extract features from the fused sensing data; The point cloud classification module is used to classify the features of the fused sensing data into point clouds, obtain the predicted probability of the known categories, and constrain them using a preset penalty loss function. An anomaly detection module is used to perform anomaly detection on the features of the fused sensing data, obtain the predicted probability of the anomaly category, and use a pointwise rejection loss function for constraint. The pointwise rejection loss function is: ， in, Let the pointwise rejection loss function be... m The total number of data scenarios. S m For the collection of training datasets, n The number of point clouds in a single scene. c Let P be the number of defined categories, and P be the probability of all categories. The predicted probability of the known category. The predicted probability for the anomaly category. This is the penalty value.

2. The point cloud anomaly detection method according to claim 1, characterized in that, Before inputting the point cloud data of the vehicle's surrounding environment into the pre-established and trained point cloud anomaly detection network model, the process also includes: The point cloud anomaly detection network model is trained and its performance is tested based on the fused sensing data.

3. The point cloud anomaly detection method according to claim 1, characterized in that, The fusion module is specifically used for: Adjust the scale of the pseudo-anomaly object; The pseudo-abnormal object, after being proportionally adjusted, is placed in the known scene to obtain the fused perception data.

4. The point cloud anomaly detection method according to claim 3, characterized in that, The preset penalty loss function is a pointwise penalty loss function: ， in, Let the pointwise penalty loss function be... For the function that takes the maximum value, For indicator functions, c For the number of defined categories, m in and m out is the hyperparameter of the penalty loss function.

5. The point cloud anomaly detection method according to claim 3, characterized in that, After setting the scaled-adjusted pseudo-anomaly object in the known scene, the process further includes: The pseudo-anomaly object with the adjusted scale is then matched with the known scene using point cloud sparsity matching.

6. The point cloud anomaly detection method according to claim 5, characterized in that, Before adjusting the scale of the pseudo-anomaly object, the method further includes: The pseudo-anomaly object undergoes a state transformation within the known scene, and this state transformation is at least one of rotation and movement.

7. The point cloud anomaly detection method according to claim 6, characterized in that, The preset penalty loss function is a point-by-point dynamic penalty loss function, which is: ， in, Let the pointwise dynamic penalty loss function be... For the function that takes the maximum value, This is an indicator function; the parentheses contain a conditional expression. It returns 1 if the expression is true, and 0 otherwise. c For the number of defined categories, m in , m out and m sout For the hyperparameters of the penalty loss function; , and For learnable hyperparameters, is m The coefficient.

8. A point cloud anomaly detection system, characterized in that, include: The acquisition module is used to acquire point cloud data of the vehicle's surrounding environment as perceived by the lidar. The detection module is used to input the point cloud data of the vehicle's surrounding environment into a pre-established and trained point cloud anomaly detection network model, and output the point cloud anomaly detection results. The point cloud anomaly detection network model is used to classify and identify fused sensing data, and is trained through supervised learning using a preset loss function. The fused sensing data is the data obtained by fusing the known scene sensed by lidar with the pseudo-anomaly object generated based on the point cloud deep learning dataset; The point cloud anomaly detection network model includes: The known scene acquisition module is used to acquire the known scene sensed by the lidar; The pseudo-anomaly generation module is used to generate pseudo-anomaly objects based on the ShapeNet dataset. The fusion module is used to fuse the known scene sensed by the lidar and the pseudo-anomaly object generated based on the ShapeNet dataset to obtain the fused sensing data. The feature extraction module is used to extract features from the fused sensing data; The point cloud classification module is used to classify the features of the fused sensing data into point clouds, obtain the predicted probability of the known categories, and constrain them using a preset penalty loss function. An anomaly detection module is used to perform anomaly detection on the features of the fused sensing data, obtain the predicted probability of the anomaly category, and use a pointwise rejection loss function for constraint. The pointwise rejection loss function is: ， in, Let the pointwise rejection loss function be... m The total number of data scenarios. S m For the collection of training datasets, n The number of point clouds in a single scene. c Let P be the number of defined categories, and P be the probability of all categories. The predicted probability of the known category. The predicted probability for the anomaly category. This is the penalty value.

9. An autonomous vehicle, characterized in that, Including the point cloud anomaly detection system as described in claim 8.

Images