A method for identifying fake point cloud objects in autonomous driving systems
By dividing the point cloud space in an autonomous driving system and using PointNet or DGCNN models to identify local components and eliminate fake objects, the problem of universal defense against point cloud forgery attacks is solved, and the detection accuracy and robustness are improved.
Patent Information
- Application Number
- CN202310998755.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2023-08-09
- Publication Date
- 2026-01-06
- Estimated Expiration
- 2043-08-09
AI Technical Summary
Existing autonomous driving systems lack universal defense methods against point cloud spoofing attacks, leading to false detections of fake objects and causing security issues.
By dividing the point cloud input space and using a recognition model to identify local components, fake objects in the point cloud entity detection model are eliminated. Point cloud classification models with PointNet or DGCNN structures are trained to identify real object components, and the authenticity of objects is judged through crowdsourcing.
It effectively defends against various point cloud entity forgery attacks, improves the detection accuracy and robustness of the point cloud entity detection model, reduces the attack success rate, and does not affect the normal working process of the original model.
Smart Images
Figure CN116977967B_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the field of artificial intelligence model security technology, specifically relating to a method for identifying fake point cloud objects in autonomous driving systems. Background Technology
[0002] With the continuous development of deep learning technology, the application of deep learning models has become widespread in real life, and integrating deep learning models into autonomous driving systems has gradually become a mainstream trend in the industry. Among these, LiDAR-generated point cloud data, due to its high accuracy and rich spatial information, is the primary sensor for most autonomous driving systems. Point cloud entity detection models, based on LiDAR-generated point cloud data, have become the core models in autonomous driving systems. However, attackers can use laser emitters to inject fake points into the point cloud data generated by the vehicle-mounted LiDAR, causing the point cloud entity detection model within the autonomous driving system to detect non-existent objects. This can lead to the autonomous vehicle mistakenly stopping or braking suddenly, causing more serious real-world safety problems. Existing defense methods are generally only used for noise reduction or to defend against specific point cloud entity spoofing attacks, lacking universality and still vulnerable to adaptive attacks. Summary of the Invention
[0003] This invention proposes a method for identifying fake point cloud objects in autonomous driving systems in response to point cloud entity forgery attacks, thereby achieving effective defense against various point cloud entity forgery attacks.
[0004] Autonomous driving systems deploy a point cloud-based entity detection neural network model, referred to as the target model. Based on point cloud data generated by the vehicle's LiDAR, the target model perceives various objects in the environment and sends the results to subsequent modules for path planning and obstacle avoidance. In this scenario, an attacker can use a laser emitter to fire a laser at the vehicle's LiDAR sensor, injecting specially designed fake points into the LiDAR's detection point cloud data. This causes the target model, which perceives the environment based on this point cloud data, to detect objects that do not exist in the real scene—i.e., fake objects.
[0005] This invention leverages the common characteristics of existing point cloud entity forgery attacks in terms of attack capability and target: forged point clouds contain fewer effective local components and their local components differ significantly from those of real objects. By dividing the point cloud input space and using a recognition model to identify local components, forged objects detected by the point cloud entity detection model are eliminated. This invention can be integrated with point cloud entity detection models without modifying their structure or workflow, thus exhibiting good algorithm versatility. Furthermore, unlike other defense methods, this invention provides effective defense against various existing point cloud entity forgery attacks, demonstrating good algorithm robustness.
[0006] The proposed method for identifying fake point cloud objects in autonomous driving systems is described in the following article. Figure 1 As shown, it includes: the recognition model preparation stage, the detection result segmentation stage, and the recognition model detection stage;
[0007] In the model preparation stage, the point cloud data used to train the point cloud entity detection model is segmented to generate local point cloud samples, and these samples are used to train a point cloud recognition model that identifies whether local components belong to real entities.
[0008] In the detection result segmentation stage, the input point cloud data and output object detection results processed by the point cloud entity detection model are collected. The input point cloud data is divided into several equal-sized local regions according to spatial location. Based on the IoU (Intersection over Union) between these local regions and the object detection results, i.e. the ratio of the intersection to the union of the two regions, it is determined which local component of the detected object corresponds to each local region.
[0009] In the recognition model detection stage, the recognition model is used to identify the point cloud data contained in all local regions of the corresponding local components of the detected object, to determine whether each local region corresponds to a local component of the real object, and to determine whether each detected object is a real object through crowdsourcing.
[0010] The method for identifying fake point cloud objects in autonomous driving systems proposed in this invention comprises the following steps:
[0011] Step 1: Model Preparation: Train a point cloud recognition model to identify components of real objects. The specific process is as follows:
[0012] (1.1) First, the prepared point cloud entity detection dataset is divided into several L×L local regions at equal intervals along the x-axis and y-axis. The point data contained in each local region is recorded, and each local region is labeled according to the IoU between the local region and the real object bounding box: if the IoU is greater than the preset IoU threshold, the local region is labeled as a real object component; otherwise, the local region is labeled as a fake object component.
[0013] The preferred preset IoU threshold in this invention is ∈=1e -3 See the values in the examples;
[0014] (1.2) Then, a point cloud classification model with PointNet structure [1] or DGCNN structure [2] is built as a recognition model. The above-mentioned re-labeled samples are used as training data and cross-entropy loss is used as the model loss function to train a recognition model for recognizing local components of real objects.
[0015] Step 2, Detection Result Segmentation: Segment and match the input point cloud data and entity detection results of the point cloud entity detection model. The specific process is as follows:
[0016] (2.1) The point cloud entity detection model works normally, predicting the category of all entities contained in the input point cloud data as well as the position and size of the entity bounding boxes;
[0017] (2.2) Cut the input point cloud data into L×L equal-sized local regions at equal intervals along the x-axis and y-axis, and record the point data contained in each local region;
[0018] (2.3) Calculate the IoU between each local region and the bounding box of each detected entity. If the IoU is greater than the preset IoU threshold, then record the local region as the local component to be identified of the detected entity.
[0019] Step 3: Recognition Model Detection: Use the recognition model to identify whether each component to be identified is a component of a real object, and distinguish between real and fake objects through crowdsourcing. The specific process is as follows:
[0020] (3.1) First, use the recognition model trained in step one to identify all local regions that are recorded as local components to be detected, and record the recognition results;
[0021] (3.2) Then, for each detected object, the proportion of its local components that are identified as real object components is calculated. If the proportion is greater than the preset proportion threshold B, the detected object is identified as a real object and retained in the detection results of the point cloud entity detection model; otherwise, the detected object is identified as a fake object and removed from the detection results of the point cloud entity detection model.
[0022] The preset ratio threshold B used in this invention is preferably 0.5, as shown in the examples.
[0023] In this invention, the point cloud entity forgery attack refers to an attacker injecting a cluster of points with specifically designed coordinates and other features into a location in the input space corresponding to point cloud data that does not contain any objects. This causes the point cloud entity detection model to detect a non-existent object of the category specified by the attacker at that location. In a real-world scenario, this type of attack first uses a broadcast diode to detect the laser emitted by the LiDAR deployed on an autonomous vehicle to determine the specific attack time; then, a laser emitter is used to emit a laser at a specific angle and intensity towards the target LiDAR, thereby injecting forged points into the point cloud data generated by the target LiDAR.
[0024] In this invention, the universality of the recognition method refers to the fact that since this recognition method only requires the detection results of the point cloud entity detection model and the point cloud input data as support, it is highly decoupled from the design framework and specific parameters of the point cloud entity detection model. Connecting this recognition method with any point cloud entity detection model does not require fine-tuning or retraining of the corresponding model, nor does it affect the original normal working process of the corresponding model.
[0025] In this invention, the robustness of the identification method means that when an attacker uses several commonly used point cloud object forgery attack methods to attack any point cloud entity detection model that deploys this identification method, the attack effect will be significantly reduced, and in some scenarios, it can even achieve complete defense against the corresponding attacks.
[0026] This invention addresses the limitations of existing point cloud-based entity forgery attacks, which have a limited number of forged points and do not prioritize local similarity. It uses a self-trained point cloud recognition model to detect whether each local region of an object detected by an entity detection model contains components of a real object. Finally, it determines whether each detected object is a real object by crowdsourcing the recognition results.
[0027] The forged object recognition method of this invention is independent of the specific structural design and parameter values of the point cloud entity detection model, and can be interfaced with commonly used high-quality point cloud entity detection models such as PointPillars, Point R-CNN, and PV R-CNN at a lower cost. After introducing this invention, the point cloud entity detection model in the autonomous driving system outperforms the same structure model without any defense methods or with other existing point cloud defense methods in terms of detection accuracy and robustness. In simulation scenarios built using simulators, this invention can effectively defend against forged object attacks that are closer to real-world scenarios, and will only bring limited time and space overhead to the autonomous driving system, ensuring that the autonomous driving system as a whole can still operate normally and efficiently.
[0028] The model defense method of this invention, being a post-processing approach, is largely unaffected by the point cloud entity detection model during the recognition process. Therefore, it is decoupled from the specific structure of the detection model and can be widely applied to various point cloud entity detection models with different architectures, such as PointPillars, Point R-CNN, and PV R-CNN. Furthermore, the model defense method of this invention exhibits good robustness and effectiveness against existing point cloud entity forgery attacks and corresponding adaptive modification attacks, providing an effective tool for the safety protection of autonomous driving systems.
[0029] References
[0030] [1]QI, Charles R., et al. Pointnet: Deep learning on point sets for 3d classification and segmentation. In: Proceedings of the IEEE conference on computer vision and pattern recognition. 2017.p.652-660.
[0031] [2]WANG, Yue, et al.Dynamic graph cnn for learning on point clouds.ACMTransactions on Graphics(tog),2019,38.5:1-12. Attached Figure Description
[0032] Figure 1 This is a schematic diagram of the method flow of the present invention.
[0033] Figure 2 This is a schematic diagram comparing the defense effects of an example.
[0034] Figure 3 This is a schematic diagram illustrating the working effect of the present invention in a real system. Detailed Implementation
[0035] This embodiment focuses on point cloud-based entity detection tasks, using the KITTI dataset. The method described in this embodiment is employed to harden the point cloud entity detection models PointPillars, Point RCNN, and PV RCNN used in the perception module of autonomous driving systems, removing falsely detected objects, especially falsely detected vehicles, caused by malicious attacks or sampling noise during the model detection process. Figure 1 As shown, this embodiment specifically includes:
[0036] Step 1: Model preparation: Train a point cloud recognition model to identify components of real objects.
[0037] (1.1) First, all samples in the KITTI dataset are divided into several L×L local regions at equal intervals along the x and y axes. The point data contained in each local region are recorded, and each local region is labeled according to the IoU between the local region and the bounding box of the real object: if the IoU is greater than a preset IoU threshold ∈ , then the local region is labeled as a real object component; otherwise, the local region is labeled as a fake object component. The preset IoU threshold ∈ used in this invention is 1e -3 In this embodiment, L = 1m is set. Furthermore, since the number of local regions labeled as fake object components in the KITTI dataset is significantly greater than the number of local regions labeled as real object components, uniform random sampling of the local regions labeled as fake object components is used to ensure that the ratio of samples labeled as real object components to those labeled as fake object components in the processed dataset is 1:1.
[0038] (1.2) Then, a point cloud classification model with PointNet or DGCNN structure is built as a recognition model. The re-labeled samples mentioned above are used as training data, and cross-entropy loss is used as the model loss function to train a recognition model for recognizing local components of real objects.
[0039] Step 2, Detection Result Segmentation: Segment and match the input point cloud data and entity detection results of the point cloud entity detection model.
[0040] (2.1) The point cloud entity detection model works normally, predicting the category of all entities contained in the input point cloud data as well as the position and size of the entity bounding boxes.
[0041] (2.2) Cut the input point cloud data into L×L equal-sized local regions at equal intervals along the x-axis and y-axis, and record the point data contained in each local region.
[0042] (2.3) Calculate the IoU between each local region and the bounding box of each detected entity. If the IoU is greater than the preset IoU threshold, then record the local region as the local component to be identified of the detected entity.
[0043] Step 3: Recognition Model Detection: Use the recognition model to identify whether each local component to be identified is a local component of a real object, and distinguish between real objects and fake objects through crowd voting.
[0044] (3.1) First, use the recognition model trained in step one to identify all local regions that are recorded as local components to be detected, and record the recognition results.
[0045] (3.2) Then, for each detected object, the proportion of its local components that are identified as real object components is calculated. If this proportion is greater than a preset proportion threshold B, the detected object is identified as a real object and retained in the detection results of the point cloud entity detection model; otherwise, the detected object is identified as a fake object and removed from the detection results of the point cloud entity detection model. The preset proportion threshold B used in this invention is preferably 0.5.
[0046] Existing point cloud forgery attacks typically involve copying real object point cloud data under specific conditions and adjusting the object's distance, or injecting point clusters into a specific space and then using adversarial attack algorithms to increase the confidence of the forged object corresponding to the point clusters. Previous defense methods relied on removing outliers during preprocessing or removing abnormal detection boxes during post-processing. Figure 2 As shown, the experiment compared the defensive performance of this invention and other previous defense methods against two common point cloud entity spoofing attacks on the PointPillars point cloud entity detection model. The two attack methods in the experiment were: (1) a physical invariance-based attack method, which involves copying the point cloud corresponding to an object located at a distance with a small number of points and spoofing it at a location close to the vehicle-mounted LiDAR; and (2) an optimization-based attack method, which involves directly injecting a fixed number of point clusters at a location close to the vehicle-mounted LiDAR and then using an adversarial attack algorithm to optimize the confidence of the detected object in the area corresponding to the point clusters to achieve the attack. The comparison of the defensive performance and deployment model prediction accuracy of this invention with other defense methods is as follows:
[0047] Physical:
[0048] SRS (Random Removal Point Strategy): In the best-case scenario, the defense effectiveness is 56.78%, and the deployment model accuracy is 72.35%.
[0049] SOR (Outlier Removal Strategy): In the best-case scenario, the defense effectiveness is 87.44%, and the deployment model accuracy is 77.38%.
[0050] CARLO (a strategy that removes fake objects based on the ratio of actual visible area to predicted visible area): In the best case, the defense effectiveness is 65.08%, and the deployment model accuracy is 71.22%.
[0051] Shadow-Catcher (a strategy for removing fake objects based on object depth and object point density ratio): In the best case, the defense effect is 62.50%, and the deployment model accuracy is 70.92%.
[0052] This invention (using PointNet as the recognition model): In the best case, the defense effect is 83.17%, and the deployment model accuracy is 79.23%.
[0053] This invention (using DGCNN as the recognition model): Under optimal conditions, the defense effect is 85.67%, and the deployment model accuracy is 80.83%;
[0054] Baseline:
[0055] SRS (Random Removal Point Strategy): In the best-case scenario, the defense effectiveness is 45.00%, and the deployment model accuracy is 67.07%.
[0056] SOR (Outlier Removal Strategy): In the best-case scenario, the defense effectiveness is 83.67%, and the deployment model accuracy is 72.30%.
[0057] CARLO (a strategy that removes fake objects based on the ratio of actual visible area to predicted visible area): In the best case, the defense effectiveness is 57.03%, and the deployment model accuracy is 66.00%.
[0058] Shadow-Catcher (a strategy for removing fake objects based on object depth and object point density ratio): In the best case, the defense effect is 59.50%, and the deployment model accuracy is 67.12%.
[0059] This invention (using PointNet as the recognition model): In the best case, the defense effect is 79.61%, and the deployment model accuracy is 76.08%.
[0060] This invention (using DGCNN as the recognition model): Under optimal conditions, the defense effect is 82.97%, and the deployment model accuracy is 78.08%.
[0061] As can be seen, using this invention to defend against point cloud forgery attacks can not only significantly reduce the success rate of point cloud forgery attacks without significantly impairing the detection accuracy of the point cloud entity detection model, but also show better overall performance compared to most previous defense methods.
[0062] like Figure 3 As shown, the present invention was deployed into the Apollo autonomous driving system version 6.0.0, and the Apollo autonomous driving system was connected to the LGSVL autonomous driving simulation simulator for simulation experiments. The experimental results are as follows: the present invention significantly improves the detection accuracy and robustness of the point cloud entity detection model in the autonomous driving system while slightly impairing the working efficiency of the autonomous driving system.
[0063] The model defense method provided by this invention is a post-processing method for points cloud entity detection, and its recognition process is largely unaffected by the specific structure of the detection model. Therefore, it is decoupled from the specific structure of the detection model and can be widely applied to various point cloud entity detection models with different architectures, such as PointPillars, Point R-CNN, and PV R-CNN. Furthermore, the model defense method of this invention exhibits good robustness and effectiveness against existing point cloud entity forgery attacks and corresponding adaptive modification attacks, providing an effective tool for the safety protection of autonomous driving systems.
[0064] In summary, this invention is highly decoupled from the target protection model, allowing it to interface with the point cloud entity detection model in the autonomous driving system without affecting its normal operation. This prevents the perception module of the autonomous driving system from being threatened by object forgery attacks, and the target protection model's performance in the original entity detection task is not negatively affected in any way; in fact, it may even improve its performance.
Claims
1. A method for identifying a fake point cloud object for an autonomous driving system, wherein, The automatic driving system is deployed with a point cloud-based entity detection neural network model, referred to as a target model; the target model perceives various objects in the environment according to the point cloud data generated by the vehicle-mounted laser radar detection, and sends the results to subsequent modules to further realize the functions of path planning and obstacle avoidance; in this scenario, an attacker can inject specifically designed fake points into the point cloud data detected by the vehicle-mounted laser radar sensor by using a laser emitter to emit laser to the vehicle-mounted laser radar sensor, so that the target model based on the perception of the environment based on these point cloud data detects objects that do not exist in the real scene, namely fake objects; characterized in that: The common characteristics of the existing point cloud entity forgery model in attack ability and attack target are that the effective local components contained in the fake point cloud are less and the difference with the real object local components is obvious, and by dividing the point cloud input space and using the recognition model to recognize the local components, the fake objects detected by the point cloud entity detection model are eliminated; including: identification model preparation stage, detection result division stage, identification model detection stage; wherein: In the identification model preparation stage, the point cloud data used to train the point cloud entity detection model is cut to generate local point cloud samples, and the point cloud recognition model for identifying whether the local component belongs to a real entity is trained using these samples; In the detection result division stage, the input point cloud data processed by the point cloud entity detection model and the object detection results output are collected, the input point cloud data is cut into several equal local regions according to the spatial position, and according to the IoU size of these local regions and the object detection results, that is, the ratio of the intersection and union of the two regions, it is determined which local component of each detection object corresponds to each local region; In the identification model detection stage, the point cloud data contained in all local regions corresponding to the local components of the detected objects is identified respectively using the identification model, to determine whether each local region corresponds to the local component of a real object, and whether each detected object is a real object is determined through crowd voting.
2. The counterfeit point cloud object identification method of claim 1, wherein, The specific steps are: Step one, identification model preparation: train a point cloud recognition model for identifying real object components; the specific process is: (1.1) First, the prepared point cloud entity detection dataset is cut into several local regions of equal size along the x and y axes, and the point data contained in each local region is recorded and labeled according to the IoU size of the local region with the real object bounding box: if the IoU is greater than a preset IoU threshold, the local region is labeled as a real object component, otherwise the local region is labeled as a fake object component; L L (1.2) Then, build a point cloud classification model with a PointNet structure or a DGCNN structure as an identification model, use the above re-labeled samples as training data, and use cross-entropy loss as the model loss function, train an identification model for identifying real object local components; Step two, detection result division: cut and match the input point cloud data and entity detection results of the point cloud entity detection model, the specific process is: (2.1) The point cloud entity detection model works normally, and predicts the classes of all entities contained in the input point cloud data and the positions and sizes of the entity bounding boxes; (2.2) cutting the input point cloud data according to the x-axis and the y-axis to generate L L equal local regions, and recording the point data contained in each local region; (2.3) Calculate the IoU of each local region and each detected entity bounding box, if the IoU is greater than the preset IoU threshold, record the local region as the local component to be identified of the detected entity; Step three, identification model detection: use the identification model to identify whether the to-be-identified local component is a local component of a real object, and distinguish between real objects and fake objects through crowd voting, and the specific process is as follows: (3.1) First, use the identification model trained in step one to identify all local regions recorded as detection local components, and record the identification results; (3.2) Then, for each detection object, count the proportion of local components identified as real object components; If the proportion is greater than a preset proportion threshold, the detection object is identified as a real object and retained in the detection results of the point cloud entity detection model; Otherwise, the detection object is identified as a fake object and excluded from the detection results of the point cloud entity detection model.