A Method and Apparatus for Generating 3D Point Cloud Adversarial Examples Based on Dynamic Proxy Gradients

Abstract

This invention discloses a method and apparatus for generating adversarial examples from 3D point clouds based on dynamic proxy gradients. The method includes: inputting the acquired full-scene point cloud into a 3D target detection model for target detection, and obtaining a predicted bounding box of the attackable object based on the detection result; obtaining the target object point cloud based on the predicted bounding box; forming an optimization rule by maximizing the detection loss of the 3D target detection model for the attack target and constraining the L2 norm of the perturbation to not exceed a preset budget; generating an adversarial scene, dynamic point voxel mapping, voxel gradient transformation and fusion, and updating the perturbation under constraints through multiple rounds of perturbation superposition according to the optimization rule, obtaining the final adversarial target point cloud and adversarial scene point cloud for scoring, screening high-value adversarial examples and merging them with the original clean examples to obtain the final adversarial training dataset; thereby generating an adversarial point cloud with a high attack success rate and further improving detection capabilities.

Description

Technical Field

[0001] This invention relates to the field of deep neural network adversarial attack technology, and in particular to a method for generating adversarial examples of 3D point clouds based on dynamic proxy gradients, a computer-readable storage medium, a computer device, and a device for generating adversarial examples of 3D point clouds based on dynamic proxy gradients. Background Technology

[0002] In related technologies, LiDAR 3D target detection is a key component of intelligent driving environmental perception systems. Its task is to detect targets such as vehicles, pedestrians, cyclists, and traffic obstacles from 3D point clouds and output the target's 3D position, size, orientation, and category. With the development of deep neural networks, 3D detectors such as PointPillars, SECOND, Part-A2, and PV-RCNN have been widely used in intelligent driving perception tasks. These detectors have achieved high detection accuracy on publicly available datasets, but numerous studies have shown that 3D point cloud detection models remain significantly vulnerable to minor perturbations in the input point cloud. Attackers only need to make small coordinate perturbations to the original point cloud to potentially cause missed target detections, decreased confidence levels, or misclassifications, thereby affecting the intelligent driving system's safety assessment of surrounding traffic participants.

[0003] Existing adversarial attack methods for 3D point clouds can be broadly categorized into three types. The first type consists of gradient-based general attack methods, such as FGSM and PGD. These methods calculate the gradient of the loss function with respect to the input point cloud and update the point coordinates along the gradient direction, thereby inducing the model to produce erroneous outputs. While simple to implement, these methods primarily target classification networks or directly differentiable input spaces. When directly applied to 3D object detection tasks, they often require perturbing the entire frame of point clouds, resulting in the meaningless modification of numerous background points. The second type targets point cloud geometry, such as Adv-PC, GeoA3, and HiT-Adv. These methods improve the naturalness and imperceptibility of generated point clouds by introducing autoencoders, geometric regularization, or local structural constraints. However, most of these methods still focus on point cloud classification tasks or are difficult to directly adapt to 3D object detectors that involve complex processes such as voxelization, sparse convolution, and candidate box regression. The third type targets LiDAR-based 3D object detection. These methods attempt to generate perturbations in the point cloud of the entire autonomous driving scenario, causing the object detector to miss detections or degrade performance. However, this type of method still has the following shortcomings.

[0004] First, many existing 3D target detection attack schemes employ full-scene perturbation or large-scale point cloud perturbation. In reality, target detection model predictions are primarily determined by foreground target points, while background points far from the target typically have little impact on the detection results. Directly perturbing the entire frame's point cloud introduces a large amount of redundant perturbation, reducing perturbation budget utilization; furthermore, it can easily cause unnecessary anomalous changes in the spatial range of adversarial examples, which is detrimental to target-level security assessment in real-world intelligent driving scenarios.

[0005] Secondly, modern LiDAR 3D target detectors generally rely on voxelization or columnar voxelization preprocessing. The original point cloud is first divided into regular voxel or columnar regions, and then processed by modules such as voxel feature encoding networks, sparse convolutional networks, and 2D BEV detection heads. The attack perturbation is actually applied to the original point coordinates, but the detector's internal gradient mainly exists in the voxel feature space or detection loss space. Since the allocation of points to voxels is a discrete process, the same point may fall into different voxels after the perturbation, and the position index of points within a voxel may also change. Therefore, the gradient of the detection loss is difficult to directly and stably backpropagate to the coordinates of the attacked point. This is an important reason why existing methods are unstable when attacking voxel-based detectors.

[0006] Furthermore, existing attack methods typically only use adversarial examples to assess detector vulnerabilities, rarely exploring their value in improving detection performance. In fact, adversarial examples that successfully cause target misses often reflect the detector's weak or difficult examples. If high-quality, informative target-level adversarial examples can be selected and used in conjunction with clean examples for model training or fine-tuning, it can help the detector learn more robust target representations, improving the reliability of intelligent driving perception systems in complex scenarios.

[0007] Therefore, there is an urgent need for a technical solution that can focus on successfully detected target objects without performing full-frame redundant perturbations; can establish a stable proxy gradient path from voxel gradient to point-level perturbation for the discrete point-to-voxel mapping problem in voxel detectors; and can further use the generated target-level adversarial examples for adversarial training, thereby simultaneously serving model security assessment and detection performance enhancement. Summary of the Invention

[0008] This invention aims to at least partially solve one of the technical problems in the aforementioned technologies. To this end, one objective of this invention is to propose a method for generating adversarial examples of 3D point clouds based on dynamic proxy gradients. By adapting the voxelization preprocessing logic of target detectors, and under the constraint of imperceptible perturbation budget, this method can efficiently generate highly mobile adversarial point clouds capable of directionally deceiving various voxel-based target detectors, significantly reducing the detector's accuracy in target recognition and localization.

[0009] A second objective of this invention is to provide a computer-readable storage medium.

[0010] The third objective of this invention is to provide a computer device.

[0011] The fourth objective of this invention is to propose a three-dimensional point cloud adversarial sample generation device based on dynamic proxy gradient.

[0012] To achieve the above objectives, a first aspect of this invention proposes a method for generating adversarial examples from 3D point clouds based on dynamic proxy gradients, comprising the following steps: inputting the acquired full-scene point cloud into a 3D target detection model for target detection, outputting detection results, and obtaining a predicted bounding box of an attackable object based on the detection results; extracting all points whose coordinates fall within the predicted bounding box of the attackable object to obtain a target object point cloud; taking maximizing the detection loss of the 3D target detection model for the attack target as the optimization objective, constraining the L2 norm of the perturbation to not exceed a preset budget, forming a constrained optimization rule; generating an adversarial scene, dynamic point voxel mapping, voxel gradient transformation and fusion, and closed-loop optimization of updating the perturbation under constraints based on the constrained optimization rule, the full-scene point cloud, and the target object point cloud through multiple rounds of iterative perturbation superposition, obtaining a final adversarial target point cloud and a final adversarial scene point cloud that conform to the constrained optimization rule; scoring the final adversarial scene point cloud to select high-value adversarial examples; merging the selected high-value adversarial examples with the original clean examples to obtain the final adversarial training dataset.

[0013] The adversarial sample generation method for 3D point clouds based on dynamic proxy gradient according to the present invention has the following advantages: by adapting the voxelization preprocessing logic of the target detector, under the constraint of imperceptible perturbation budget, it can efficiently generate highly mobile adversarial point clouds that can be directed to deceive various voxel-based target detectors, thereby significantly reducing the detector's accuracy in identifying and locating targets.

[0014] In addition, the 3D point cloud adversarial example generation method based on dynamic proxy gradient proposed in the above embodiments of the present invention may also have the following additional technical features: Optionally, obtaining the predicted bounding box of the attackable object based on the detection result includes: matching the predicted bounding box in the detection result with the real bounding box, and taking the predicted bounding box that satisfies the three-dimensional IoU between the predicted bounding box and the real bounding box ≥ a preset intersection-union ratio threshold and the confidence level ≥ a preset confidence level threshold as the attackable object.

[0015] Optionally, extracting all points whose coordinates fall within the predicted bounding box of the attackable object to obtain a target object point cloud includes: performing field-of-view filtering on the full scene point cloud to obtain a filtered point cloud; and extracting all points whose coordinates fall within the predicted bounding box from the filtered point cloud based on the three-dimensional coordinate range of the predicted bounding box of the attackable object to obtain a target object point cloud.

[0016] Optionally, based on the constrained optimization rules, the full-scene point cloud and the target object point cloud undergo closed-loop optimization through multiple rounds of iterative dynamic voxel mapping, voxel gradient transformation, gradient fusion and constrained projection to obtain the final adversarial target point cloud and the final adversarial scene point cloud that conform to the constrained optimization rules. This includes: obtaining the current round perturbation, applying the perturbation to the target object point cloud to generate the adversarial target point cloud, and replacing the corresponding position of the adversarial target point cloud back to the full-scene point cloud to obtain the adversarial scene point cloud; performing voxelization preprocessing on the adversarial scene point cloud that is completely consistent with the detector, recording the voxel index and voxel position index corresponding to each point in the adversarial target point cloud point by point to obtain a dynamic mapping relationship; inputting the adversarial scene point cloud into the detector, calculating the detection loss of the current round forward, obtaining the gradient of the loss with respect to the voxel input features through backpropagation, and extracting the corresponding position from the gradient point by point according to the dynamic mapping relationship. The coordinate dimension gradient components are used to obtain the point-level proxy gradient; target points belonging to the same voxel are grouped according to the voxel index, and the point-level proxy gradients of all points in the group are averaged and aggregated to obtain the voxel group gradient. The point-level proxy gradient and the voxel group gradient are L2 normalized respectively, and then weighted and fused according to the fusion coefficient to obtain the fused gradient; the fused gradient is L2 normalized and superimposed on the current round perturbation to obtain the updated candidate perturbation; the L2 norm of the candidate perturbation is calculated, and if it exceeds the preset budget, it is scaled back to the allowable range by the projection operator to obtain the perturbation for the next round of iteration; it is determined whether the maximum number of iterations has been reached. If not, the next round of iteration begins; if it is reached, the iteration stops to obtain the final optimized perturbation; the final optimized perturbation is applied to the target object point cloud to generate the final adversarial target point cloud, and the final adversarial target point cloud is replaced back to the corresponding position of the full scene point cloud to obtain the final adversarial scene point cloud.

[0017] To achieve the above objectives, a second aspect of the present invention provides a computer-readable storage medium storing a 3D point cloud adversarial example generation program based on dynamic proxy gradients. When executed by a processor, the 3D point cloud adversarial example generation program based on dynamic proxy gradients implements the 3D point cloud adversarial example generation method based on dynamic proxy gradients as described above.

[0018] To achieve the above objectives, a third aspect of the present invention provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, it implements the above-described method for generating adversarial examples of 3D point clouds based on dynamic proxy gradients.

[0019] To achieve the above objectives, a fourth aspect of this invention proposes a 3D point cloud adversarial example generation device based on dynamic proxy gradients, comprising: an attack target determination module, used to input the acquired full-scene point cloud into a 3D target detection model for target detection, outputting detection results, and obtaining a predicted bounding box of the attackable object based on the detection results; a target object point cloud extraction module, used to extract all points whose coordinates fall within the predicted bounding box of the attackable object, to obtain the target object point cloud; and an attack optimization target setting module, used to optimize the detection loss of the 3D target detection model on the attack target by maximizing the detection loss of the attack target, constraining the L2 norm of the perturbation to not exceed a preset budget, and forming... The system comprises: a constraint-based optimization rule; a voxel-aware adversarial perturbation iterative optimization module, used to generate adversarial scenarios, dynamic voxel mapping, voxel gradient transformation and fusion, and closed-loop optimization of perturbation updates under constraints, based on the constraint-based optimization rule, full-scene point cloud, and target object point cloud through multiple rounds of iterative perturbation superposition, to obtain the final adversarial target point cloud and final adversarial scenario point cloud that conform to the constraint-based optimization rule; an adversarial sample quality and value evaluation module, used to score the final adversarial scenario point cloud to screen high-value adversarial samples; and an adversarial training data construction module, used to merge the screened high-value adversarial samples with the original clean samples to obtain the final adversarial training dataset. Attached Figure Description

[0020] Figure 1 This is a flowchart illustrating the method for generating adversarial examples of 3D point clouds based on dynamic proxy gradients according to an embodiment of the present invention. Figure 2 This is a schematic diagram of the overall network structure according to an embodiment of the present invention; Figure 3 This is a block diagram of a 3D point cloud adversarial sample generation device based on dynamic proxy gradient according to an embodiment of the present invention. Detailed Implementation

[0021] Embodiments of the present invention are described in detail below, examples of which are illustrated in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and intended to explain the present invention, and should not be construed as limiting the present invention.

[0022] To better understand the above technical solutions, exemplary embodiments of the present invention will be described in more detail below with reference to the accompanying drawings. Although exemplary embodiments of the present invention are shown in the drawings, it should be understood that the present invention can be implemented in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided to enable a more thorough understanding of the present invention and to fully convey the scope of the invention to those skilled in the art.

[0023] It should be noted that this application uses the LiDAR 3D target detection task in the intelligent driving scenario as an example for illustration, but this application is not limited to this application scenario, and can also be extended to tasks such as robot navigation, vehicle-road cooperative perception, low-speed unmanned vehicle perception, and safety testing platform that rely on 3D point cloud detection models. Figure 2 The overall flowchart of this application is shown.

[0024] To better understand the above technical solutions, the following will provide a detailed explanation of the technical solutions in conjunction with the accompanying drawings and specific implementation methods.

[0025] Figure 1 This is a flowchart illustrating a method for generating adversarial examples from 3D point clouds based on dynamic proxy gradients according to an embodiment of the present invention. Figure 1 As shown, the method for generating 3D point cloud adversarial examples based on dynamic proxy gradients includes the following steps: S101 inputs the acquired full-scene point cloud into the 3D target detection model for target detection, outputs the detection results, and obtains the predicted bounding box of the attackable object based on the detection results.

[0026] As an example, obtaining the predicted bounding box of the attackable object based on the detection result includes: matching the predicted bounding box in the detection result with the ground truth bounding box, and taking the predicted bounding box that satisfies the three-dimensional IoU between the predicted bounding box and the ground truth bounding box ≥ a preset intersection-union ratio threshold and the confidence level ≥ a preset confidence level threshold as the attackable object.

[0027] Specifically, first, a frame of the original LiDAR full-scene point cloud is acquired, denoted as... This full-scene point cloud is typically obtained by scanning with a vehicle-mounted LiDAR, including roads, buildings, background points, and foreground traffic targets such as vehicles, pedestrians, and cyclists. The 3D target detection model to be attacked is loaded, denoted as... ,in, This represents the network parameters of the detection model. In this embodiment, For voxel-type detectors such as SECOND, PointPillars, and PartA PV-RCNN, etc.

[0028] Original full-scene point cloud Input 3D target detection model Then, a set of predicted 3D bounding boxes, category information, and confidence information are obtained. Each prediction frame is denoted as Its confidence level is denoted as The actual bounding box that matches it is denoted as The intersection-union ratio (IU) threshold used to determine whether the predicted bounding box corresponds to the ground truth bounding box is denoted as . In a preferred embodiment, only when the prediction box... With real frame The three-dimensional IoU is not lower than And confidence level Not lower than the threshold Only when the predicted bounding box is displayed is the target considered an attackable object. The significance of this approach is that this application prioritizes attacking real traffic targets that could be correctly identified by the detector, thereby making the generated adversarial examples more reflective of the actual vulnerability of the detection model in intelligent driving safety scenarios.

[0029] After obtaining the attackable target, this application does not indiscriminately perturb the entire scene background point cloud, but instead limits the attack range to the point cloud of the target object. This reduces modifications to irrelevant background areas, allowing the perturbation budget to focus on the foreground target region that truly affects the target detection results, thus avoiding a large amount of redundant perturbation and unnecessary background changes in traditional full-scene perturbation methods.

[0030] S102, extract all points whose coordinates fall within the predicted bounding box of the attackable object to obtain the point cloud of the target object.

[0031] As one embodiment, extracting all points whose coordinates fall within the bounding box of the attackable object to obtain the target object point cloud includes: performing field-of-view filtering on the full scene point cloud to obtain a filtered point cloud; and extracting all points whose coordinates fall within the bounding box from the filtered point cloud based on the three-dimensional coordinate range of the attackable object's bounding box to obtain the target object point cloud.

[0032] Specifically, in determining the attackable prediction box Then, from the original lidar frame The entire point cloud of the frame is read. Considering that 3D object detection tasks typically only evaluate points within the camera's field of view or effective perception range, this embodiment first performs field-of-view filtering on the original point cloud to obtain the filtered point cloud. Then, based on the 3D extent of the target prediction bounding box, from... Extract the points whose coordinates are located inside the 3D bounding box to form the original target object point cloud. The formula for extracting object point clouds is expressed as:

[0033] in, Indicates lidar points, , , These represent the coordinates of the point in three-dimensional space; , , This represents the extent of the target's 3D bounding box along the three coordinate axes. The above formula ensures the accuracy of the extracted point set. It originates from inside the selected traffic target object, rather than from background points or other target points.

[0034] This step is a crucial foundation that distinguishes this application from full-scene attacks. Full-scene attacks create disturbances in irrelevant areas such as roads, buildings, and the ground, while this application only processes the point cloud of the target object. In intelligent driving scenarios, the security risks of detection systems often stem from missed or false detections of key targets such as vehicles and pedestrians. Therefore, object-level extraction can make attack targets more explicit and make subsequent adversarial training samples more suitable as high-value, difficult samples for detectors.

[0035] S103 sets the optimization objective as maximizing the detection loss of the 3D target detection model for the attack target, and constrains the L2 norm of the perturbation to not exceed the preset budget, thus forming a constrained optimization rule.

[0036] It should be noted that the attack objective of this application is to make the detection model more effective under the premise of limited perturbation amplitude. Confrontational scenarios This results in detection failures or a decrease in detection performance. Specifically, it refers to the detection loss used during detector training or inference. As an optimization objective, by increasing this loss, the model is made to misclassify, locate, or predict the confidence level of the target object. The attack optimization problem can be expressed as:

[0037] in, This represents the actual label or detection supervision information corresponding to the target object. This indicates the maximum permissible disturbance budget. This method is used to constrain the overall amplitude of the perturbation and prevent significant and unreasonable deformation of the target point cloud. Since this method only optimizes within the scope of the object point cloud... Therefore, under the same budget, the perturbation can more effectively affect the detector's judgment of the foreground target.

[0038] The key challenge in achieving this optimization objective lies in the perturbation. Defined in the original point coordinate space, most efficient 3D object detectors first voxelize or columnarize the point cloud before performing feature extraction and detection prediction in a structured feature space. The mapping from points to voxels is a discrete operation; a small displacement of the target point may result in it entering different voxels. Therefore, the gradient of the detection loss with respect to voxel features cannot be directly equivalent to the gradient with respect to point coordinates. To address this issue, this application introduces a dynamic surrogate gradient strategy.

[0039] S104. Based on the constrained optimization rules, the adversarial scene is generated by multi-round iterative perturbation superposition of the full scene point cloud and the target object point cloud, dynamic point voxel mapping, voxel gradient transformation and fusion, and closed-loop optimization of updating perturbation under constraints, so as to obtain the final adversarial target point cloud and the final adversarial scene point cloud that conform to the constrained optimization rules.

[0040] As an example, based on constrained optimization rules, the final adversarial target point cloud and the final adversarial scene point cloud are obtained through closed-loop optimization of multi-round iterative dynamic voxel mapping, voxel gradient transformation, gradient fusion, and constrained projection, conforming to the constrained optimization rules. This includes: obtaining the current round perturbation, applying perturbation to the target object point cloud to generate the adversarial target point cloud, and replacing the corresponding positions in the full scene point cloud with the adversarial target point cloud to obtain the adversarial scene point cloud; performing voxelization preprocessing on the adversarial scene point cloud that is completely consistent with the detector, recording the voxel index and intra-voxel position index corresponding to each point in the adversarial target point cloud point by point to obtain the dynamic mapping relationship; inputting the adversarial scene point cloud into the detector, calculating the detection loss for the current round forward, obtaining the gradient of the loss with respect to the voxel input features through backpropagation, and extracting the coordinates of the corresponding positions from the gradient point by point according to the dynamic mapping relationship. The gradient components of the dimension are labeled to obtain the point-level proxy gradient. Target points belonging to the same voxel are grouped according to the voxel index. The point-level proxy gradients of all points in the group are averaged and aggregated to obtain the voxel group gradient. The point-level proxy gradient and the voxel group gradient are L2 normalized separately and then weighted and fused according to the fusion coefficient to obtain the fused gradient. The fused gradient is L2 normalized and superimposed on the current round of perturbation to obtain the updated candidate perturbation. The L2 norm of the candidate perturbation is calculated. If it exceeds the preset budget, it is scaled back to the allowable range using the projection operator to obtain the perturbation for the next round of iteration. It is determined whether the maximum number of iterations has been reached. If not, the next round of iteration begins; if it has been reached, the iteration stops to obtain the final optimized perturbation. The final optimized perturbation is applied to the target object point cloud to generate the final adversarial target point cloud. The final adversarial target point cloud is then replaced with the corresponding position in the full scene point cloud to obtain the final adversarial scene point cloud.

[0041] Specifically, (1) First, object-level perturbation construction and adversarial scenario generation are performed: Obtaining the point cloud of the original target object Then, initialize point-level perturbations of the same dimension. Disturbance Only applies to point clouds of the target object The points in the cloud do not affect the original full-scene point cloud. Background points in the target object point cloud. The target object point cloud is perturbed to obtain the adversarial target point cloud. Then Replace the target point cloud with its original location in the scene to obtain the adversarial scene point cloud. Its expression is:

[0042] in, Indicates the original scene In the middle, using adversarial target point clouds Replace the original target point cloud Other background points and non-target points remain unchanged. This operation ensures that the final input to the detector is still a complete full-scene point cloud, rather than an isolated target point cloud, thus better conforming to the input format of a realistic 3D object detection model.

[0043] Therefore, by recording the original target point cloud Point cloud in all scenarios The point indexing is complete. Updated in each iteration. Then, simply replace the corresponding point coordinates with these indices to generate new ones. This ensures that the attack is focused on the target object without violating the detection model's requirement for a complete scene input format.

[0044] (2) Next, establish the dynamic point-to-voxel mapping relationship: Confrontational scenarios Before inputting a voxel-type or columnar detector, it needs to undergo voxelization. To convert the voxel feature gradients obtained from backpropagation of the detector into point-level gradients that can be used to update the target point cloud, this application reconstructs the attacked target point cloud in each attack iteration. The mapping relationship between voxel features and the mapping relationship between the voxel features. This dynamic mapping relationship is denoted as... Its definition is:

[0045] in, This represents the voxel index corresponding to each point in the point cloud of the attacked target. This indicates the point's ordinal position or local storage location within the corresponding voxel. In other words, Determine which voxel this point belongs to. Determine "where the point is located in the point list or feature tensor of the voxel".

[0046] This step must be executed dynamically, and not just once on the initial point cloud. The reason is that during the attack iteration process… Will follow As the point cloud position changes with each update, the assignment of points to voxels may also change. If the old mapping is still used, subsequent gradients will come from incorrect voxel positions, failing to accurately reflect the computation path after the current adversarial point cloud input detector, thus causing the perturbation update direction to fail. Therefore, this application updates the voxel position in each update... After that, all were recalculated. This ensures that the proxy gradient always corresponds to the current actual input.

[0047] (3) Then perform voxel gradient acquisition and point-level proxy gradient transformation: After establishing the dynamic mapping relationship, the current adversarial scenario will be... Input to voxelization function The voxel features actually processed by the detector are obtained. Subsequently, Input detection model Calculate detection loss And obtain the loss on voxel features through backpropagation. gradient The process is represented as:

[0048] in, This indicates the voxelization or columnarization preprocessing in the detector. Representation of features in voxel space This represents the inverse gradient of the detection loss in the voxel feature space. Since... Located in the voxel feature space, and the variable to be updated Located in point coordinate space, therefore a mapping relationship is required. Convert the voxel gradient into a point-level surrogate gradient.

[0049] Based on the voxel index corresponding to each attacked point and its location within the voxel ,from Extract the gradient components related to the coordinates of the point to obtain the point-level surrogate gradient. :

[0050] in, This indicates that the first three gradient channels or gradient components of the corresponding coordinate dimensions related to the three-dimensional coordinates are extracted. While not a strictly analytical gradient of the detection loss with respect to the original point coordinates, it extracts surrogate directions from the voxel space that can be used for point coordinate updates through the actual mapping relationship from the current point to the voxels. This design solves the problem of gradients not being directly propagable due to voxelization discretization operations, enabling voxel-based 3D detectors to be effectively attacked at the object level.

[0051] (4) Then perform voxel gradient aggregation and gradient fusion: Using only point-level agent gradients There are two potential problems: first, during voxelization, multiple points share the same voxel, making the gradient at a single point susceptible to local discrete allocation; second, some points have weak information corresponding to voxel features, and direct truncation may cause gradient instability. To improve the stability of the update direction, this application further groups attacked points belonging to the same voxel or adjacent voxels and aggregates them to obtain voxel group gradients. .

[0052] In one specific implementation, for attacked points within the same voxel, their proxy gradients are averaged or weighted to obtain the local trend direction corresponding to that voxel group; then, this voxel group direction is assigned to points within the group or used for fusion with point-level gradients. Thus, It can reflect the stable update trend of multiple points in the same local voxel structure, reducing single-point gradient noise.

[0053] Subsequently, the point-level agent gradient voxel gradient After normalization and fusion, the final gradient directions used for perturbation updates are obtained. :

[0054] in, Represents the fusion coefficient. This indicates a normalization operation. Preserving point-level local sensitivity allows attacks to be applied more precisely to specific target points; Providing stable trends at the voxel level can alleviate the instability caused by discretization and single-point gradient noise. The fusion of these two methods maintains the precision of the target point cloud perturbation while improving the continuity and stability of the attack iteration direction.

[0055] This step is one of the core components of the "dynamic proxy gradient" in this application. Its technical significance lies in the fact that instead of simply and crudely copying the detector gradient into the point space, it utilizes the current mapping relationship from points to voxels and the local structural relationship within voxels to transform the voxel space gradient into a proxy update direction that is more suitable for point cloud perturbation optimization.

[0056] (5) Next, perform perturbation update and budget projection: Obtain the fusion gradient Then, regarding the current disturbance Iterative updates are performed. To ensure that the update direction is not affected by the gradient scale, this application first... according to Norm normalization is performed, and then the normalized value is added to the current perturbation, and then the perturbation is passed through the projection operator. Limit the disturbance to within the budget. The update formula is:

[0057] in, This indicates the updated perturbation. This represents the disturbance budget projection operation, used to ensure that the updated disturbance still satisfies the constraints. If the updated disturbance exceeds the budget, it is scaled back to the allowable range; if it does not exceed the budget, the original update result is retained.

[0058] After each iteration, let Regenerate and Repeat the process until the preset maximum number of iterations is reached. Or it may reach the preset attack stopping condition. Since the point-to-voxel mapping relationship is reconstructed after each iteration, this application can continuously adapt to the movement of the attacked point cloud in space, avoiding the problem that "the perturbation has changed the position of the point cloud, but the gradient still comes from the old voxel position".

[0059] (6) Finally, the adversarial target point cloud is returned and the adversarial sample is saved: After the iterative optimization is completed, the final perturbation is obtained. And generate the final adversarial target point cloud. Then Return to the original full-scene point cloud The target location is obtained to determine the final confrontation scenario. The adversarial scenario retains most of the background and non-target objects of the original scene, with only slight perturbations to the target object's point cloud. Therefore, it can more accurately simulate the situation in intelligent driving scenarios where key traffic targets are disturbed, leading to detection failure.

[0060] When saving adversarial examples, the original sample index, target category, and original detection bounding box can be recorded simultaneously. Matching real bounding boxes Counter-target point cloud disturbance This includes information such as whether the attack was successful. This information can be used for subsequent security assessments, adversarial training, sample selection, and visualization analysis.

[0061] S105 scores the point cloud of the final adversarial scenario to select high-value adversarial samples.

[0062] As an example, the point cloud of the final adversarial scenario is scored to screen high-value adversarial samples, including: calculating the sample quality index based on the L2 distance between the final adversarial target point cloud and the target object point cloud; calculating the sample value index based on the number of points in the target object point cloud; and normalizing the sample quality index and the sample value index and then weighting and summing them to obtain a comprehensive score for each adversarial sample, so as to screen high-value adversarial samples.

[0063] Specifically, to ensure that the generated adversarial examples can be used not only for attack evaluation but also to improve the robustness and performance of the detector, this application further screens the generated adversarial examples. The screening process considers both sample quality and sample value. Sample quality is primarily measured by the perturbation amplitude; the smaller the perturbation, the closer the adversarial example is to the original point cloud, and the higher its geometric fidelity. Sample quality metrics are as follows: Defined as:

[0064] in, Represents the adversarial target point cloud. This represents the original target point cloud. The smaller the value, the less disturbance there is, and the higher the sample quality.

[0065] The sample value is primarily measured by the number of points included in the target point cloud. Generally, the more target points there are, the richer the geometric structure, providing more comprehensive supervisory information for the detector. Therefore, the sample value metric... Defined as:

[0066] Where |X| represents the target point cloud. The number of points in the sample is considered. While a target with too few points may be successfully attacked, its geometric information is limited, and its contribution to improving the detector when used as adversarial training samples may be weak. Therefore, this application includes the number of points as part of the sample value.

[0067] The final sample score is obtained by combining the perturbation quality and the sample value. :

[0068] in, and These represent the normalization processing of the disturbance amplitude and the number of points, respectively. Samples with smaller perturbations receive higher scores. The more points a sample has, the higher its score. This scoring method allows for the priority selection of high-quality adversarial samples that are "less perturbative and more informative".

[0069] S106, the selected high-value adversarial samples are merged with the original clean samples to obtain the final adversarial training dataset.

[0070] Specifically, based on sample scores All adversarial examples are sorted, and the top-ranked examples are selected to form the filtered set of adversarial examples. :

[0071] in, This represents the complete set of generated adversarial examples. Indicates according to the rating This involves selecting a certain percentage of the samples. In practice, the top 50% or other percentages of samples can be selected based on training resources and data size.

[0072] Finally, the filtered set of adversarial examples is merged with the original clean training set to form the training set for adversarial training:

[0073] in, This represents the original clean training set. This represents the dataset ultimately used to train or fine-tune the detector. Through this training set, the detector can be exposed to both normal samples and object-level adversarial difficult samples during training, thereby learning more stable target geometry features and more robust detection boundaries.

[0074] It should be noted that the adversarial training in this step does not simply involve adding all attack samples to the training set. Instead, the samples are first evaluated for quality and value using the formula mentioned above, and then the samples with greater training significance are selected. This avoids introducing noisy supervision from samples with excessive perturbation or low information content, making the adversarial training process more stable.

[0075] Additionally, in a preferred embodiment, the perturbation budget It can be set to 0.20, which determines the number of attack iterations. It can be set to 10. This setting achieves a good balance between computational efficiency and attack effectiveness. For different datasets, different detectors, or different target categories, the setting can be adjusted appropriately based on the target point cloud scale and detector sensitivity. and .

[0076] voxelization function The target detector's own voxelization module can be used, without requiring additional changes to the detector structure. Mapping relationship This can be obtained by recording the voxel index and intra-voxel position of each target point during the voxelization process. If the detector uses sparse voxel representation, then... It can represent sparse voxel coordinates or voxel numbers. This can represent the point order within a voxel. The above substitutions do not change the core idea of ​​this method.

[0077] Gradient fusion coefficient Used for equilibrium point-level proxy gradients and voxel group gradients. When the value is large, the update direction depends more on point-level local sensitivity; When the size is small, the update direction depends more on the stable trend of the voxel set. In actual deployment, an appropriate voxel size can be selected based on the target detector's voxel size, the number of points within each voxel, and the target point cloud density. .

[0078] The output of this application can be used for two purposes. The first is to assess the security of intelligent driving 3D target detectors, specifically observing whether attacked targets experience missed detections, decreased confidence, or location shifts. The second is to construct adversarial training data to improve the detector's performance under difficult targets and perturbed scenarios. Because this application only perturbs the target object point cloud while preserving the complete scene structure, the generated samples possess clear object semantics and real-world traffic scene significance, making them suitable as robustness evaluation and training enhancement samples for intelligent driving perception systems.

[0079] In summary, this application proposes a method for generating adversarial examples of 3D point clouds based on dynamic proxy gradients. This method does not indiscriminately modify the entire frame of LiDAR point clouds. Instead, it first locates traffic targets in a clean scene that have been successfully detected by the detector, extracts the object point clouds within their 3D detection boxes, applies limited perturbations only to the target object point clouds, and then returns the perturbed object point clouds to the original scene to form an adversarial scene. Thus, this application can concentrate the limited perturbation budget on safety-critical foreground targets such as vehicles, pedestrians, and cyclists, avoiding large-scale ineffective modifications to the background area.

[0080] To implement the above embodiments, this invention also proposes a computer-readable storage medium storing a 3D point cloud adversarial example generation program based on dynamic proxy gradients. When executed by a processor, the 3D point cloud adversarial example generation program based on dynamic proxy gradients implements the 3D point cloud adversarial example generation method based on dynamic proxy gradients as described above.

[0081] To implement the above embodiments, this invention also proposes a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, it implements the above-described method for generating adversarial examples of 3D point clouds based on dynamic proxy gradients.

[0082] To implement the above embodiments, this invention also proposes a 3D point cloud adversarial example generation device based on dynamic proxy gradients, such as... Figure 3 As shown, the 3D point cloud adversarial sample generation device based on dynamic proxy gradient includes: an attack target determination module 10, a target object point cloud extraction module 20, an attack optimization target setting module 30, a voxel-aware adversarial perturbation iterative optimization module 40, an adversarial sample quality and value evaluation module 50, and an adversarial training data construction module 60.

[0083] The attack target determination module 10 is used to input the acquired full-scene point cloud into the 3D target detection model for target detection, output the detection results, and obtain the predicted bounding box of the attackable object based on the detection results; the target object point cloud extraction module 20 is used to extract all points whose coordinates fall within the predicted bounding box of the attackable object, so as to obtain the target object point cloud; the attack optimization target setting module 30 is used to set the optimization objective as maximizing the detection loss of the 3D target detection model on the attack target, and constrain the L2 norm of the perturbation to not exceed the preset budget, forming a constrained optimization rule; the voxel-aware adversarial perturbation iterative optimization module... Block 40 is used to generate adversarial scenarios, dynamic point voxel mapping, voxel gradient transformation and fusion, and closed-loop optimization of perturbation updates under constraints based on constrained optimization rules, full-scene point clouds, and target object point clouds through multiple rounds of iterative perturbation superposition, to obtain the final adversarial target point cloud and the final adversarial scenario point cloud that conform to the constrained optimization rules; the adversarial sample quality and value evaluation module 50 is used to score the final adversarial scenario point cloud to screen high-value adversarial samples; the adversarial training data construction module 60 is used to merge the screened high-value adversarial samples with the original clean samples to obtain the final adversarial training dataset.

[0084] It should be noted that the above description of the 3D point cloud adversarial example generation method based on dynamic proxy gradient also applies to the 3D point cloud adversarial example generation device based on dynamic proxy gradient, and will not be repeated here.

[0085] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0086] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0087] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0088] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0089] It should be noted that any reference signs placed between parentheses in the claims should not be construed as limiting the claims. The word "comprising" does not exclude the presence of components or steps not listed in the claims. The word "a" or "an" preceding a component does not exclude the presence of a plurality of such components. The invention can be implemented by means of hardware comprising several different components and by means of a suitably programmed computer. In a unit claim enumerating several means, several of these means may be embodied by the same item of hardware. The use of the words first, second, and third, etc., does not indicate any order. These words can be interpreted as names.

[0090] Although preferred embodiments of the invention have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments as well as all changes and modifications falling within the scope of the invention.

[0091] Obviously, those skilled in the art can make various modifications and variations to this invention without departing from its spirit and scope. Therefore, if these modifications and variations fall within the scope of the claims of this invention and their equivalents, this invention also intends to include these modifications and variations.

[0092] In the description of this invention, it should be understood that the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Therefore, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature. In the description of this invention, "a plurality of" means two or more, unless otherwise explicitly specified.

[0093] In this invention, unless otherwise explicitly specified and limited, the terms "installation," "connection," "linking," and "fixing," etc., should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral part; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; they can refer to the internal communication of two components or the interaction between two components. Those skilled in the art can understand the specific meaning of the above terms in this invention according to the specific circumstances.

[0094] In this invention, unless otherwise explicitly specified and limited, "above" or "below" the second feature can mean that the first feature is in direct contact with the second feature, or that the first feature is in indirect contact with the second feature through an intermediate medium. Furthermore, "above," "over," and "on top" of the second feature can mean that the first feature is directly above or diagonally above the second feature, or simply that the first feature is at a higher horizontal level than the second feature. "Below," "below," and "under" the second feature can mean that the first feature is directly below or diagonally below the second feature, or simply that the first feature is at a lower horizontal level than the second feature.

[0095] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., refer to specific features, structures, materials, or characteristics described in connection with that embodiment or example, which are included in at least one embodiment or example of the present invention. In this specification, the illustrative expressions of the above terms should not be construed as necessarily referring to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of different embodiments or examples.

[0096] Although embodiments of the present invention have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting the present invention. Those skilled in the art can make changes, modifications, substitutions and variations to the above embodiments within the scope of the present invention.

Claims

1. A method for generating adversarial examples of 3D point clouds based on dynamic proxy gradients, characterized in that, Includes the following steps: The acquired full-scene point cloud is input into the 3D target detection model for target detection, and the detection results are output. Based on the detection results, the predicted bounding box of the attackable object is obtained. Extract all points whose coordinates fall within the predicted bounding box of the attackable object to obtain the point cloud of the target object; The optimization objective is to maximize the detection loss of the 3D target detection model for the attack target, and the L2 norm of the perturbation is constrained to not exceed the preset budget, thus forming a constrained optimization rule; Based on the constrained optimization rules, the adversarial scene is generated by multi-round iterative perturbation superposition of the full scene point cloud and the target object point cloud, dynamic point voxel mapping, voxel gradient transformation and fusion, and closed-loop optimization of updating perturbation under constraints, so as to obtain the final adversarial target point cloud and the final adversarial scene point cloud that conform to the constrained optimization rules. The point cloud of the final adversarial scenario is scored to select high-value adversarial samples; The selected high-value adversarial samples are merged with the original clean samples to obtain the final adversarial training dataset.

2. The method for generating adversarial examples of 3D point clouds based on dynamic proxy gradients as described in claim 1, characterized in that, Based on the detection results, a predicted bounding box of the attackable object is obtained, including: The predicted bounding boxes in the detection results are matched with the ground truth bounding boxes, and the predicted bounding boxes that satisfy the three-dimensional IoU between the predicted bounding box and the ground truth bounding box ≥ a preset intersection-union ratio threshold and the confidence level ≥ a preset confidence level threshold are identified as attackable targets.

3. The method for generating adversarial examples of 3D point clouds based on dynamic proxy gradients as described in claim 1, characterized in that, Extract all points whose coordinates fall within the predicted bounding box of the attackable object to obtain the target object point cloud, including: The entire scene point cloud is filtered by the field of view to obtain the filtered point cloud. Based on the 3D coordinate range of the predicted bounding box of the attackable object, all points whose coordinates fall within the bounding box are extracted from the filtered point cloud to obtain the target object point cloud.

4. The method for generating adversarial examples of 3D point clouds based on dynamic proxy gradients as described in claim 1, characterized in that, Based on the constrained optimization rules, the full-scene point cloud and the target object point cloud undergo closed-loop optimization through multiple rounds of iterative dynamic voxel mapping, voxel gradient transformation, gradient fusion and constrained projection to obtain the final adversarial target point cloud and the final adversarial scene point cloud that conform to the constrained optimization rules, including: Obtain the current round perturbation, apply the perturbation to the target object point cloud to generate an adversarial target point cloud, and replace the adversarial target point cloud back to the corresponding position of the full scene point cloud to obtain the adversarial scene point cloud; The adversarial scenario point cloud is subjected to voxelization preprocessing that is completely consistent with that of the detector. The voxel index and voxel position index corresponding to each point in the adversarial target point cloud are recorded point by point to obtain a dynamic mapping relationship. The adversarial scenario point cloud is input into the detector, the detection loss of the current round is calculated forward, the gradient of the loss with respect to the voxel input features is obtained through backpropagation, and the coordinate dimension gradient component of the corresponding position is extracted from the gradient point by point according to the dynamic mapping relationship to obtain the point-level proxy gradient. Based on the voxel index, target points belonging to the same voxel are grouped together. The point-level proxy gradients of all points in the group are averaged and aggregated to obtain the voxel group gradient. The point-level proxy gradient and the voxel group gradient are L2 normalized respectively, and then fused according to the fusion coefficient to obtain the fused gradient. The fused gradient is L2 normalized and then superimposed on the current perturbation to obtain the updated candidate perturbation; the L2 norm of the candidate perturbation is calculated, and if it exceeds the preset budget, it is scaled back to the allowable range by the projection operator to obtain the perturbation for the next iteration; it is determined whether the maximum number of iterations has been reached. If not, the next iteration begins; if it has been reached, the iteration stops to obtain the final optimized perturbation. The final optimized perturbation is applied to the target object point cloud to generate the final adversarial target point cloud, and the final adversarial target point cloud is replaced back to the corresponding position of the full scene point cloud to obtain the final adversarial scene point cloud.

5. The method for generating adversarial examples of 3D point clouds based on dynamic proxy gradients as described in claim 1, characterized in that, The point cloud of the final adversarial scenario is scored to screen high-value adversarial samples, including: The sample quality index is calculated based on the L2 distance between the final adversarial target point cloud and the target object point cloud. The sample value index is calculated based on the number of points in the point cloud of the target object. After normalizing the sample quality indicators and sample value indicators, a weighted sum is obtained to obtain a comprehensive score for each adversarial sample, in order to screen high-value adversarial samples.

6. A computer-readable storage medium, characterized in that, It stores a 3D point cloud adversarial example generation program based on dynamic proxy gradient, which, when executed by the processor, implements the 3D point cloud adversarial example generation method based on dynamic proxy gradient as described in any one of claims 1-5.

7. A computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the method for generating adversarial examples of three-dimensional point clouds based on dynamic proxy gradients as described in any one of claims 1-5.

8. A device for generating adversarial examples from 3D point clouds based on dynamic proxy gradients, characterized in that, include: The attack target determination module is used to input the acquired full-scene point cloud into the 3D target detection model for target detection, output the detection results, and obtain the predicted bounding box of the attackable object based on the detection results. The target object point cloud extraction module is used to extract all points whose coordinates fall within the predicted bounding box of the attackable object to obtain the target object point cloud. The attack optimization target setting module is used to optimize the detection loss of the 3D target detection model on the attack target by maximizing the detection loss of the target, while constraining the L2 norm of the perturbation to not exceed the preset budget, thus forming a constrained optimization rule. The voxel-aware adversarial perturbation iterative optimization module is used to generate an adversarial scene, dynamic voxel mapping, voxel gradient transformation and fusion, and closed-loop optimization of perturbation update under constraints based on the constrained optimization rules, full scene point cloud and target object point cloud through multiple rounds of iterative perturbation superposition, to obtain the final adversarial target point cloud and final adversarial scene point cloud that conform to the constrained optimization rules. The adversarial example quality and value assessment module is used to score the final adversarial scenario point cloud in order to screen high-value adversarial examples. The adversarial training data construction module is used to merge the selected high-value adversarial samples with the original clean samples to obtain the final adversarial training dataset.

Images