Detection method, device and equipment for laser radar attack and storage medium

Abstract

The present specification relates to the technical field of automatic driving, and provides a detection method and device for laser radar attack, equipment and storage medium, the method comprising: converting the point cloud data sets collected by a first laser radar and a second laser radar at the same time into a first surface grid and a second surface grid; the first laser radar and the second laser radar are located on different vehicles and the distance therebetween meets a preset condition; obtaining a first difference value between the first surface grid and the second surface grid; according to the size relationship between the first difference value and a first difference value threshold, determining an attack detection result for the first laser radar and the second laser radar. The embodiment of the present specification can detect the attack on the laser radar and improve the driving safety of the automatic driving vehicle.

Description

Technical Field

[0001] This specification relates to the field of autonomous driving technology, and in particular to a detection method, apparatus, device, and storage medium for LiDAR attacks. Background Technology

[0002] LiDAR (Light Detection and Ranging) is an indispensable driving environment sensor in the perception module of autonomous driving systems, providing real-time three-dimensional (3D) data of the vehicle's surroundings. Due to its active beam emission, LiDAR is less affected by ambient light (e.g., low light) compared to cameras, and the object detection module of autonomous driving systems can directly utilize the 3D data collected by LiDAR for object recognition. In recent years, some researchers have proposed attacking LiDAR using physical devices to insert points. Due to the inherent limitations of object detection models, these attacks often manage to mislead the victim vehicle's autonomous driving system into believing that an object is nearby by inserting a small number of points, causing it to brake suddenly or otherwise compromise driving safety. Summary of the Invention

[0003] The purpose of the embodiments in this specification is to provide a method, apparatus, device, and storage medium for detecting attacks against lidar, so as to detect attacks against lidar and improve the driving safety of autonomous vehicles.

[0004] To achieve the above objectives, in one aspect, embodiments of this specification provide a detection method for lidar attacks, including:

[0005] The point cloud datasets collected by the first lidar and the second lidar at the same time are respectively converted into the first surface mesh and the second surface mesh; the first lidar and the second lidar are located on different vehicles and their spacing meets the preset conditions.

[0006] Obtain the first difference value between the first surface mesh and the second surface mesh;

[0007] Based on the relationship between the first difference value and the first difference value threshold, the attack detection results for the first lidar and the second lidar are determined.

[0008] In the detection method for lidar attacks in the embodiments of this specification, the point cloud datasets collected by the first lidar and the second lidar at the same time are respectively converted into a first surface mesh and a second surface mesh, including:

[0009] The first point cloud dataset collected by the first lidar at the current moment and the second point cloud dataset collected by the second lidar at the current moment are respectively input into the region candidate network to obtain the first candidate box set and the second candidate box set for selecting objects.

[0010] Merge the first candidate box set and the second candidate box set into a target candidate box set;

[0011] In the target candidate box set, determine the first subset of point cloud data corresponding to each candidate box in the first point cloud dataset and the second subset of point cloud data corresponding to each candidate box in the second point cloud dataset;

[0012] The first subset of point cloud data corresponding to all candidate boxes in the target candidate box set in the first point cloud dataset is input into the surface mesh generation model to obtain the first surface mesh; and the second subset of point cloud data corresponding to all candidate boxes in the target candidate box set in the second point cloud dataset is input into the surface mesh generation model to obtain the second surface mesh.

[0013] In the detection method for lidar attacks in the embodiments of this specification, the point cloud datasets collected by the first lidar and the second lidar at the same time are respectively converted into a first surface mesh and a second surface mesh, including:

[0014] The first point cloud dataset collected by the first lidar at the current moment and the second point cloud dataset collected by the second lidar at the current moment are respectively input into the region candidate network to obtain the first candidate box set and the second candidate box set for selecting objects.

[0015] Merge the first candidate box set and the second candidate box set into a target candidate box set;

[0016] In the target candidate box set, determine the first subset of point cloud data corresponding to each candidate box in the first point cloud dataset and the second subset of point cloud data corresponding to each candidate box in the second point cloud dataset;

[0017] Based on the symmetry relationship of the objects within each candidate box, the first point cloud data subset and the second point cloud data subset of each candidate box are mirrored to obtain the mirror image of the first point cloud data subset and the mirror image of the second point cloud data subset of each candidate box.

[0018] The first point cloud data subset and the mirror image of the first point cloud data subset of each candidate box are superimposed to obtain a new first point cloud data subset of each candidate box. The second point cloud data subset and the mirror image of the second point cloud data subset of each candidate box are superimposed to obtain a new second point cloud data subset of each candidate box.

[0019] A new first point cloud data subset of all candidate boxes in the target candidate box set is input into the surface mesh generation model to obtain a first surface mesh; and a new second point cloud data subset of all candidate boxes in the target candidate box set is input into the surface mesh generation model to obtain a second surface mesh.

[0020] In the detection method for lidar attacks in the embodiments of this specification, obtaining a first difference value between the first surface mesh and the second surface mesh includes:

[0021] Discretize the first surface grid and the second surface grid using squares with a specified side length as units, and use the distance of the center point of each square relative to the ground as the value of that square.

[0022] Determine the average difference between the abnormal square portions corresponding to the same candidate box in the discretized first surface mesh and the discretized second surface mesh.

[0023] In the detection method for LiDAR attacks in the embodiments of this specification, determining the average difference between the abnormal square portions corresponding to the same candidate box in the discretized first surface mesh and the discretized second surface mesh includes:

[0024] The average difference between the outlier square portions corresponding to the same candidate box in the discretized first surface mesh and the discretized second surface mesh is determined using the following formula:

[0025]

[0026] in, Let g be the average difference between the outlier squares corresponding to the i-th candidate box in the discretized first surface mesh and the discretized second surface mesh, and let g be the g-th outlier square in the i-th candidate box. Let G be the set of g. To discretize the value of the g-th anomaly square in the i-th candidate box of the first surface mesh, To discretize the value of the g-th anomalous square in the i-th candidate box of the second surface mesh, |G| is the number of anomalous squares in G, and θ is a preset threshold value used to distinguish normal squares from anomalous squares.

[0027] In the detection method for LiDAR attacks in the embodiments of this specification, the attack detection results for the first LiDAR and the second LiDAR are determined based on the relationship between the first difference value and the first difference value threshold, including:

[0028] When the first difference value is greater than the first difference value threshold, it is confirmed that at least one of the first lidar and the second lidar has been attacked.

[0029] When the first difference value is not greater than the first difference value threshold, it is confirmed that the first lidar and the second lidar have not been attacked.

[0030] The detection method for lidar attacks in the embodiments of this specification further includes:

[0031] The point cloud datasets collected by the third lidar at the same time are converted into a third surface mesh; the first lidar, the second lidar, and the third lidar are located on different vehicles and their spacing meets a preset condition.

[0032] Obtain a second difference value between the first surface mesh and the third surface mesh, and a third difference value between the second surface mesh and the third surface mesh;

[0033] Based on the relationship between the second difference value and the second difference value threshold, the attack detection results for the first lidar and the third lidar are determined; based on the relationship between the third difference value and the third difference value threshold, the attack detection results for the second lidar and the third lidar are determined.

[0034] The first, second, and third lidars are identified as being attacked by combining the results of each attack detection.

[0035] The detection method for lidar attacks in the embodiments of this specification also includes:

[0036] When it is confirmed that a lidar has been attacked, the corresponding candidate box is discarded.

[0037] On the other hand, embodiments of this specification also provide a detection device for lidar attacks, including:

[0038] The conversion module is used to convert the point cloud datasets collected by the first lidar and the second lidar at the same time into the first surface mesh and the second surface mesh respectively; the first lidar and the second lidar are located on different vehicles and their spacing meets the preset conditions.

[0039] The acquisition module is used to acquire a first difference value between the first surface mesh and the second surface mesh;

[0040] The determination module is used to determine the attack detection results for the first lidar and the second lidar based on the relationship between the first difference value and the first difference value threshold.

[0041] On the other hand, embodiments of this specification also provide a computer device, including a memory, a processor, and a computer program stored in the memory, wherein the computer program, when run by the processor, executes instructions for the above-described method.

[0042] On the other hand, embodiments of this specification also provide a computer storage medium storing a computer program thereon, which, when run by the processor of a computer device, executes instructions for the above-described method.

[0043] On the other hand, embodiments of this specification also provide a computer program product, which includes a computer program that, when run by the processor, executes instructions for the above-described method.

[0044] As can be seen from the technical solutions provided in the embodiments of this specification above, in the embodiments of this specification, after converting the point cloud data from different vehicle lidars (the distance between the two lidars meets the preset conditions) into surface meshes respectively, the difference between the two surface meshes is calculated, and the relationship between the difference between the two surface meshes and the difference value threshold is used to determine whether the two lidars have been attacked. This realizes the automatic detection of attacks against lidars, so that the autonomous driving system can take action based on the detection results, which is beneficial to improving the driving safety of autonomous vehicles. Attached Figure Description

[0045] To more clearly illustrate the technical solutions in the embodiments or prior art of this specification, the drawings used in the description of the embodiments or prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in this specification. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort. In the drawings:

[0046] Figure 1 Flowcharts of detection methods for lidar attacks in some embodiments of this specification are shown;

[0047] Figure 2 A schematic diagram showing the distance relationship between lidar on different vehicles in one embodiment of this specification is shown;

[0048] Figure 3a This specification shows a schematic diagram of point cloud data acquired by a lidar in one embodiment;

[0049] Figure 3b This specification illustrates an embodiment based on a region candidate network. Figure 3a The diagram shows the processing results obtained by processing the point cloud data.

[0050] Figure 4 A schematic diagram of the process for generating surface meshes in some embodiments of this specification is shown;

[0051] Figure 5 Schematic diagrams illustrating the process of generating surface meshes in other embodiments of this specification are shown;

[0052] Figure 6 This diagram illustrates the calculation of the value of the g-th anomalous square in the i-th candidate frame of a discretized surface mesh according to one embodiment of this specification.

[0053] Figure 7 Flowcharts of detection methods for lidar attacks in other embodiments of this specification are shown;

[0054] Figure 8 The present specification shows a structural block diagram of a detection device against lidar attacks in some embodiments;

[0055] Figure 9 A structural block diagram of a computer device in some embodiments of this specification is shown.

[0056] [Explanation of Labels in the Attached Image]

[0057] LR1, the first lidar;

[0058] LR2, the second lidar;

[0059] 10. Discretize the first surface network;

[0060] 20. Discretize the second surface network;

[0061] 30. Ground;

[0062] 81. Conversion module;

[0063] 82. Acquisition Module;

[0064] 83. Determine the module;

[0065] 902. Computer equipment;

[0066] 904, Processor;

[0067] 906. Memory;

[0068] 908. Drive mechanism;

[0069] 910. Input / output interfaces;

[0070] 912. Input devices;

[0071] 914. Output devices;

[0072] 916. Presentation equipment;

[0073] 918. Graphical User Interface;

[0074] 920. Network interface;

[0075] 922. Communication link;

[0076] 924. Communication bus. Detailed Implementation

[0077] To enable those skilled in the art to better understand the technical solutions in this specification, the technical solutions in the embodiments of this specification will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this specification, and not all embodiments. Based on the embodiments in this specification, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of this specification.

[0078] Autonomous driving systems rely on the collaborative efforts of artificial intelligence, computer vision, radar, monitoring devices, and navigation and positioning systems to automatically and safely control vehicles (i.e., self-driving vehicles) without any active human intervention. Autonomous driving systems often use lidar (LiDAR) to detect the driving environment around the vehicle. In recent years, researchers have proposed many machine learning models that can accurately measure the depth of objects and detect them using collected data. However, the application of these models introduces new vulnerabilities that may compromise the safety of autonomous vehicles. For example, lidar can be attacked (e.g., by physically intervening at the lidar point), causing the data provided to the machine learning model to become inaccurate (e.g., generating false object detection results). This could affect the accuracy of object detection in the machine learning model, thereby impacting the safety of the autonomous vehicle. Therefore, how to detect attacks targeting lidar has become a pressing technical problem to be solved.

[0079] In view of this, the embodiments of this specification provide a detection method for lidar attacks, which can be applied to the autonomous driving system of autonomous vehicles. (Reference) Figure 1 As shown, in some embodiments, the detection method for lidar attacks may include:

[0080] Step 101: Convert the point cloud datasets collected by the first lidar and the second lidar at the same time into a first surface mesh and a second surface mesh respectively; the first lidar and the second lidar are located on different vehicles and their spacing meets the preset conditions.

[0081] Step 102: Obtain the first difference value between the first surface mesh and the second surface mesh.

[0082] Step 103: Determine the attack detection results for the first lidar and the second lidar based on the relationship between the first difference value and the first difference value threshold.

[0083] In the detection method of the embodiments of this specification, after converting the point cloud data from LiDARs of different vehicles (the distance between the two LiDARs meets the preset conditions) into surface meshes, the difference between the two surface meshes is calculated, and the relationship between the difference between the two surface meshes and the difference value threshold is used to determine whether the two LiDARs have been attacked. This realizes the automatic detection of attacks against LiDARs, so that the autonomous driving system can take action based on the detection results, which is beneficial to improving the driving safety of autonomous vehicles.

[0084] The embodiments in this specification can be applied to multi-vehicle scenarios, i.e., scenarios where the autonomous driving systems of multiple (e.g., two or more) autonomous vehicles cooperate to complete the task, and these vehicles use LiDAR as a driving environment perception module. When multiple autonomous vehicles are close together, their multiple LiDARs can collect partially overlapping driving environment data, which can then be used to collaboratively identify whether the LiDARs are under attack. Therefore, it is necessary to set preset conditions for the spacing between multiple LiDARs. For example, the distance between multiple LiDARs can be set to not exceed a specified distance (e.g., 5 meters, 8 meters, 10 meters, etc.). In specific implementations, the specified distance can be appropriately set according to the actual situation. For example, in scenarios such as... Figure 2 In the embodiment shown, taking two autonomous vehicles as an example, the two corresponding LiDARs are LiDAR LR1 and LiDAR LR2. When the distance between LiDAR LR1 and LiDAR LR2 does not exceed a specified distance, it can be considered that the distance between LiDAR LR1 and LiDAR LR2 meets the preset conditions.

[0085] The point cloud data collected by lidar is a massive set of points representing the surface characteristics of an object (e.g., Figure 3a As shown in the figure, in order to facilitate subsequent object recognition, the point cloud data can be converted into the corresponding surface mesh.

[0086] refer to Figure 4 As shown, in some embodiments, converting the point cloud datasets collected by the first lidar and the second lidar at the same time into a first surface mesh and a second surface mesh respectively may include the following steps:

[0087] Step 401: Input the first point cloud dataset collected by the first lidar at the current moment and the second point cloud dataset collected by the second lidar at the current moment into the Region Proposal Network to obtain the first candidate box set and the second candidate box set for selecting objects.

[0088] Region candidate networks are pre-trained neural network models that can generate candidate boxes based on point cloud data. The candidate boxes represent the regions where objects may exist (i.e., the candidate boxes represent the features of the relationships between points in the regions where objects may exist in the point cloud data). They are intermediate results of object detection tasks, and the final results of object detection tasks are selected from the candidate boxes.

[0089] Inputting the first point cloud dataset collected by the first LiDAR at the current moment into the region candidate network yields a first candidate box set. Similarly, inputting the second point cloud dataset collected by the second LiDAR at the current moment into the region candidate network yields a second candidate box set. Typical driving environments generally contain areas where multiple objects (e.g., pedestrians, vehicles, etc.) may exist. Therefore, inputting each point cloud dataset into the region candidate network yields multiple candidate boxes, thus forming a candidate box set. For example, in an exemplary embodiment, using... Figure 3a Taking the point cloud data shown as an example, after inputting it into the region candidate network, the following results can be obtained: Figure 3b The recognition results shown (in) Figure 3b The three small squares in the middle represent the obtained set of candidate boxes.

[0090] Step 402: Merge the first candidate box set and the second candidate box set into a target candidate box set.

[0091] In the embodiments of this specification, due to factors such as acquisition location, object occlusion, and device performance, even if the first and second lidars, which meet the preset conditions, acquire point cloud data at the same time, they may still collect slightly different driving environments. Merging the two candidate bounding box sets can help obtain a more complete set of candidate bounding boxes, thereby facilitating the acquisition of a more accurate driving environment.

[0092] Both the first and second lidar sensors acquire point cloud data in a coordinate system with their own center as the origin, and merging them requires operating within the same coordinate system. Therefore, both the first and second candidate box sets can be transformed to the world coordinate system. The transformation matrices between the coordinate systems of the first and second lidar sensors and the world coordinate system can be predetermined. Thus, using these transformation matrices, the first and second candidate box sets can be rotated, translated, and transformed to obtain a complete set of candidate boxes, which is the target candidate box set.

[0093] Step 403: Determine the first subset of point cloud data corresponding to each candidate box in the first point cloud dataset and the second subset of point cloud data corresponding to each candidate box in the second point cloud dataset.

[0094] Each candidate box in the target candidate box set has a unique position in the world coordinate system, but the LiDAR uses its own center point as the origin, which is different from the world coordinate system. Therefore, for each candidate box in the target candidate box set, it is necessary to obtain the portion of the point cloud data it covers in the first point cloud dataset (i.e., the first subset of point cloud data) and the portion of the point cloud data it covers in the second point cloud dataset (i.e., the second subset of point cloud data).

[0095] Step 404: Input the first subset of point cloud data corresponding to all candidate boxes in the target candidate box set in the first point cloud dataset into the surface mesh generation model to obtain the first surface mesh; and input the second subset of point cloud data corresponding to all candidate boxes in the target candidate box set in the second point cloud dataset into the surface mesh generation model to obtain the second surface mesh.

[0096] A surface mesh is a meshed surface of an object. A surface mesh generation model uses points from point cloud data to generate a model of the object's surface (in the form of a mesh). In some embodiments, any suitable mesh reconstruction method can be used to generate the surface mesh. For example, in one embodiment, the surface mesh generation model can employ mesh reconstruction methods such as Volumetric Range Image Processing (VRIP) or Possion.

[0097] refer to Figure 5 As shown, in some other embodiments, converting the point cloud datasets collected by the first lidar and the second lidar at the same time into a first surface mesh and a second surface mesh respectively may include the following steps:

[0098] Step 501: Input the first point cloud dataset collected by the first lidar at the current time and the second point cloud dataset collected by the second lidar at the current time into the region candidate network to obtain the first candidate box set and the second candidate box set for selecting objects.

[0099] Step 502: Merge the first candidate box set and the second candidate box set into a target candidate box set.

[0100] Step 503: Determine the first subset of point cloud data corresponding to each candidate box in the first point cloud dataset and the second subset of point cloud data corresponding to each candidate box in the second point cloud dataset.

[0101] Step 504: Based on the symmetry relationship of the objects within each candidate box, mirror the first point cloud data subset and the second point cloud data subset of each candidate box to obtain the mirror image of the first point cloud data subset and the mirror image of the second point cloud data subset of each candidate box.

[0102] In this context, symmetry is defined by the object's centerline as the axis of symmetry. For example, if a candidate box contains a car, the line connecting the midpoint of the car's front and rear can be used as the axis of symmetry. Mirroring is a prerequisite for subsequent mirror overlay.

[0103] Step 505: Mirror and superimpose the first point cloud data subset and the first point cloud data subset of each candidate box to obtain a new first point cloud data subset of each candidate box; and mirror and superimpose the second point cloud data subset and the second point cloud data subset of each candidate box to obtain a new second point cloud data subset of each candidate box.

[0104] The purpose of point cloud data overlay is to fill the blank areas in the candidate frame that were not scanned by the LiDAR with a mirror copy of the original point cloud data. Since some areas in the original point cloud data may be missing points, directly using such point cloud data to generate the surface mesh would reduce accuracy. By creating a mirror copy based on the object's symmetry and overlaying it with the original point cloud data, some areas that were originally missing points may now have points, thus filling these blank areas and generating a more complete surface mesh. This provides a more accurate basis for subsequent comparisons of surface mesh differences.

[0105] Step 506: Input the new first point cloud data subset of all candidate boxes in the target candidate box set into the surface mesh generation model to obtain the first surface mesh; and input the new second point cloud data subset of all candidate boxes in the target candidate box set into the surface mesh generation model to obtain the second surface mesh.

[0106] In some embodiments, obtaining a first difference value between the first surface mesh and the second surface mesh may include the following steps:

[0107] (1) Discretize the first surface grid and the second surface grid in units of squares with a specified side length, and use the distance of the center point of each square relative to the ground as the value of the square.

[0108] For example, in one embodiment, the first surface grid and the second surface grid can be discretized in units of squares with a side length of 0.1 meters.

[0109] (2) Determine the average difference between the abnormal square parts corresponding to the same candidate box in the discretized first surface mesh and the discretized second surface mesh.

[0110] In some embodiments, the average difference between the outlier square portions corresponding to the same candidate box in the discretized first surface mesh and the discretized second surface mesh can be determined according to the following formula:

[0111]

[0112] in, Let g be the average difference between the outlier squares corresponding to the i-th candidate box in the discretized first surface mesh and the discretized second surface mesh, and let g be the g-th outlier square in the i-th candidate box. Let G be the set of g. To discretize the value of the g-th anomaly square in the i-th candidate box of the first surface mesh, To discretize the value of the g-th anomalous square in the i-th candidate frame of the second surface mesh, |G| is the number of anomalous squares in G, and θ is a preset threshold value used to distinguish normal squares from anomalous squares (i.e., the average distance of all squares from the ground). When a square g satisfies When this happens, the square g is considered an abnormal square.

[0113] For example, in such Figure 6 If the same candidate box exists in both the discretized first surface mesh 10 and the discretized second surface mesh 20 (see the black lines within the discretized first surface mesh 10 and the discretized second surface mesh 20), then the value of the distance 30 from the ground for the g-th anomaly square within that candidate box in the discretized first surface mesh 10 can be determined as follows: Figure 6 The dashed line on the left indicates that the distance 30 from the ground to the g-th anomaly square within the candidate box in the discretized second surface mesh 20 can be determined as follows: Figure 6 The right-hand dashed line in the middle indicates...

[0114] Under safe conditions (i.e., neither lidar is under attack), a large number of discrete surface mesh distance samples can be generated to determine the value distribution of normal discrete surface mesh distances. Based on the value distribution of normal discrete surface mesh distances, a threshold α is selected along with a specified false alarm rate r to distinguish between distance anomalies caused by attacks and normal values ​​under no-attack conditions. The relationship between them is as follows: Where r represents the specified false alarm rate, α is the threshold for the distance between discrete surface grids (i.e., the difference threshold), and #samples of dist>α represents the number of samples exceeding the threshold among a large number of distance samples collected under safe conditions. #samples of dist represents the total number of distance samples collected under safe conditions. When the average difference between the abnormal square portions corresponding to the same candidate box in two discrete surface grids is greater than this difference threshold, it indicates that the object in the candidate box is a false object.

[0115] Therefore, based on the relationship between the first difference value and the first difference value threshold, the attack detection results for the first lidar and the second lidar are determined, which may specifically include the following two cases:

[0116] 1) When the first difference value is greater than the first difference value threshold, it is confirmed that at least one of the first lidar and the second lidar has been attacked.

[0117] 2) When the first difference value is not greater than the first difference value threshold, it is confirmed that the first lidar and the second lidar have not been attacked.

[0118] In some embodiments, the above-described detection method for lidar attacks may further include:

[0119] When it is confirmed that the LiDAR has been attacked, it indicates that the object in the corresponding candidate box is an inserted spurious object. Therefore, the corresponding candidate box can be discarded. In this way, the spurious object in the candidate box will no longer be the object to be identified by the object recognition processing module of the autonomous driving system, thereby improving the recognition accuracy of the object recognition processing module of the autonomous driving system and improving the safety of autonomous vehicles.

[0120] The aforementioned method for detecting LiDAR attacks can only identify whether one of two LiDARs meeting preset conditions has been attacked, but it cannot make further distinctions; that is, it cannot differentiate which specific LiDAR has been attacked, or whether both have been attacked. Therefore, this specification provides another method for detecting LiDAR attacks, which can be applied to the autonomous driving system of autonomous vehicles. (See reference...) Figure 7 As shown, the detection method for lidar attacks may include the following steps:

[0121] Step 701: Convert the point cloud datasets collected by the first lidar, the second lidar, and the third lidar at the same time into the first surface mesh, the second surface mesh, and the third surface mesh respectively; the first lidar, the second lidar, and the third lidar are located on different vehicles and their spacing meets the preset conditions.

[0122] Step 702: Obtain the first difference value between the first surface mesh and the second surface mesh, the second difference value between the first surface mesh and the third surface mesh, and the third difference value between the second surface mesh and the third surface mesh.

[0123] Step 703: Determine the attack detection results for the first lidar and the second lidar based on the relationship between the first difference value and the first difference value threshold; determine the attack detection results for the first lidar and the third lidar based on the relationship between the second difference value and the second difference value threshold; determine the attack detection results for the second lidar and the third lidar based on the relationship between the third difference value and the third difference value threshold.

[0124] Step 704: Use the combination of various attack detection results to identify whether the first lidar, the second lidar, and the third lidar have been attacked.

[0125] For example, taking lidar a (first lidar), lidar b (second lidar), and lidar c (third lidar) as examples; based on the attack detection results, if the combination of lidar a and lidar b is not attacked, the combination of lidar a and lidar c is attacked, and the combination of lidar b and lidar c is attacked, then it indicates that lidar c is attacked, while lidar a and lidar b are both normal (i.e., neither is attacked). This achieves specific detection of whether lidar a, lidar b, and lidar c are individually attacked.

[0126] Similarly, when there are three or more lidar units located on different vehicles and whose spacing meets preset conditions, the above can be referred to. Figure 7 The embodiments shown are not described in detail here.

[0127] Corresponding to the above-described detection method for lidar attacks, this specification also provides a detection device for lidar attacks, which can be configured in the aforementioned autonomous driving system. (Refer to...) Figure 8 As shown, the detection device for lidar attacks may include:

[0128] The conversion module 81 can be used to convert the point cloud datasets collected by the first lidar and the second lidar at the same time into the first surface mesh and the second surface mesh respectively; the first lidar and the second lidar are located on different vehicles and their spacing meets the preset conditions.

[0129] The acquisition module 82 can be used to acquire a first difference value between the first surface mesh and the second surface mesh;

[0130] The determination module 83 can be used to determine the attack detection results for the first lidar and the second lidar based on the relationship between the first difference value and the first difference value threshold.

[0131] In some embodiments of the detection device for lidar attacks, the point cloud datasets collected by the first lidar and the second lidar at the same time are respectively converted into a first surface mesh and a second surface mesh, including:

[0132] The first point cloud dataset collected by the first lidar at the current moment and the second point cloud dataset collected by the second lidar at the current moment are respectively input into the region candidate network to obtain the first candidate box set and the second candidate box set for selecting objects.

[0133] Merge the first candidate box set and the second candidate box set into a target candidate box set;

[0134] In the target candidate box set, determine the first subset of point cloud data corresponding to each candidate box in the first point cloud dataset and the second subset of point cloud data corresponding to each candidate box in the second point cloud dataset;

[0135] The first subset of point cloud data corresponding to all candidate boxes in the target candidate box set in the first point cloud dataset is input into the surface mesh generation model to obtain the first surface mesh; and the second subset of point cloud data corresponding to all candidate boxes in the target candidate box set in the second point cloud dataset is input into the surface mesh generation model to obtain the second surface mesh.

[0136] In some embodiments of the detection device for lidar attacks, the point cloud datasets collected by the first lidar and the second lidar at the same time are respectively converted into a first surface mesh and a second surface mesh, including:

[0137] The first point cloud dataset collected by the first lidar at the current moment and the second point cloud dataset collected by the second lidar at the current moment are respectively input into the region candidate network to obtain the first candidate box set and the second candidate box set for selecting objects.

[0138] Merge the first candidate box set and the second candidate box set into a target candidate box set;

[0139] In the target candidate box set, determine the first subset of point cloud data corresponding to each candidate box in the first point cloud dataset and the second subset of point cloud data corresponding to each candidate box in the second point cloud dataset;

[0140] Based on the symmetry relationship of the objects within each candidate box, the first point cloud data subset and the second point cloud data subset of each candidate box are mirrored to obtain the mirror image of the first point cloud data subset and the mirror image of the second point cloud data subset of each candidate box.

[0141] The first point cloud data subset and the mirror image of the first point cloud data subset of each candidate box are superimposed to obtain a new first point cloud data subset of each candidate box. The second point cloud data subset and the mirror image of the second point cloud data subset of each candidate box are superimposed to obtain a new second point cloud data subset of each candidate box.

[0142] A new first point cloud data subset of all candidate boxes in the target candidate box set is input into the surface mesh generation model to obtain a first surface mesh; and a new second point cloud data subset of all candidate boxes in the target candidate box set is input into the surface mesh generation model to obtain a second surface mesh.

[0143] In some embodiments of the detection apparatus against lidar attacks, acquiring a first difference value between the first surface mesh and the second surface mesh includes:

[0144] Discretize the first surface grid and the second surface grid using squares with a specified side length as units, and use the distance of the center point of each square relative to the ground as the value of that square.

[0145] Determine the average difference between the abnormal square portions corresponding to the same candidate box in the discretized first surface mesh and the discretized second surface mesh.

[0146] In some embodiments of the detection apparatus for lidar attacks, determining the average difference between the abnormal square portions corresponding to the same candidate box in the discretized first surface mesh and the discretized second surface mesh includes:

[0147] The average difference between the outlier square portions corresponding to the same candidate box in the discretized first surface mesh and the discretized second surface mesh is determined using the following formula:

[0148]

[0149] in, Let g be the average difference between the outlier squares corresponding to the i-th candidate box in the discretized first surface mesh and the discretized second surface mesh, and let g be the g-th outlier square in the i-th candidate box. Let G be the set of g. To discretize the value of the g-th anomaly square in the i-th candidate box of the first surface mesh, To discretize the value of the g-th anomalous square in the i-th candidate box of the second surface mesh, |G| is the number of anomalous squares in G, and θ is a preset threshold value used to distinguish normal squares from anomalous squares.

[0150] In some embodiments of the detection device for LiDAR attacks, determining the attack detection results for the first LiDAR and the second LiDAR based on the relationship between the first difference value and a first difference value threshold includes:

[0151] When the first difference value is greater than the first difference value threshold, it is confirmed that at least one of the first lidar and the second lidar has been attacked.

[0152] When the first difference value is not greater than the first difference value threshold, it is confirmed that the first lidar and the second lidar have not been attacked.

[0153] In some embodiments of the detection device for lidar attacks, the conversion module can also be used to: convert the point cloud dataset collected by the third lidar at the same time into a corresponding third surface mesh; the first lidar, the second lidar, and the third lidar are located on different vehicles and their spacing meets a preset condition;

[0154] The acquisition module can also be used to acquire a second difference value between the first surface mesh and the third surface mesh, and a third difference value between the second surface mesh and the third surface mesh;

[0155] The determining module can also be used to determine the attack detection results for the first lidar and the third lidar based on the relationship between the second difference value and the second difference value threshold; determine the attack detection results for the second lidar and the third lidar based on the relationship between the third difference value and the third difference value threshold; and use the combination of various attack detection results to identify whether the first lidar, the second lidar, and the third lidar have been attacked.

[0156] In some embodiments of the detection device for LiDAR attacks, the detection device further includes a processing module; the processing module can be used to discard the corresponding candidate box when it is confirmed that a LiDAR has been attacked.

[0157] For ease of description, the above devices are described in terms of function, divided into various units. Of course, in implementing this specification, the functions of each unit can be implemented in one or more software and / or hardware components.

[0158] Although the process described above includes multiple operations that occur in a specific order, it should be clearly understood that these processes may include more or fewer operations, which may be executed sequentially or in parallel (e.g., using parallel processors or a multithreaded environment).

[0159] Embodiments of this specification also provide a computer device. For example... Figure 9 As shown, in some embodiments of this specification, the computer device 902 may include one or more processors 904, such as one or more central processing units (CPUs) or graphics processing units (GPUs), each of which may implement one or more hardware threads. The computer device 902 may also include any memory 906 for storing any kind of information such as code, settings, data, etc. In one specific embodiment, a computer program is stored on the memory 906 and can run on the processor 904. When the computer program is run by the processor 904, it can execute instructions for the detection method against lidar attacks described in any of the above embodiments. Non-limitingly, for example, the memory 906 may include any type of RAM, any type of ROM, flash memory, hard disk, optical disk, etc. More generally, any memory can use any technology to store information. Further, any memory can provide volatile or non-volatile retention of information. Further, any memory can represent a fixed or removable component of the computer device 902. In one case, when the processor 904 executes associated instructions stored in any memory or combination of memories, the computer device 902 can perform any operation of the associated instructions. The computer device 902 also includes one or more drive mechanisms 908 for interacting with any memory, such as a hard disk drive mechanism, an optical disk drive mechanism, etc.

[0160] Computer device 902 may also include an input / output interface 910 (I / O) for receiving various inputs (via input device 912) and providing various outputs (via output device 914). A specific output mechanism may include a presentation device 916 and an associated graphical user interface 918 (GUI). In other embodiments, the input / output interface 910 (I / O), input device 912, and output device 914 may be omitted, and the device may function solely as a computer device within a network. Computer device 902 may also include one or more network interfaces 920 for exchanging data with other devices via one or more communication links 922. One or more communication buses 924 couple the components described above together.

[0161] Communication link 922 can be implemented in any way, such as via a local area network (LAN), a wide area network (WAN) (e.g., the Internet), a point-to-point connection, or any combination thereof. Communication link 922 may include any combination of hardwired links, wireless links, routers, gateway functions, name servers, etc., governed by any protocol or combination of protocols.

[0162] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), computer-readable storage media, and computer program products according to some embodiments of this specification. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processor to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processor, create a mechanism for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0163] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processor to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0164] These computer program instructions may also be loaded onto a computer or other programmable data processor, causing a series of operational steps to be performed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable device for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0165] In a typical configuration, a computer device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0166] Memory may include non-persistent storage in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of computer-readable media.

[0167] Computer-readable media, including both permanent and non-permanent, removable and non-removable media, can store information using any method or technology. Information can be computer-readable instructions, data structures, program modules, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, disk storage or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by computer equipment. As defined in this specification, computer-readable media does not include transient media, such as modulated data signals and carrier waves.

[0168] Those skilled in the art will understand that the embodiments of this specification can be provided as methods, systems, or computer program products. Therefore, the embodiments of this specification can take the form of entirely hardware embodiments, entirely software embodiments, or embodiments combining software and hardware aspects. Furthermore, the embodiments of this specification can take the form of computer program products implemented on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0169] The embodiments described in this specification can be described in the general context of computer-executable instructions, such as program modules, that are executed by a computer. Generally, program modules include routines, programs, objects, components, data structures, etc., that perform a specific task or implement a specific abstract data type. The embodiments of this specification can also be practiced in distributed computing environments where tasks are performed by remote processors connected via a communication network. In distributed computing environments, program modules can reside in local and remote computer storage media, including storage devices.

[0170] It should also be understood that, in the embodiments of this specification, the term "and / or" is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, and B existing alone. Additionally, the character " / " in this document generally indicates that the preceding and following related objects have an "or" relationship.

[0171] The various embodiments in this specification are described in a progressive manner. Similar or identical parts between embodiments can be referred to interchangeably. Each embodiment focuses on describing the differences from other embodiments. In particular, the system embodiments are basically similar to the method embodiments, so the description is relatively simple; relevant parts can be referred to the descriptions in the method embodiments.

[0172] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., refer to specific features, structures, materials, or characteristics described in connection with that embodiment or example, which are included in at least one embodiment or example of the embodiments of this specification. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of different embodiments or examples.

[0173] The above description is merely an embodiment of this application and is not intended to limit the scope of this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of the claims of this application.

Claims

1. A detection method for lidar attacks, characterized in that, include: The point cloud datasets collected by the first lidar and the second lidar at the same time are respectively converted into the first surface mesh and the second surface mesh. The first lidar and the second lidar are located on different vehicles and their spacing meets preset conditions; Obtain the first difference value between the first surface mesh and the second surface mesh; The first difference value is the average height difference of the abnormal squares corresponding to the same candidate box in the first surface grid and the second surface grid. The abnormal square is a square in which the height difference between the two surface grids at the same square position exceeds a preset critical value. Based on the relationship between the first difference value and the first difference value threshold, the attack detection results for the first lidar and the second lidar are determined.

2. The detection method for lidar attacks as described in claim 1, characterized in that, The point cloud datasets collected by the first and second lidars at the same time are converted into first and second surface meshes respectively, including: The first point cloud dataset collected by the first lidar at the current moment and the second point cloud dataset collected by the second lidar at the current moment are respectively input into the region candidate network to obtain the first candidate box set and the second candidate box set for selecting objects. Merge the first candidate box set and the second candidate box set into a target candidate box set; In the target candidate box set, determine the first subset of point cloud data corresponding to each candidate box in the first point cloud dataset and the second subset of point cloud data corresponding to each candidate box in the second point cloud dataset; The first subset of point cloud data corresponding to all candidate boxes in the target candidate box set in the first point cloud dataset is input into the surface mesh generation model to obtain the first surface mesh; and the second subset of point cloud data corresponding to all candidate boxes in the target candidate box set in the second point cloud dataset is input into the surface mesh generation model to obtain the second surface mesh.

3. The detection method for lidar attacks as described in claim 1, characterized in that, The point cloud datasets collected by the first and second lidars at the same time are converted into first and second surface meshes respectively, including: The first point cloud dataset collected by the first lidar at the current moment and the second point cloud dataset collected by the second lidar at the current moment are respectively input into the region candidate network to obtain the first candidate box set and the second candidate box set for selecting objects. Merge the first candidate box set and the second candidate box set into a target candidate box set; In the target candidate box set, determine the first subset of point cloud data corresponding to each candidate box in the first point cloud dataset and the second subset of point cloud data corresponding to each candidate box in the second point cloud dataset; Based on the symmetry relationship of the objects within each candidate box, the first point cloud data subset and the second point cloud data subset of each candidate box are mirrored to obtain the mirror image of the first point cloud data subset and the mirror image of the second point cloud data subset of each candidate box. The first point cloud data subset and the mirror image of the first point cloud data subset of each candidate box are superimposed to obtain a new first point cloud data subset of each candidate box. The second point cloud data subset and the mirror image of the second point cloud data subset of each candidate box are superimposed to obtain a new second point cloud data subset of each candidate box. A new first point cloud data subset of all candidate boxes in the target candidate box set is input into the surface mesh generation model to obtain a first surface mesh; and a new second point cloud data subset of all candidate boxes in the target candidate box set is input into the surface mesh generation model to obtain a second surface mesh.

4. The detection method for lidar attacks as described in claim 1, characterized in that, Obtaining the first difference value between the first surface mesh and the second surface mesh includes: Discretize the first surface grid and the second surface grid using squares with a specified side length as units, and use the distance of the center point of each square relative to the ground as the value of that square. Determine the average difference between the abnormal square portions corresponding to the same candidate box in the discretized first surface mesh and the discretized second surface mesh.

5. The detection method for lidar attacks as described in claim 4, characterized in that, Determine the average difference between the outlier square portions corresponding to the same candidate box in the discretized first surface mesh and the discretized second surface mesh, including: The average difference between the outlier square portions corresponding to the same candidate box in the discretized first surface mesh and the discretized second surface mesh is determined using the following formula: in, To discretize the first surface mesh and the second surface mesh, the first... The average difference between the abnormal squares corresponding to each candidate box For the first The first candidate box An abnormal square, for The set, To discretize the first surface mesh in the first surface mesh The first candidate box The values ​​of the abnormal squares, To discretize the second surface mesh The first candidate box The values ​​of the abnormal squares, for The number of abnormal squares in the data. This is a preset threshold value used to distinguish between normal and abnormal squares.

6. The detection method for lidar attacks as described in claim 1, characterized in that, Based on the relationship between the first difference value and the first difference value threshold, the attack detection results for the first lidar and the second lidar are determined, including: When the first difference value is greater than the first difference value threshold, it is confirmed that at least one of the first lidar and the second lidar has been attacked. When the first difference value is not greater than the first difference value threshold, it is confirmed that the first lidar and the second lidar have not been attacked.

7. The detection method for lidar attacks as described in claim 1, characterized in that, The method further includes: The point cloud datasets collected by the third lidar at the same time are converted into a third surface mesh; the first lidar, the second lidar, and the third lidar are located on different vehicles and their spacing meets a preset condition. Obtain a second difference value between the first surface mesh and the third surface mesh, and a third difference value between the second surface mesh and the third surface mesh; Based on the relationship between the second difference value and the second difference value threshold, the attack detection results for the first lidar and the third lidar are determined; based on the relationship between the third difference value and the third difference value threshold, the attack detection results for the second lidar and the third lidar are determined. The first, second, and third lidars are identified as being attacked by combining the results of each attack detection.

8. The detection method for lidar attacks as described in claim 6 or 7, characterized in that, The method further includes: When it is confirmed that a lidar has been attacked, the corresponding candidate box is discarded.

9. A detection device for laser radar attacks, characterized in that, include: The conversion module is used to convert the point cloud datasets collected by the first lidar and the second lidar at the same time into the first surface mesh and the second surface mesh respectively. The first lidar and the second lidar are located on different vehicles and their spacing meets preset conditions; The acquisition module is used to acquire a first difference value between the first surface mesh and the second surface mesh; The first difference value is the average height difference of the abnormal squares corresponding to the same candidate box in the first surface grid and the second surface grid. The abnormal square is a square in which the height difference between the two surface grids at the same square position exceeds a preset critical value. The determination module is used to determine the attack detection results for the first lidar and the second lidar based on the relationship between the first difference value and the first difference value threshold.

10. A computer device comprising a memory, a processor, and a computer program stored in the memory, characterized in that, When the computer program is run by the processor, it executes the instructions of the method according to any one of claims 1-8.

11. A computer storage medium having a computer program stored thereon, characterized in that, When the computer program is run by the processor of the computer device, it executes the instructions of the method according to any one of claims 1-8.

12. A computer program product, characterized in that, The computer program product includes a computer program that, when executed by a processor, implements the method of any one of claims 1 to 8.

Images