A method, apparatus, device and medium for detecting and locating a sensor attack

Abstract

The application relates to the technical field of unmanned driving and discloses a sensor attack detection and positioning method, a sensor attack detection and positioning device, equipment and a medium, which comprises the following steps: analyzing target detection results of multiple sensors to obtain tracking identifiers, observation angles, yaw angles and multiple detection boxes corresponding to the multiple sensors; selecting the detection boxes corresponding to different sensors to calculate inconsistencies between the different sensors and obtain multiple inconsistencies; inputting the tracking identifiers, the observation angles, the yaw angles and the multiple inconsistencies of the previous moment into a Gaussian process-based inconsistency estimation model to obtain a confidence space of the multiple inconsistencies of the current moment; and judging whether the multiple inconsistencies of the current moment are within the confidence space to detect whether the sensors are attacked and to position the attack. The application can reduce modeling time and consumption of computing resources, improve the robustness of the model, and effectively and timely detect and position attacks on multiple sensors.

Description

Technical Field

[0001] This invention relates to the field of autonomous driving technology, specifically to a method, apparatus, device, and medium for detecting and locating sensor attacks. Background Technology

[0002] Autonomous driving technology is a comprehensive technology involving many cutting-edge technologies such as artificial intelligence, sensing technology, mapping technology, and computer science. Autonomous driving relies heavily on onboard sensors, which are highly vulnerable to external attacks or interference, leading to unreliability. Attacking sensors is a simple, direct, brute-force, and effective method because it does not require entering the autonomous driving system, making it very easy to access and posing a significant security threat to unmanned systems.

[0003] Existing technologies mainly include three methods for detecting and locating sensor attacks: detection methods focused on the functional level, data-driven detection methods, and data integrity reasoning and judgment detection methods. However, detection methods focused on the functional level mainly target a single sensor and cannot perform attack detection in real time; data-driven detection methods are time-consuming and have high data requirements, and also cannot perform attack detection in real time; data integrity reasoning and judgment detection methods require further uncertainty assessment, separating attack detection and attack location, and cannot guarantee the real-time performance of attack detection and location. Summary of the Invention

[0004] In view of this, the present invention provides a method, apparatus, device and medium for detecting and locating sensor attacks, so as to solve the problem that the prior art cannot detect and locate sensor attacks in real time.

[0005] In a first aspect, the present invention provides a method for detecting and locating sensor attacks, the method comprising:

[0006] Acquire target detection results from multiple sensors, and analyze the target detection results to obtain the target's tracking identifier, observation angle, yaw angle, and multiple detection boxes corresponding to the various sensors;

[0007] By selecting detection frames corresponding to different sensors, the inconsistencies between different sensors are calculated, and multiple inconsistencies are obtained.

[0008] The tracking identifier, observation angle, yaw angle, and multiple inconsistencies from the previous moment are input into a pre-built inconsistency estimation model based on Gaussian processes to obtain the confidence space of multiple inconsistencies at the current moment.

[0009] By determining whether multiple inconsistencies are within the confidence space at the current moment, the system can detect whether multiple sensors are under attack and locate the attack.

[0010] The sensor attack detection and localization method provided in this invention obtains the target's tracking identifier, observation angle, yaw angle, and multiple detection boxes corresponding to various sensors by acquiring target detection results from multiple sensors. Based on these detection boxes, inconsistencies between different sensors are calculated. The tracking identifier, observation angle, yaw angle, and inconsistencies from the previous time step are input into a Gaussian process-based inconsistency estimation model to obtain the confidence space of inconsistencies at the current time step. Sensor attacks are detected and localized based on whether multiple inconsistencies at the current time step fall within this confidence space. This invention uses a Gaussian process to perform non-parametric modeling and analysis of the spatial correlation between sensors, eliminating the need for numerous hyperparameters, reducing modeling time and computational resource consumption, improving model robustness, and enabling effective real-time attack detection and localization for attacks targeting multiple sensors.

[0011] In one optional implementation, the process of acquiring target detection results from multiple sensors and analyzing these results to obtain the target's tracking identifier, observation angle, yaw angle, and multiple detection frames corresponding to the various sensors includes: acquiring target detection results from a lidar; analyzing the target detection results using a first preset algorithm to obtain the lidar 3D detection frame and yaw angle of the target, and representing the lidar 3D detection frame using the center point coordinates, length, width, and height; converting the lidar 3D detection frame into a lidar 2D detection frame through coordinate transformation; and acquiring target detection results from a binocular camera. As a result, the target detection results are analyzed using a second preset algorithm to obtain the target's 2D detection frame, which is represented by the center point coordinates, width, and height of the 2D detection frame. The 2D detection frame includes a first 2D detection frame and a second 2D detection frame. The observation angle of the target is calculated based on the center point coordinates, width, and height of the 2D detection frame. The target is tracked by performing feature extraction, target association, motion prediction, and data association on the target within the 2D detection frame at different times. When the target is detected in consecutive time periods, the target's tracking identifier is obtained.

[0012] This invention can acquire various information about the target by using the detection results of multiple sensors, and can perform sensor attack detection based on the various information. Moreover, the process of acquiring various information is simple and easy to implement, thus reducing the consumption of computing resources for attack detection and localization.

[0013] In one optional implementation, the process of selecting detection frames corresponding to different sensors and calculating inconsistencies between different sensors to obtain multiple inconsistencies includes: selecting the LiDAR 2D detection frame of the LiDAR and any camera 2D detection frame of the binocular camera; calculating the intersection and union between the LiDAR 2D detection frame and any camera 2D detection frame, and obtaining a first inconsistency between the LiDAR and the binocular camera based on the ratio of the intersection and union; selecting the first camera 2D detection frame and the second camera 2D detection frame of the binocular camera; calculating the intersection and union between the first camera 2D detection frame and the second camera 2D detection frame, and obtaining a second inconsistency between the two cameras in the binocular camera based on the ratio of the intersection and union.

[0014] This invention utilizes the cross-correlation of sensors to calculate the inconsistencies between different sensors, thereby obtaining the spatial correlation between sensors and enabling the detection and location of sensor attacks based on changes in spatial correlation.

[0015] In one optional implementation, the process of inputting the tracking identifier, observation angle, yaw angle, and multiple inconsistencies from the previous time step into a pre-constructed Gaussian process-based inconsistency estimation model to obtain the confidence spaces of multiple inconsistencies at the current time step includes: acquiring multiple sets of datasets corresponding to multiple targets, each set of datasets including the tracking identifier, observation angle, yaw angle, first inconsistency, and second inconsistency of the same target; standardizing the data in each set of datasets; constructing a Gaussian process inconsistency evaluation model, which includes a mean function, a covariance function, and a noise term following a Gaussian distribution; using the multiple sets of datasets from the previous time step as prediction inputs, and using the first and second inconsistencies in the multiple sets of datasets at the current time step as prediction outputs, obtaining the joint distribution, marginal distribution, and conditional distribution of the Gaussian process model; and obtaining the first confidence space and the second confidence space of the first inconsistency at the current time step based on the joint distribution, marginal distribution, and conditional distribution.

[0016] This invention utilizes the autocorrelation of sensor time series data and employs a Gaussian process inconsistency evaluation model to predict the inconsistency confidence space of the current sensor based on the inconsistency of the previous time step. This enables the detection and localization of sensor attacks based on the confidence space. Furthermore, using Gaussian processes for non-parametric modeling of target detection results provides a flexible framework for modeling complex and nonlinear relationships in the data, improving model robustness and enabling the detection and localization of various sensor attacks, resulting in more comprehensive and broader functionality. Moreover, Gaussian process-based modeling does not require training a large number of hyperparameters, reducing training time and minimizing the consumption of limited computing resources in autonomous driving, thus achieving real-time detection and localization.

[0017] In one optional implementation, the process of detecting whether multiple sensors are under attack and locating the attack by determining whether multiple inconsistencies are within a confidence space at the current moment includes: sequentially determining whether a first inconsistency is within a first confidence space and whether a second inconsistency is within a second confidence space; detecting whether multiple sensors are under attack based on the determination results and locating the attack; wherein, if both the first and second inconsistencies are within a confidence space at the current moment, then neither the LiDAR nor the binocular camera is under attack; and / or, if neither the first nor the second inconsistency is within a confidence space at the current moment, then at least one of the LiDAR and the binocular camera is under attack; and / or, if the first inconsistency is not within a confidence space and the second inconsistency is within a confidence space at the current moment, then the LiDAR is under attack and the binocular camera is not under attack; and / or, if the first inconsistency is within a confidence space and the second inconsistency is not within a confidence space at the current moment, then the binocular camera is under attack and the LiDAR is not under attack.

[0018] This invention performs sensor attack detection and location judgment based on pre-set rules, which can reduce time consumption and further improve real-time performance.

[0019] In one optional implementation, the process of converting the 3D detection frame of the LiDAR into a 2D detection frame of the LiDAR through coordinate transformation includes: obtaining the vertex coordinates of multiple vertices of the 3D detection frame of the LiDAR based on the center point coordinates, length, width, and height; converting the vertex coordinates of multiple vertices of the 3D detection frame of the LiDAR into multiple homogeneous coordinates in the image coordinate system based on the rotation matrix of the image coordinate system of the LiDAR to the binocular camera, the camera projection matrix of the binocular camera, and the camera rotation matrix; and determining the 2D detection frame of the LiDAR based on the maximum value, minimum value, maximum value, and minimum value along the horizontal axis, and along the vertical axis of the multiple homogeneous coordinates.

[0020] This invention converts 3D detection boxes into 2D detection boxes and obtains the coordinate information of the detection boxes. It can obtain the overlap ratio between detection boxes based on the coordinate information, and then calculate the inconsistency between sensors based on the overlap ratio, which is equivalent to judging the spatial correlation between sensors.

[0021] In one optional implementation, the process of obtaining the first inconsistency between the LiDAR and the binocular camera by calculating the intersection and union between the LiDAR 2D detection frame and any camera 2D detection frame, and based on the ratio of the intersection and union, includes: taking the intersection of the LiDAR 2D detection frame and any camera 2D detection frame as the first intersection, and taking the union of the LiDAR 2D detection frame and any camera 2D detection frame as the first union; obtaining the coordinates of the upper left and lower right corners of the first intersection; calculating the area of ​​the first intersection based on the coordinates of the upper left and lower right corners of the first intersection; calculating the area of ​​the LiDAR 2D detection frame based on the coordinate information of the LiDAR 2D detection frame, and calculating the area of ​​any camera 2D detection frame based on the coordinate information of the camera 2D detection frame; subtracting the area of ​​the first intersection from the total area between the areas of the LiDAR 2D detection frame and the area of ​​any camera 2D detection frame to obtain the area of ​​the first union; and using the ratio of the area of ​​the first intersection to the area of ​​the first union as the first inconsistency between the LiDAR and the binocular camera; and Alternatively, the process of calculating the intersection and union of the first and second camera 2D detection frames, and obtaining the second inconsistency between the two cameras in the binocular camera system based on the ratio of the intersection and union, includes: taking the intersection of the first and second camera 2D detection frames as the second intersection, and taking the combination of the first and second camera 2D detection frames as the second union; obtaining the coordinates of the upper left and lower right corners of the second intersection; calculating the area of ​​the second intersection based on the coordinates of the upper left and lower right corners; calculating the area of ​​the first and second camera 2D detection frames based on the coordinate information of the first camera 2D detection frames, and calculating the area of ​​the second camera 2D detection frames based on the coordinate information of the second camera 2D detection frames; subtracting the area of ​​the second intersection from the total area between the areas of the first and second camera 2D detection frames to obtain the area of ​​the second union; and using the ratio of the area of ​​the second intersection to the area of ​​the second union as the second inconsistency between the two cameras in the binocular camera system.

[0022] This invention uses the ratio of the area of ​​the intersecting portion to the area of ​​the combined portion between detection frames as the overlap ratio between sensors. This overlap ratio characterizes the inconsistency between sensors. The calculation method is simple and can determine the attacked sensor based on the changes in inconsistency during attack detection, thereby realizing sensor attack localization.

[0023] Secondly, the present invention provides a sensor attack detection and location device, the device comprising:

[0024] The data acquisition module is used to acquire target detection results from multiple sensors and analyze the target detection results to obtain the target's tracking identifier, observation angle, yaw angle, and multiple detection boxes corresponding to the various sensors.

[0025] The inconsistency calculation module is used to select the detection boxes corresponding to different sensors, calculate the inconsistency between different sensors, and obtain multiple inconsistencies.

[0026] The model prediction module is used to input the tracking identifier, observation angle, yaw angle and multiple inconsistencies from the previous time step into a pre-built inconsistency estimation model based on Gaussian process to obtain the confidence space of multiple inconsistencies at the current time step.

[0027] The attack detection module is used to detect whether multiple sensors are under attack by determining whether multiple inconsistencies are within the confidence space at the current moment, and to locate the attack.

[0028] The sensor attack detection and localization device provided in this invention acquires the target's tracking identifier, observation angle, yaw angle, and multiple detection boxes corresponding to various sensors by obtaining target detection results from multiple sensors. Based on these detection boxes, it calculates the inconsistencies between different sensors. The tracking identifier, observation angle, yaw angle, and inconsistencies from the previous moment are input into a Gaussian process-based inconsistency estimation model to obtain the confidence space of the inconsistencies at the current moment. The sensor attack is detected and located based on whether the multiple inconsistencies at the current moment fall within this confidence space. This invention uses a Gaussian process to perform non-parametric modeling and analysis of the spatial correlation between sensors, eliminating the need for numerous hyperparameters, reducing modeling time and computational resource consumption, improving model robustness, and enabling effective real-time attack detection and localization for attacks targeting multiple sensors.

[0029] Thirdly, the present invention provides a computer device, comprising: a memory and a processor, wherein the memory and the processor are communicatively connected to each other, the memory stores computer instructions, and the processor executes the computer instructions to perform the sensor attack detection and location method of the first aspect or any corresponding embodiment described above.

[0030] Fourthly, the present invention provides a computer-readable storage medium storing computer instructions for causing a computer to execute the sensor attack detection and localization method of the first aspect or any corresponding embodiment described above. Attached Figure Description

[0031] To more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the specific embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0032] Figure 1 This is a flowchart illustrating a sensor attack detection and localization method according to an embodiment of the present invention;

[0033] Figure 2 This is a schematic diagram illustrating the specific operation of a sensor attack detection and localization method according to an embodiment of the present invention.

[0034] Figure 3 This is a flowchart illustrating another sensor attack detection and localization method according to an embodiment of the present invention;

[0035] Figure 4 This is a schematic diagram of lidar target detection in another sensor attack detection and localization method according to an embodiment of the present invention;

[0036] Figure 5 This is a schematic diagram of lidar coordinate transformation for another sensor attack detection and localization method according to an embodiment of the present invention;

[0037] Figure 6 This is a schematic diagram of binocular camera target detection in another sensor attack detection and localization method according to an embodiment of the present invention.

[0038] Figure 7 This is a flowchart illustrating another sensor attack detection and localization method according to an embodiment of the present invention;

[0039] Figure 8 This is a schematic diagram of the overlap ratio of another sensor attack detection and localization method according to an embodiment of the present invention;

[0040] Figure 9 This is a schematic diagram of the sensor installation position in another sensor attack detection and location method according to an embodiment of the present invention;

[0041] Figure 10 This is a schematic diagram of a laser radar simulated attack detection method for another sensor attack detection and localization method according to an embodiment of the present invention;

[0042] Figure 11 This is a schematic diagram illustrating the attack time consumption of another sensor attack detection and localization method according to an embodiment of the present invention;

[0043] Figure 12 This is a schematic diagram of a binocular camera simulated attack detection method for another sensor attack detection and localization method according to an embodiment of the present invention;

[0044] Figure 13 This is a structural block diagram of a sensor attack detection and location device according to an embodiment of the present invention;

[0045] Figure 14 This is a schematic diagram of the hardware structure of a computer device according to an embodiment of the present invention. Detailed Implementation

[0046] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0047] This invention is applicable to scenarios involving the detection and localization of attacks on the onboard sensors of autonomous vehicles. Currently, related technologies include:

[0048] (1) This approach focuses on the functional level, such as environmental perception technology and sensor fusion technology. This method primarily targets individual sensors in its research on sensor security and reliability, implementing improvements and defensive measures to enhance sensor robustness and reduce the impact of attacks to some extent. For example, enhancing LiDAR data by adding point confidence scores to frontal views improves defensive capabilities. However, it does not monitor sensors in real time, and a comprehensive and systematic security detection method for perception systems has not yet been developed. This functional-level approach mainly improves the defensive performance against attacks on individual sensors, but it has significant limitations. Performance improvements are limited, and it cannot respond to new attacks. For example, while enhancing the defensive performance of cameras, the camera's light sensitivity cannot be changed, and attackers can attack the LiDAR to attack the camera and thus attack the detection system.

[0049] (2) Data-driven methods. These methods use data-driven techniques such as neural networks and deep learning, inputting a large amount of recognition results from the detection system for training, and finally obtaining a model. The trained model is used to judge whether the process or result of the detection system is correct. For example, the PreNet deep learning model is used to predict the current sensor data based on historical data, and the difference between the predicted data and the real data is used to judge whether an attack has occurred. These methods are very time-consuming and have high data requirements. The training process of existing data-driven methods requires training a large number of hyperparameters, which will increase the training time and consume the limited computing resources of autonomous driving, making it difficult to achieve real-time detection and localization. For example, models such as neural networks and support vector machines require training architecture hyperparameters, regularization hyperparameters, and learning rate, while support vector machines also require training kernel hyperparameters and regularization parameters. Moreover, existing data-driven methods rely on a large amount of data, but the abnormal data of autonomous driving is very limited, so they will perform poorly on small datasets due to insufficient data.

[0050] (3) Methods for reasoning and judging the integrity and correctness of data. For example, there are not many point clouds behind a normal object, so if a detected target has a large number of laser point cloud points behind it, then the object may not exist. For example, a normal object has a regular shadow area in the lidar point cloud; if the shadow area of ​​the object is abnormal, then the object may not exist. Existing methods for reasoning and judging the integrity and correctness of data require additional steps to estimate uncertainty. For example, while giving the detection result, a probability assessment of the judgment result is also performed, that is, attack detection and attack location are performed separately, or even attack location is confirmed. This is done because subsequent implementation needs to consider issues such as communication latency, and real-time performance cannot be guaranteed.

[0051] Therefore, embodiments of the present invention provide a method for detecting and locating sensor attacks. This method uses Gaussian process modeling to perform temporal prediction of sensor spatial inconsistencies, achieving real-time detection and location of sensor attacks. It should be noted that the steps shown in the flowcharts can be executed in a computer system, such as a set of computer-executable instructions. Furthermore, although a logical order is shown in the flowcharts, in some cases, the steps shown or described may be executed in a different order than that presented here.

[0052] This embodiment provides a method for detecting and locating sensor attacks, which can be used in the aforementioned computer. Figure 1This is a flowchart of a sensor attack detection and localization method according to an embodiment of the present invention, such as... Figure 1 As shown, the process includes the following steps:

[0053] Step S101: Obtain target detection results from multiple sensors, and analyze the target detection results to obtain the target's tracking identifier, observation angle, yaw angle, and multiple detection boxes corresponding to the multiple sensors.

[0054] Specifically, in this embodiment of the invention, the autonomous vehicle is equipped with a lidar and a binocular camera. During autonomous driving, it can acquire a wealth of target information, including but not limited to: distance, angle, reflection intensity, and speed, thereby generating a multi-dimensional image of the target. This embodiment of the invention acquires the target detection results from the lidar and the binocular camera, and processes them using a corresponding preset algorithm to obtain the target's tracking identifier Id, observation angle alpha, yaw angle yaw, one lidar 2D detection frame, and two camera 2D detection frames.

[0055] Step S102: Select the detection frames corresponding to different sensors to calculate the inconsistencies between different sensors and obtain multiple inconsistencies.

[0056] Specifically, in this embodiment of the invention, the inconsistency between sensors is characterized by the Intersection over Union (IoU) ratio between the detection frames corresponding to different sensors, taking into account the spatial correlation between sensors. For example... Figure 2 As shown, since the sensors in this embodiment of the invention include a LiDAR and a binocular camera, wherein the binocular camera includes two cameras, either the LiDAR or the binocular camera is selected as the first set of sensors, and both cameras are selected as the second set of sensors. Then, the Inconsistency Extraction algorithm is used to calculate the inconsistency between the first set of sensors and the second set of sensors based on the LiDAR detection frame and the camera detection frame, respectively obtaining the first inconsistency IoU1 between the LiDAR and the camera and the second inconsistency IoU2 between the two cameras. The LiDAR and the camera on the left side of the binocular camera selected as the first set of sensors in this embodiment of the invention are merely examples and not intended to be limiting.

[0057] Step S103: Input the tracking identifier, observation angle, yaw angle and multiple inconsistencies from the previous moment into the pre-built inconsistency estimation model based on Gaussian process to obtain the confidence space of multiple inconsistencies at the current moment.

[0058] Specifically, in this embodiment of the invention, a Gaussian process-based inconsistency estimation model (GP-based inconsistency estimation model) is pre-constructed. The Gaussian process (GP) is a Bayesian machine learning model capable of predicting distributions based on given data input points. This embodiment of the invention utilizes the autocorrelation of sensor time series data to predict inconsistent distributions, thereby comprehensively considering the spatiotemporal correlation of sensors and enabling sensor attack detection and localization based on this correlation. Furthermore, modeling with Gaussian processes allows for expanded training datasets, resulting in higher accuracy. Because Gaussian processes can interpolate between data points, they provide reasonable predictions in areas with scarce or unavailable data, significantly enriching the data and improving model accuracy. Gaussian processes can expand the limited data samples for autonomous driving to obtain richer training data, leading to higher model accuracy and further enhancing the data and model's overall accuracy. This embodiment of the invention utilizes the covariance structure encoded in the training data for prediction, providing meaningful results even with limited observations. Because autonomous vehicles operate under safe conditions most of the time, abnormal attacks are extremely rare, resulting in small datasets. Therefore, this invention can effectively handle small datasets, making it suitable for autonomous driving scenarios where collecting large amounts of data is expensive or time-consuming. Furthermore, the training process of the model based on Gaussian processes does not require training a large number of hyperparameters, which reduces training time and the consumption of limited computing resources for autonomous driving, enabling real-time detection and localization. This is because the Gaussian process model typically only requires training the hyperparameter kernel function to achieve non-parametric modeling. Gaussian process models can efficiently process large-scale data and provide real-time inference and decision-making through methods such as sparse matrix factorization.

[0059] In some optional implementations, the inconsistency evaluation model based on Gaussian processes constructed in this embodiment of the invention has five inputs and two outputs. The inputs are the tracking identifier Id of the same target at the previous time step, the observation angle alpha, the yaw angle yaw, the first inconsistency IoU1 between the lidar and the left camera in the binocular camera, and the second inconsistency IoU2 between the two cameras in the binocular camera. The outputs are the confidence spaces of the first inconsistency IoU1 and the second inconsistency IoU2 at the current time step. This embodiment of the invention performs uncertainty estimation while making inference judgments, because Gaussian processes can provide confidence estimation information and reasonable uncertainty estimates. At the same time, Gaussian processes can perform high-precision non-parametric modeling of the relationship between detection results and angles, enabling attack localization during detection.

[0060] Step S104: By determining whether multiple inconsistencies are within the confidence space at the current moment, detect whether multiple sensors are under attack and locate the attack.

[0061] Specifically, in this embodiment of the invention, a judgment rule for sensor attack detection is predefined. After obtaining the confidence space of the first inconsistency IoU1 and the second inconsistency IoU2 at the current time, the system detects whether multiple sensors are under attack by determining whether the first inconsistency IoU1 or the second inconsistency IoU2 is within the confidence space at the current time, and locates the attack. The judgment rule is shown in the table below:

[0062]

[0063] In some alternative implementations, when neither the first inconsistency IoU1 nor the second inconsistency IoU2 is within the corresponding confidence space, only the presence of the sensor can be detected, but the specific attack type cannot be detected, and further inspection of the three sensors is required.

[0064] The sensor attack detection and localization method provided in this invention obtains the target's tracking identifier, observation angle, yaw angle, and multiple detection boxes corresponding to various sensors by acquiring target detection results from multiple sensors. Based on these detection boxes, inconsistencies between different sensors are calculated. The tracking identifier, observation angle, yaw angle, and inconsistencies from the previous time step are input into a Gaussian process-based inconsistency estimation model to obtain the confidence space of inconsistencies at the current time step. Sensor attacks are detected and localized based on whether multiple inconsistencies at the current time step fall within this confidence space. This invention uses a Gaussian process to perform non-parametric modeling and analysis of the spatial correlation between sensors, eliminating the need for numerous hyperparameters, reducing modeling time and computational resource consumption, improving model robustness, and enabling effective real-time attack detection and localization for attacks targeting multiple sensors.

[0065] This embodiment provides a method for detecting and locating sensor attacks, which can be used in the aforementioned computer. Figure 3 This is a flowchart of a sensor attack detection and localization method according to an embodiment of the present invention, such as... Figure 3 As shown, the process includes the following steps:

[0066] Step S301: Obtain target detection results from multiple sensors, and analyze the target detection results to obtain the target's tracking identifier, observation angle, yaw angle, and multiple detection boxes corresponding to the multiple sensors.

[0067] Specifically, step S301 includes:

[0068] Step S3011: Obtain the target detection results from the lidar.

[0069] Specifically, in this embodiment of the invention, the autonomous vehicle includes a lidar and a binocular camera, therefore, as Figure 2 As shown, during the operation of the autonomous vehicle, detection images from LiDAR and binocular cameras are acquired in real time, with the binocular camera detection images consisting of two parts.

[0070] Step S3012: The target detection result is analyzed using the first preset algorithm to obtain the target's 3D detection frame and yaw angle, and the 3D detection frame is represented by the center point coordinates, length, width and height of the 3D detection frame.

[0071] Specifically, in embodiments of the present invention, such as Figure 4 As shown, the LiDAR uses the PointPillars (3D Object Proposal Generation and Detection from Point Cloud) algorithm to obtain 3D object detection boxes, but it is not limited to this. The algorithm input data is the raw point cloud data from the LiDAR, in the format [x, y, z, intensity]. The raw features of the LiDAR are a set of point clouds, which can be represented as vectors. (64-line LiDAR, 1800 point clouds per line scan per revolution), where x represents the horizontal coordinate of the point cloud, y represents the vertical coordinate, z represents the height, and intensity represents the reflection intensity. A two-stage approach is used, employing PointNet++ as the backbone network. First, segmentation is performed to determine the label of each 3D point. For each point classified as foreground, a bounding box is generated using features. Then, the bounding box is optimized by cropping the region of interest, resulting in the LiDAR 3D detection bounding box. The LiDAR 3D detection bounding box uses vectors. It means that X v Y v Z v This represents the center point coordinates of the target object in the lidar coordinate system, [L, W, H]. T This indicates the length, width, and height of the detection frame.

[0072] Step S3013: Convert the 3D detection box of the LiDAR into a 2D detection box of the LiDAR through coordinate transformation.

[0073] Specifically, in this embodiment of the invention, the obtained 3D detection box of the LiDAR is projected onto the image coordinate system, thereby converting it into a 2D detection box of the LiDAR.

[0074] In some optional implementations, step S3013 above includes:

[0075] Step a1: Obtain the vertex coordinates of multiple vertices of the LiDAR 3D detection frame based on the center point coordinates, length, width, and height.

[0076] Step a2: Based on the rotation matrix of the image coordinate system of the LiDAR to the binocular camera, the camera projection matrix of the binocular camera, and the camera rotation matrix, convert the vertex coordinates of multiple vertices of the LiDAR 3D detection box into multiple homogeneous coordinates in the image coordinate system.

[0077] Step a3: Determine the 2D detection box of the LiDAR based on the maximum value, minimum value, maximum value and minimum value along the horizontal axis, and maximum value along the vertical axis in multiple homogeneous coordinates.

[0078] Specifically, in embodiments of the present invention, such as Figure 5 As shown, the LiDAR 3D detection box contains eight vertices. These eight vertices are then transformed to obtain eight points in the camera coordinate system. The coordinates of these eight points are shown below:

[0079] Y 第1个6D检测框点齐次坐标 =[X v,1 ,Y v,1 Z v,1 ] T =[X v -W / 2,Y v +L / 2,Z v +H / 2]

[0080] Y 第2个6D融合框点齐次坐标 =[X v,2 ,Y v,2 Z v,2 ] T =[X v -W / 2,Y v +L / 2,Z v -H / 2]

[0081] Y 第3个3D检测框点齐次坐标 =[X v,3 ,Y v,3 Z v,3 ] T =[X v -W / 2,Y v -L / 2,Z v +H / 2]

[0082] Y 第4个3D检测框点齐次坐标 =[X v,4 ,Y v,4 Z v,4 ] T =[X v -W / 2,Y v-L / 2,Z v -H / 2]

[0083] Y 第5个3D检测框点齐次坐标 =[X v,51 ,Y v,5 Z v,5 ] T =[X v +W / 2,Y v +L / 2,Z v -H / 2]

[0084] Y 第6个3D检测框点齐次坐标 =[X v,6 ,Y v,6 Z v,6 ] T =[X v +W / 2,Y v +L / 2,Z v +H / 2]

[0085] Y 第7个3D检测框点齐次坐标 =[X v,7 ,Y v,7 Z v,7 ] T =[X v +W / 2,Y v -l / 2,Z v +H / 2]

[0086] Y 第8个3D检测框点齐次坐标 =[X v,8 ,Y v,8 Z v,8 ] T =[X v +W / 2,Y v -L / 2,Z v -H / 2]

[0087] In some alternative implementations, the coordinate system transformation formula is as follows:

[0088] Y i =P*R*Tr v elo t o c am*Y 第i个3D检测框点齐次坐标

[0089] Among them, Y 第i个3D融合检测齐次坐标 =[X v,i Y v,i Z v,i ] T The homogeneous coordinates of the target positions within the LiDAR 3D detection bounding box are represented by i = 1, 2, 3, ..., 8, which represent the eight vertices of the LiDAR 3D detection bounding box. P is the camera projection matrix. R is the camera rotation matrix. Trv elo t o c am is the rotation matrix from the 3x4 laser radar to the camera coordinate system. In this embodiment of the invention, Y = [uv 1] is used. T This represents the homogeneous coordinates of the LiDAR 2D detection box projected onto the image coordinate system, where u and v represent the coordinates of the image coordinate system in the x and y directions, respectively.

[0090] In one optional implementation, after coordinate transformation, eight points corresponding to the eight vertices of the LiDAR 3D detection box can be obtained. The maximum value in the x-direction, the maximum value in the y-direction, the minimum value in the x-direction, and the minimum value in the y-direction are selected from these eight points, and these four values ​​are used to represent the LiDAR 2D detection box, as shown in the following expression:

[0091] Y 激光雷达2d检测框 =[min(u),min(v),max(u),max(v)]

[0092] In one alternative implementation, the lidar obtains multiple 3D detection boxes at each moment, so the embodiment of the present invention calculates the yaw angle of the target based on the multiple detection boxes.

[0093] Step S3014: Obtain the target detection results from the binocular camera.

[0094] Step S3015: The target detection result is analyzed using the second preset algorithm to obtain the camera 2D detection frame of the target. The camera 2D detection frame is represented by the center point coordinates, width and height of the camera 2D detection frame. The camera 2D detection frame includes a first camera 2D detection frame and a second camera 2D detection frame.

[0095] Specifically, in embodiments of the present invention, such as Figure 6 As shown, the YOLOv3 algorithm is used to perform 2D object detection on images captured by two cameras in a stereo camera system to obtain the target center point, but this is not a limitation. In this embodiment of the invention, the captured images are input into the object detection system in RGB format. The detection system calls a deep convolutional neural network algorithm to extract features from the RGB images. Common structures in neural network algorithms include convolutional layers, pooling layers, activation layers, dropout layers, batch normalization (BN) layers, or fully connected layers. The features extracted from the images effectively describe the information of the target object. The input data format is the RGB image from the camera, and the output is the coordinates of the center point of the 2D detection box of the camera in the image coordinate system, along with its width and height, in the following format: Where X camera Represents the value in the x-direction of the image coordinate system, Y... camerarepresents the value in the y-direction, w represents the width of the camera's 2D detection box, and h represents the height of the camera's 2D detection box.

[0096] Step S3016: Calculate the observation angle of the target based on the center point coordinates, width, and height of the camera's 2D detection frame.

[0097] Specifically, in this embodiment of the invention, the observation angle alpha of the target is a parameter used to describe the orientation angle of the target object on the horizontal plane, representing the rotation angle of the object's bounding box relative to the horizontal axis of the image, and the calculation formula is as follows:

[0098] alpha = arctan2(X) camera -image width / 2,focal lengthx -arctan2((w / 2)*sin(alpha), lengthx )

[0099] Among them, image width The width of the image. α is the focal length of the camera. The focal lengths of the two cameras are 7.070493000000e+02 and 7.215377000000e+02, respectively. sin(alpha) is the aspect ratio of the 2D detection box of the target object.

[0100] Step S3017: Track the target by performing feature extraction, target association, motion prediction, and data association on the target within the 2D detection frame of the camera at different times. When the target is detected in a continuous time period, the tracking identifier of the target is obtained.

[0101] Specifically, in this embodiment of the invention, identifying the tracking identification (Tracking ID) of targets is crucial in multi-target detection or tracking scenarios. The tracking ID uniquely identifies and distinguishes each target, ensuring accurate tracking and analysis. Through the tracking ID, information such as the target's position, angle, and trajectory can be collected for deeper analysis and understanding. In multi-target scenarios, targets often occlude or come into contact with each other, causing their trajectories to intersect or change. The tracking ID allows for the association of targets across different frames, ensuring accurate tracking. If a target is lost or re-enters the field of view during tracking, the tracking ID makes it easier to identify the target and associate it with previous tracking results, thus preventing incorrect re-identification or confusion with other targets.

[0102] In some optional implementations, embodiments of the present invention utilize the DeepSORT algorithm to calculate the tracking identifier for each target, as follows: 1. Target Detection: First, the DeepSORT algorithm uses 2D target detection to detect target objects in video frames. The target detector will identify the bounding box and category information of each target. 2. Feature Extraction: For each detected target, the DeepSORT algorithm uses a deep neural network to extract high-dimensional feature vectors. These feature vectors will help the DeepSORT algorithm determine the uniqueness and similarity of targets in subsequent target tracking. 3. Target Association: At each time step, the DeepSORT algorithm uses the Hungarian algorithm from SORT to establish an association between the target in the current frame and the corresponding target in the previous frame. The Hungarian algorithm matches the target in the current frame with the target in the previous frame by minimizing the cost function of association matching. 4. Motion Prediction: The DeepSORT algorithm uses a Kalman filter to predict and estimate the motion of the target, thereby providing a more accurate position estimate during target tracking. 5. Data Association: During target tracking, the DeepSORT algorithm also considers the similarity of target features, using the distance between feature vectors to determine whether targets are similar and belong to the same tracking object. 6. State Update and Deletion: In each time step, the DeepSORT algorithm updates the target's tracking state based on the target's detection information, association information, and state changes. If a target is not detected in multiple consecutive time steps, or cannot be effectively associated with previous targets, the algorithm will delete the target from the tracking list. Through the above process, this embodiment of the invention can obtain the tracking identifier of targets within the camera's 2D detection frame.

[0103] Step S302: Select the detection frames corresponding to different sensors, calculate the inconsistencies between different sensors, and obtain multiple inconsistencies. For details, please refer to [link to relevant documentation]. Figure 1 Step S102 of the illustrated embodiment will not be described again here.

[0104] Step S303 involves inputting the tracking identifier, observation angle, yaw angle, and multiple inconsistencies from the previous time step into a pre-built inconsistency estimation model based on Gaussian processes to obtain the confidence space of multiple inconsistencies at the current time step. For details, please refer to [link to relevant documentation]. Figure 1 Step S103 of the illustrated embodiment will not be described again here.

[0105] Step S304 involves determining whether multiple inconsistencies are within the confidence space at the current moment, detecting whether various sensors are under attack, and locating the attack. For details, please refer to [link to relevant documentation]. Figure 1 Step S104 of the illustrated embodiment will not be described again here.

[0106] The sensor attack detection and localization method provided in this invention obtains the target's tracking identifier, observation angle, yaw angle, and multiple detection boxes corresponding to various sensors by acquiring target detection results from multiple sensors. Based on these detection boxes, inconsistencies between different sensors are calculated. The tracking identifier, observation angle, yaw angle, and inconsistencies from the previous time step are input into a Gaussian process-based inconsistency estimation model to obtain the confidence space of inconsistencies at the current time step. Sensor attacks are detected and localized based on whether multiple inconsistencies at the current time step fall within this confidence space. This invention uses a Gaussian process to perform non-parametric modeling and analysis of the spatial correlation between sensors, eliminating the need for numerous hyperparameters, reducing modeling time and computational resource consumption, improving model robustness, and enabling effective real-time attack detection and localization for attacks targeting multiple sensors.

[0107] This embodiment provides a method for detecting and locating sensor attacks, which can be used in the aforementioned computer. Figure 7 This is a flowchart of a sensor attack detection and localization method according to an embodiment of the present invention, such as... Figure 7 As shown, the process includes the following steps:

[0108] Step S701: Obtain target detection results from multiple sensors, and analyze the results to obtain the target's tracking identifier, observation angle, yaw angle, and multiple detection boxes corresponding to the various sensors. For details, please refer to [link to relevant documentation]. Figure 3 Step S301 of the illustrated embodiment will not be described again here.

[0109] Step S702: Select the detection frames corresponding to different sensors to calculate the inconsistencies between different sensors and obtain multiple inconsistencies.

[0110] Specifically, step S702 above includes:

[0111] Step S7021: Select the 2D detection frame of the LiDAR and either the 2D detection frame of the binocular camera. For details, please refer to [link to relevant documentation]. Figure 1 Step S102 of the illustrated embodiment will not be described again here.

[0112] Step S7022: By calculating the intersection and union between the 2D detection frame of the LiDAR and the 2D detection frame of any camera, the first inconsistency between the LiDAR and the binocular camera is obtained based on the ratio of the intersection and union.

[0113] Specifically, in embodiments of the present invention, such as Figure 8 As shown, there will be overlap between the 2D detection boxes of the LiDAR and the 2D detection boxes of the camera for the same target. Therefore, the overlap ratio of the two detection boxes is used as the first inconsistency IoU1 between the LiDAR and the binocular camera.

[0114] In some optional implementations, step S7022 above includes:

[0115] Step b1: Take the intersection of the LiDAR 2D detection frame and any camera 2D detection frame as the first intersection, and take the combination of the LiDAR 2D detection frame and any camera 2D detection frame as the first union;

[0116] Step b2: Obtain the coordinates of the top left corner and the bottom right corner of the first intersection;

[0117] Step b3: Calculate the area of ​​the first intersection based on the coordinates of the top left corner and the bottom right corner of the first intersection;

[0118] Step b4: Calculate the area of ​​the LiDAR 2D detection frame based on the coordinate information of the LiDAR 2D detection frame, and calculate the area of ​​the 2D detection frame of any camera based on the coordinate information of the 2D detection frame of any camera.

[0119] Step b5: Subtract the area of ​​the first intersection from the total area between the area of ​​the LiDAR 2D detection frame and the area of ​​any camera 2D detection frame to obtain the area of ​​the first union.

[0120] Step b6: The ratio of the area of ​​the first intersection to the area of ​​the first union is used as the first inconsistency between the lidar and the binocular camera.

[0121] Specifically, in embodiments of the present invention, such as Figure 8 As shown, based on the coordinates of the center point of the camera's 2D detection frame... Calculate the coordinates (X, Y, F, Z) of the four vertices of the camera's 2D bounding box. camera,i ,Y camera,i ), and set the coordinates of the top left vertex in (X) a1 ,Y a1 This indicates that the coordinates of the lower right vertex are expressed in terms of (X). a2 ,Y a2 (This is represented by the symbol ) Then, based on the 2D detection frame Y of the LiDAR... 激光雷达2d检测框 =[min(u), min(v), max(u), max(v)], which sets the top left vertex of the LiDAR 2D detection box to (X) b1 ,Y b1 This indicates that the coordinates of the lower right vertex are expressed in terms of (X). b2 ,Y b2 The coordinates of the top-left corner of the intersection of the camera's 2D detection bounding box and the LiDAR's 2D detection bounding box are obtained using the following formula:

[0122] x1 = max(X a1 ,X b1 ),y1=min(Y a1 ,Yb1 )

[0123] The coordinates of the lower right corner of the intersecting part are:

[0124] x2=min(X a2 ,X b2 ),y2=max(Y a2 ,Y b2 )

[0125] Therefore, the formula for calculating the area representing the first intersection of the camera 2D detection frame and the LiDAR 2D detection frame is:

[0126] intersection=max(x2-x1,0)·max(y2-y1+1.0,0)

[0127] The purpose of setting max(*, 0) is to avoid the occurrence of negative numbers.

[0128] In some optional implementations, the areas of the camera 2D detection frame and the LiDAR 2D detection frame are respectively:

[0129] S A =(x a2 -x a1 +1.0)×(y a2 -y a1 +1.0)

[0130] S B =(x b2 -x b2 +1.0)×(y b2 -y b1 +1.0)

[0131] Therefore, the area of ​​the first union is obtained by subtracting the area of ​​the first intersection from the total area of ​​the LiDAR 2D detection frame and the area of ​​any camera 2D detection frame, as shown in the following formula:

[0132] union = S A +S B -intersection

[0133] In some alternative implementations, adding 1 to the above formula is intended to take into account the pixel gaps within the bounding box when calculating its width and height. Since the gap between two pixels is one pixel, subtracting the top-left corner coordinate from the bottom-right corner coordinate and adding 1 when calculating the bounding box's width and height ensures that every pixel within the bounding box is included.

[0134] In some alternative implementations, the ratio of the area of ​​the first intersection to the area of ​​the first union is used as the first inconsistency IoU1 between the LiDAR and the binocular camera, as shown in the following formula:

[0135]

[0136] Step S7023: Select the first camera 2D detection frame and the second camera 2D detection frame of the binocular camera. For details, please refer to [link to details]. Figure 1 Step S102 of the illustrated embodiment will not be described again here.

[0137] Step S7024: By calculating the intersection and union between the 2D detection frames of the first camera and the second camera, the second inconsistency between the two cameras in the binocular camera is obtained based on the ratio of the intersection and union.

[0138] Specifically, in this embodiment of the invention, there may be overlap between the 2D detection frames of the two cameras of the binocular camera for the same target. Therefore, the overlap ratio of the two detection frames is also used as the second inconsistency IoU2 between the binocular cameras.

[0139] In some optional implementations, step S7024 above includes:

[0140] Step c1: Take the intersection of the first camera 2D detection frame and the second camera 2D detection frame as the second intersection, and take the combination of the first camera 2D detection frame and the second camera 2D detection frame as the second union;

[0141] Step c2: Obtain the coordinates of the top left corner and the bottom right corner of the second intersection;

[0142] Step c3: Calculate the area of ​​the second intersection based on the coordinates of the top left corner and the bottom right corner of the two intersections;

[0143] Step c4: Calculate the area of ​​the first camera 2D detection frame based on the coordinate information of the first camera 2D detection frame, and calculate the area of ​​the second camera 2D detection frame based on the coordinate information of the second camera 2D detection frame.

[0144] Step c5: Subtract the area of ​​the second intersection from the total area between the area of ​​the first camera 2D detection frame and the area of ​​the second camera 2D detection frame to obtain the area of ​​the second union.

[0145] Step c6: The ratio of the area of ​​the second intersection to the area of ​​the second union is used as the second inconsistency between the two cameras in the binocular camera system.

[0146] Specifically, in this embodiment of the invention, the coordinates of the upper left and lower right corners of the 2D detection frames of the two cameras are obtained respectively, as shown below:

[0147] camera1 = [x 11 ,y 11 ,x 12 ,y 12 ]

[0148] camera2 = [x 21 ,y 21 ,x 22 ,y 22 ]

[0149] The formulas for calculating the top-left and bottom-right corner coordinates of the intersection of the 2D detection boxes from the two cameras are as follows:

[0150] x1 = max(x 11 ,x 21 ),y1=max(y 11 ,y 21 )

[0151] x2=max(x 12 ,x 22 ),y2=max(y 12 ,y 22 )

[0152] In one alternative implementation, the second inconsistency IoU2 between the two cameras of the binocular camera is calculated according to the method described in steps b1-b6 above.

[0153] Step S703: Input the tracking identifier, observation angle, yaw angle and multiple inconsistencies from the previous moment into the pre-built inconsistency estimation model based on Gaussian process to obtain the confidence space of multiple inconsistencies at the current moment.

[0154] Specifically, step S703 includes:

[0155] Step S7031: Obtain multiple sets of datasets corresponding to multiple targets. Each set of datasets includes the tracking identifier, observation angle, yaw angle, first inconsistency, and second inconsistency of the same target.

[0156] Specifically, in this embodiment of the invention, each sensor may detect one or more targets. When predicting the inconsistency distribution between sensors, it is necessary to make predictions based on the tracking identifier, observation angle, yaw angle, first inconsistency IoU1 and second inconsistency IoU2 of the same target. Five data points for each target are grouped together and represented by a vector x.

[0157] Step S7032: Standardize the data in each dataset.

[0158] Specifically, in this embodiment of the invention, by subtracting the mean μ of each data point within each data set and dividing by the standard deviation σ, the data can be standardized to a distribution with a mean of 0 and a standard deviation of 1. After standardization, the distribution of the data will change, and the original shape of the data distribution will be altered, but the relative relationships between the values ​​will not be affected. The standardization formula is as follows:

[0159] z=(x-μ) / σ

[0160] Step S7033: Construct an inconsistency evaluation model for the Gaussian process. The inconsistency evaluation model for the Gaussian process includes the mean function, the covariance function, and a noise term that follows a Gaussian distribution.

[0161] Specifically, in this embodiment of the invention, in the Gaussian process model, it is assumed that y is generated by a Gaussian distribution with mean f(x) and variance σ. 2 Therefore, in the Gaussian process model, two key components need to be determined: the mean function m(x) and the covariance function k(x,x′). Specifically, let m(x) be the mean function of f(x), and k(x,x′) be the covariance function between f(x) and f(x′). Assume that ∈ is a noise term following a Gaussian distribution with a mean of 0 and a variance of σ. 2 Then we can get:

[0162] y=f(x)+∈

[0163]

[0164] Step S7034: Using the multiple datasets from the previous time step as prediction inputs and the first and second inconsistencies in the multiple datasets from the current time step as prediction outputs, obtain the joint distribution, marginal distribution, and conditional distribution of the Gaussian process model.

[0165] Specifically, in this embodiment of the invention, the training dataset at different times is represented by a column vector as y = [[IOU11, IOU21], ..., [IOU1...]. n ,IOU2 n ] T If we denote the result of the dot product of vectors x and x′ at different times as K(x,x′), then the joint distribution of the Gaussian process can be expressed as:

[0166]

[0167] Where X = [x1, x2, ..., x n ] T and X * =[x n+1 ,x n+2,…,x n+m ] T f represents the sample points in the test dataset. * K(X,X) represents its corresponding output value (i.e., the test prediction value). * K(X) represents the covariance between the training sample set and the test dataset. * K(X,X) represents the covariance between the test dataset and the training dataset, and K(X,X) represents the covariance between the training datasets. * ,X * ) represents the covariance between the test datasets. This is the variance of the noise term. The most commonly used model for Gaussian processes is based on the RBF kernel function, whose kernel function formula is: RBF kernel function: in, Here, is the variance, l is the length scale, and |•| represents the Euclidean distance. Given the training dataset and covariance function, we need to derive the marginal and conditional distributions of the test dataset from their joint probability distribution for prediction and uncertainty assessment. The marginal distributions can be obtained by marginalizing the joint distribution:

[0168]

[0169] Where, m = E(f) x ), K * =Cov(f x ), f x This represents a vector of latent function values ​​associated with the test dataset. The conditional distribution can be solved using Bayes' theorem:

[0170]

[0171]

[0172] Step S7035: Obtain the first confidence space and the second confidence space of the first inconsistency at the current time based on the joint distribution, marginal distribution and conditional distribution.

[0173] Specifically, in this embodiment of the invention, the confidence space of a Gaussian process can be represented as [m] based on the joint distribution, marginal distribution, and conditional distribution. * -zσ n ,m * +zσ n ], where z represents the confidence level of the confidence interval. According to the normal distribution theory, z is generally taken as 1.96, but it is not a limitation. For example Figure 2As shown, because the embodiments of the present invention have two sets of IOU values, the inconsistency evaluation model based on Gaussian process can output the first confidence space of the first inconsistency and the second confidence space of the first inconsistency at the current time according to the tracking identifier, observation angle, yaw angle, first inconsistency and second inconsistency at the previous time.

[0174] Step S704: By determining whether multiple inconsistencies are within the confidence space at the current moment, detect whether multiple sensors are under attack and locate the attack.

[0175] Specifically, step S704 above includes:

[0176] Step S7041: Sequentially determine whether the first inconsistency is in the first confidence space and whether the second inconsistency is in the second confidence space.

[0177] Step S7042: Detect whether multiple sensors are under attack based on the judgment result, and locate the attack.

[0178] Specifically, in this embodiment of the invention, if both the first inconsistency IoU1 and the second inconsistency IoU2 are within the confidence space at the current moment, then neither the LiDAR nor the binocular camera is attacked; and / or, if neither the first inconsistency IoU1 nor the second inconsistency IoU2 are within the confidence space at the current moment, then at least one of the LiDAR and the binocular camera is attacked; and / or, if the first inconsistency IoU1 is not within the confidence space at the current moment, and the second inconsistency IoU2 is within the confidence space, then the LiDAR is attacked, and the binocular camera is not attacked; and / or, if the first inconsistency IoU1 is within the confidence space at the current moment, and the second inconsistency IoU2 is not within the confidence space, then the binocular camera is attacked, and the LiDAR is not attacked.

[0179] Specifically, in this embodiment of the invention, a multi-sensor cross-correlation localization experiment was conducted on the ROS (Robot Operating System) robot operating system, using the KITTI dataset as test data. The KITTI dataset, jointly created by the Karlsruhe Institute of Technology in Germany and Toyota Research Institute of America, is currently the largest international dataset specifically designed for autonomous driving. This dataset is used to evaluate the performance of algorithms such as lidar odometry, visual odometry, stereo imaging, optical flow, 3D object detection, and 3D tracking in an in-vehicle environment. KITTI contains real-world image data collected from urban, rural, and highway scenes. The data acquisition platform includes... Figure 9 As shown, the sensor suite includes a 64-line LiDAR, flanked by a color camera and a monochrome camera, totaling four cameras, positioned in the center of the roof. To the left and rear of the LiDAR is a combined navigation system (OXTS RT 3003), which outputs RTK / IMU combined navigation results, including latitude, longitude, and attitude, as well as raw IMU data. The installation relationship between the various sensors is as follows... Figure 9 As shown, the embodiments of the present invention demonstrate their effectiveness and feasibility through experiments on real-world autonomous vehicle datasets using two of the most common attack types: LiDAR spoofing attacks and camera blinding attacks.

[0180] In some optional implementations, embodiments of the present invention simulate the process of a LiDAR being attacked. Specifically, a LiDAR deception attack involves tampering with the LiDAR's point cloud data or camera images at different times (simulating a sensor being deceived). The effect of the deception attack is to cause the target detection algorithm to detect a non-existent target. Embodiments of the present invention involve injecting pulses into the LiDAR receiver aperture at several horizontal angles to deceive or hide points within that angle. To simulate a deception attack, embodiments of the present invention delete some laser points from the target's point cloud data. A successful deception attack on the LiDAR causes the target detected by the target detection system to disappear. After being subjected to a laser point cloud deception attack, the disappearance of part of the point cloud prevents a pure LiDAR 3D detection system from detecting the target normally. However, an enhanced fusion 3D detection algorithm and a pure camera 2D detection algorithm can still detect the target normally.

[0181] The above steps yield two sets of confidence spaces for inconsistency. Then, a judgment is made according to steps S7041-S7043, and the results are shown in the table below.

[0182] 33 -3.24 [-2.46,3.40] -0.17 [-4.82,2.65] Anomaly Anomaly LiDAR attack 35 0.03 [-2.95,3.07] -0.02 [-4.68,3.38] Normal Normal No attack 39 -0.78 [-3.52,2.58] 0.32 [-4.27,4.28] Normal Normal No attack 40 -1.07 [-3.64,2.45] 0.23 [-4.01,4.54] Normal Normal No attack 44 -1.68 [-3.46,2.36] 0.42 [-3.15,4.30] Normal Normal No attack

[0183] As can be seen from the table above, the values ​​in the first inconsistency IoU1, which consists of the IoU values ​​of the detection box results, are not within the confidence space. For example... Figure 10 As shown in (a) and (b), the IOU calculation result is abnormal because the camera's detection algorithm can detect the target normally, while the pure LiDAR target detection algorithm cannot detect the target.

[0184] In some optional implementations, the total time consumption for 1000 LiDAR deception attempts is calculated, and the total time consumption is 68ms. The composition of the time consumption is as follows: Figure 11 The task scheduling demonstrates real-time capability, thus enabling real-time attack detection and location.

[0185] In some optional implementations, embodiments of the present invention simulate a camera saturation attack. A camera saturation attack simulates a sensor being saturated by altering camera images taken at different times. The effect of a saturation attack is to prevent the target detection algorithm from detecting the present target normally. In this embodiment, the saturation attack targeting the camera is generated using image processing software, simulating the effect of a strong beam of light hitting the camera. By comparing the target detection results of one camera before and after the blinding attack on one of the binocular cameras, it can be determined that the camera images are problematic after the blinding attack. Because the angle of the attack affects the attack effect, a blinding attack cannot disable all the cameras on an autonomous vehicle at once. When one camera is subjected to a camera saturation attack, the 2D target detection system malfunctions, and two targets are not detected, but the other cameras and LiDAR can still detect the present target normally. The IoU values ​​are further calculated, and the results are shown in the table below.

[0186] 30 -0.25 [-1.63,1.05] -7.73 [-2.86,4.84] Normal Anomaly Camera attack 40 0.88 [-2.21,1.80] -5.11 [-3.19,5.26] Normal Anomaly Camera attack 41 0.30 [-1.68,0.39] 0.43 [-3.10,4.85] Normal Normal No attack 42 -0.24 [-1.70,0.00] 0.73 [-3.26,5.02] Normal Normal No attack 43 -0.39 [-1.64,0.08] 0.68 [-3.21,5.17] Normal Normal No attack 44 1.21 [0.24,2.40] -0.24 [-4.16,2.21] Normal Normal No attack 52 1.12 [-0.94,1.54] 0.07 [-3.95,3.40] Normal Normal No attack 57 0.32 [-2.39,1.06] 0.18 [-3.89,4.58] Normal Normal No attack

[0187] As can be seen from the table above, outliers appear in IoU2. Through attack detection and location rules, it can be determined that the camera has been attacked. For example... Figure 12 As shown in (a) and (b), attack detection and attack localization have been successfully achieved. The total time consumption for one thousand camera saturation attacks was statistically analyzed, with the target detection algorithm consuming 68ms. The real-time performance analysis is as follows: Figure 11 As shown, this enables real-time attack detection and location.

[0188] This embodiment provides a sensor attack detection and location device, such as... Figure 13 As shown, it includes:

[0189] The data acquisition module 1301 is used to acquire target detection results from multiple sensors and analyze the target detection results to obtain the target's tracking identifier, observation angle, yaw angle, and multiple detection boxes corresponding to the multiple sensors.

[0190] The inconsistency calculation module 1302 is used to select the detection boxes corresponding to different sensors to calculate the inconsistency between different sensors and obtain multiple inconsistencies.

[0191] The model prediction module 1303 is used to input the tracking identifier, observation angle, yaw angle and multiple inconsistencies from the previous time step into a pre-built inconsistency estimation model based on Gaussian process to obtain the confidence space of multiple inconsistencies at the current time step.

[0192] The attack detection module 1304 is used to detect whether multiple sensors are under attack by determining whether multiple inconsistencies are within the confidence space at the current time, and to locate the attack.

[0193] In some alternative implementations, the data acquisition module 1301:

[0194] The lidar data acquisition unit is used to acquire the target detection results of the lidar; it uses a first preset algorithm to analyze the target detection results to obtain the lidar 3D detection frame and yaw angle of the target, and represents the lidar 3D detection frame with the center point coordinates, length, width and height of the lidar 3D detection frame; it converts the lidar 3D detection frame into a lidar 2D detection frame through coordinate transformation.

[0195] The camera data acquisition unit is used to acquire the target detection results of the binocular camera; to analyze the target detection results using a second preset algorithm to obtain the camera 2D detection box of the target, and to represent the camera 2D detection box by the center point coordinates, width and height of the camera 2D detection box, wherein the camera 2D detection box includes a first camera 2D detection box and a second camera 2D detection box; to calculate the observation angle of the target based on the center point coordinates, width and height of the camera 2D detection box; to track the target by performing feature extraction, target association, motion prediction and data association on the target within the camera 2D detection box at different times, and to acquire the target tracking identifier when the target is detected in a continuous time period.

[0196] In some alternative implementations, the inconsistency calculation module 1302 includes:

[0197] The first inconsistency calculation unit is used to select the 2D detection frame of the lidar and any 2D detection frame of the binocular camera; by calculating the intersection and union between the 2D detection frame of the lidar and any 2D detection frame of the binocular camera, the first inconsistency between the lidar and the binocular camera is obtained according to the ratio of the intersection and union.

[0198] The second inconsistency calculation unit is used to select the first camera 2D detection frame and the second camera 2D detection frame of the binocular camera; by calculating the intersection and union between the first camera 2D detection frame and the second camera 2D detection frame, the second inconsistency between the two cameras in the binocular camera is obtained according to the ratio of the intersection and union.

[0199] In some alternative implementations, the model prediction module 1203 includes:

[0200] The data processing unit is used to acquire multiple sets of datasets corresponding to multiple targets. Each set of datasets includes the tracking identifier, observation angle, yaw angle, first inconsistency and second inconsistency of the same target. The data in each set of datasets is standardized.

[0201] The model building unit is used to construct an inconsistent evaluation model for a Gaussian process. The inconsistent evaluation model for a Gaussian process includes a mean function, a covariance function, and a noise term that follows a Gaussian distribution.

[0202] The model prediction unit is used as the prediction input of multiple datasets from the previous time step, and as the prediction output of the first and second inconsistencies in the multiple datasets at the current time step, to obtain the joint distribution, marginal distribution, and conditional distribution of the Gaussian process model; and to obtain the first confidence space and the second confidence space of the first inconsistency at the current time step based on the joint distribution, marginal distribution, and conditional distribution.

[0203] In some alternative implementations, the attack detection module 1304 includes:

[0204] The judgment unit is used to sequentially determine whether the first inconsistency is in the first confidence space and whether the second inconsistency is in the second confidence space.

[0205] An attack detection and localization unit is used to detect whether multiple sensors are under attack based on the judgment result and to locate the attack. If both the first inconsistency and the second inconsistency are within the confidence space at the current moment, then neither the LiDAR nor the binocular camera is under attack. And / or, if neither the first inconsistency nor the second inconsistency is within the confidence space at the current moment, then at least one of the LiDAR and the binocular camera is under attack. And / or, if the first inconsistency is not within the confidence space at the current moment, but the second inconsistency is within the confidence space, then the LiDAR is under attack, and the binocular camera is not under attack. And / or, if the first inconsistency is within the confidence space at the current moment, but the second inconsistency is not within the confidence space, then the binocular camera is under attack, and the LiDAR is not under attack.

[0206] In some optional implementations, the lidar data acquisition unit includes:

[0207] The coordinate transformation subunit is used to obtain the vertex coordinates of multiple vertices of the LiDAR 3D detection frame based on the center point coordinates, length, width, and height; based on the rotation matrix of the image coordinate system from the LiDAR to the binocular camera, the camera projection matrix of the binocular camera, and the camera rotation matrix, the vertex coordinates of multiple vertices of the LiDAR 3D detection frame are converted into multiple homogeneous coordinates in the image coordinate system; based on the maximum value, minimum value, maximum value, and minimum value along the horizontal axis, and the maximum value along the vertical axis, the LiDAR 2D detection frame is determined.

[0208] In some alternative implementations, the first inconsistency calculation unit includes:

[0209] The first overlap ratio calculation subunit is used to take the intersection of the LiDAR 2D detection frame and any camera 2D detection frame as the first intersection, and the combination of the LiDAR 2D detection frame and any camera 2D detection frame as the first union; obtain the coordinates of the upper left corner and the lower right corner of the first intersection; calculate the area of ​​the first intersection based on the coordinates of the upper left corner and the lower right corner of the first intersection; calculate the area of ​​the LiDAR 2D detection frame based on the coordinate information of the LiDAR 2D detection frame, and calculate the area of ​​any camera 2D detection frame based on the coordinate information of any camera 2D detection frame; subtract the area of ​​the first intersection from the total area between the area of ​​the LiDAR 2D detection frame and the area of ​​any camera 2D detection frame to obtain the area of ​​the first union; and use the ratio of the area of ​​the first intersection to the area of ​​the first union as the first inconsistency between the LiDAR and the binocular camera.

[0210] In some alternative implementations, the second inconsistency calculation unit includes:

[0211] The second overlap ratio calculation subunit is used to take the intersection of the first camera 2D detection frame and the second camera 2D detection frame as the second intersection, and the combination of the first camera 2D detection frame and the second camera 2D detection frame as the second union; obtain the upper left corner coordinates and lower right corner coordinates of the second intersection; calculate the area of ​​the second intersection based on the upper left corner coordinates and lower right corner coordinates of the two intersections; calculate the area of ​​the first camera 2D detection frame based on the coordinate information of the first camera 2D detection frame, and calculate the area of ​​the second camera 2D detection frame based on the coordinate information of the second camera 2D detection frame; subtract the area of ​​the second intersection from the total area between the areas of the first camera 2D detection frame and the areas of the second camera 2D detection frame to obtain the area of ​​the second union; and use the ratio of the area of ​​the second intersection to the area of ​​the second union as the second inconsistency between the two cameras in the binocular camera.

[0212] Further functional descriptions of the above modules and units are the same as those in the corresponding embodiments described above, and will not be repeated here.

[0213] In this embodiment, the sensor attack detection and location device is presented in the form of a functional unit. Here, a unit refers to an FPGA (Field Programmable Gate Array) circuit, a processor and memory that execute one or more software or fixed programs, and / or other devices that can provide the above functions.

[0214] This invention also provides a computer device having the above-described features. Figure 13 The device shown is for detecting and locating sensor attacks.

[0215] Please see Figure 14 , Figure 14 This is a schematic diagram of the structure of a computer device provided in an optional embodiment of the present invention, such as... Figure 14 As shown, the computer device includes one or more processors 10, memory 20, and interfaces for connecting the components, including high-speed interfaces and low-speed interfaces. The components communicate with each other via different buses and can be mounted on a common motherboard or otherwise installed as needed. The processors can process instructions executed within the computer device, including instructions stored in or on memory to display graphical information of a GUI on external input / output devices (such as display devices coupled to the interfaces). In some alternative implementations, multiple processors and / or multiple buses can be used with multiple memories and multiple memory modules, if desired. Similarly, multiple computer devices can be connected, each providing some of the necessary operations (e.g., as a server array, a group of blade servers, or a multiprocessor system). Figure 14 Take a processor 10 as an example.

[0216] Processor 10 may be a central processing unit, a network processor, or a combination thereof. Processor 10 may further include a hardware chip. The hardware chip may be an application-specific integrated circuit (ASIC), a programmable logic device (PLD), or a combination thereof. The programmable logic device may be a complex programmable logic device (CAMP), a field-programmable gate array (FPGA), a general-purpose array logic (GDA), or any combination thereof.

[0217] The memory 20 stores instructions executable by at least one processor 10 to cause at least one processor 10 to perform the method shown in the above embodiments.

[0218] The memory 20 may include a program storage area and a data storage area. The program storage area may store the operating system and applications required for at least one function; the data storage area may store data created based on the use of the computer device. Furthermore, the memory 20 may include high-speed random access memory and may also include non-transitory memory, such as at least one disk storage device, flash memory device, or other non-transitory solid-state storage device. In some alternative embodiments, the memory 20 may optionally include memory remotely located relative to the processor 10, and these remote memories may be connected to the computer device via a network. Examples of such networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof.

[0219] The memory 20 may include volatile memory, such as random access memory; the memory may also include non-volatile memory, such as flash memory, hard disk or solid-state drive; the memory 20 may also include a combination of the above types of memory.

[0220] The computer device also includes a communication interface 30 for communicating with other devices or communication networks.

[0221] This invention also provides a computer-readable storage medium. The methods described above according to embodiments of the invention can be implemented in hardware or firmware, or implemented as computer code that can be recorded on a storage medium, or implemented as computer code downloaded via a network and originally stored on a remote storage medium or a non-transitory machine-readable storage medium and then stored on a local storage medium. Thus, the methods described herein can be processed by software stored on a storage medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware. The storage medium can be a magnetic disk, optical disk, read-only memory, random access memory, flash memory, hard disk, or solid-state drive, etc.; further, the storage medium can also include combinations of the above types of memory. It is understood that computers, processors, microprocessor controllers, or programmable hardware include storage components capable of storing or receiving software or computer code, which, when accessed and executed by the computer, processor, or hardware, implements the methods shown in the above embodiments.

[0222] Although embodiments of the invention have been described in conjunction with the accompanying drawings, those skilled in the art can make various modifications and variations without departing from the spirit and scope of the invention, and such modifications and variations all fall within the scope defined by the appended claims.

Claims

1. A method for detecting and locating sensor attacks, characterized in that, include: The target detection results from multiple sensors are acquired, and the target detection results are analyzed to obtain the target's tracking identifier, observation angle, yaw angle, and multiple detection boxes corresponding to the various sensors. Select the detection boxes corresponding to different sensors to calculate the inconsistencies between the different sensors and obtain multiple inconsistencies; The tracking identifier, observation angle, yaw angle, and multiple inconsistencies from the previous moment are input into a pre-built inconsistency estimation model based on Gaussian processes to obtain the confidence space of the multiple inconsistencies at the current moment. By determining whether the multiple inconsistencies at the current moment are within the confidence space, the system detects whether the multiple sensors are under attack and locates the attack. The multiple detection frames corresponding to the various sensors include a LiDAR 2D detection frame and a binocular camera 2D detection frame, wherein the camera 2D detection frame includes a first camera 2D detection frame and a second camera 2D detection frame; the process of selecting detection frames corresponding to different sensors, calculating the inconsistencies between the different sensors, and obtaining multiple inconsistencies includes: Select the 2D detection frame of the lidar and the 2D detection frame of either the binocular camera; The first inconsistency between the lidar and the binocular camera is obtained by calculating the intersection and union between the lidar 2D detection frame and any of the camera 2D detection frames, based on the ratio of the intersection and the union. Select the first camera 2D detection frame and the second camera 2D detection frame of the binocular camera; By calculating the intersection and union of the first camera 2D detection frame and the second camera 2D detection frame, a second inconsistency between the two cameras in the binocular camera is obtained based on the ratio of the intersection to the union.

2. The method according to claim 1, characterized in that, The process of acquiring target detection results from multiple sensors and analyzing these results to obtain the target's tracking identifier, observation angle, yaw angle, and multiple detection boxes corresponding to the various sensors includes: Obtain the target detection results from the lidar; The target detection results are analyzed using a first preset algorithm to obtain the target's 3D detection frame and yaw angle using a lidar. The lidar 3D detection frame is represented by the center point coordinates, length, width, and height of the lidar. The 3D detection box of the lidar is converted into a 2D detection box of the lidar by coordinate transformation; Obtain the target detection results from the binocular camera; The target detection results are analyzed using a second preset algorithm to obtain the target's camera 2D detection box, and the camera 2D detection box is represented by the center point coordinates, width, and height of the camera 2D detection box; The observation angle of the target is calculated based on the center point coordinates, width, and height of the 2D detection frame of the camera. The target is tracked by performing feature extraction, target association, motion prediction, and data association on the target within the 2D detection frame of the camera at different times. When the target is detected in a continuous time period, the tracking identifier of the target is obtained.

3. The method according to claim 1, characterized in that, The process of inputting the tracking identifier, observation angle, yaw angle, and multiple inconsistencies from the previous moment into a pre-built inconsistency estimation model based on a Gaussian process to obtain the confidence space of the multiple inconsistencies at the current moment includes: Obtain multiple sets of datasets corresponding to multiple targets, each set of datasets including the tracking identifier, observation angle, yaw angle, first inconsistency and second inconsistency for the same target; Standardize the data within each dataset; An inconsistency evaluation model for a Gaussian process is constructed, comprising a mean function, a covariance function, and a noise term following a Gaussian distribution. Using the multiple datasets from the previous time step as prediction input, and the first inconsistency and the second inconsistency in the multiple datasets from the current time step as prediction output, the joint distribution, marginal distribution, and conditional distribution of the Gaussian process model are obtained. The first confidence space and the second confidence space of the first inconsistency at the current time are obtained based on the joint distribution, the marginal distribution, and the conditional distribution.

4. The method according to claim 3, characterized in that, The process of detecting whether the multiple inconsistencies are within the confidence space at the current time, and locating the attack, includes: Sequentially determine whether the first inconsistency is located in the first confidence space, and whether the second inconsistency is located in the second confidence space; Based on the judgment results, the system detects whether the various sensors have been attacked and locates the attack. If both the first inconsistency and the second inconsistency are within the confidence space at the current moment, then neither the lidar nor the binocular camera has been attacked. And / or, if at the current moment neither the first inconsistency nor the second inconsistency is within the confidence space, then at least one of the lidar and the binocular camera is under attack; And / or, if the first inconsistency is not within the confidence space at the current moment, and the second inconsistency is within the confidence space, then the lidar is attacked, and the binocular camera is not attacked; And / or, if the first inconsistency is within the confidence space at the current moment, and the second inconsistency is not within the confidence space, then the binocular camera is attacked, and the lidar is not attacked.

5. The method according to claim 2, characterized in that, The process of converting the 3D detection box of the LiDAR into a 2D detection box of the LiDAR through coordinate transformation includes: The vertex coordinates of multiple vertices of the LiDAR 3D detection frame are obtained based on the center point coordinates, length, width, and height of the LiDAR 3D detection frame. Based on the rotation matrix of the image coordinate system of the lidar to the binocular camera, the camera projection matrix and the camera rotation matrix of the binocular camera, the vertex coordinates of multiple vertices of the lidar 3D detection frame are converted into multiple homogeneous coordinates in the image coordinate system. The 2D detection frame of the lidar is determined based on the maximum value, minimum value, maximum value, and minimum value along the horizontal axis, and along the vertical axis of the plurality of homogeneous coordinates.

6. The method according to claim 4, characterized in that, The process of calculating the intersection and union between the LiDAR 2D detection frame and any of the camera 2D detection frames, and obtaining the first inconsistency between the LiDAR and the binocular camera based on the ratio of the intersection and the union, includes: The intersection of the LiDAR 2D detection frame and any of the camera 2D detection frames is taken as the first intersection, and the combination of the LiDAR 2D detection frame and any of the camera 2D detection frames is taken as the first union. Obtain the coordinates of the top left and bottom right corners of the first intersection; Calculate the area of ​​the first intersection based on the coordinates of its top-left and bottom-right corners; The area of ​​the LiDAR 2D detection frame is calculated based on the coordinate information of the LiDAR 2D detection frame, and the area of ​​any camera 2D detection frame is calculated based on the coordinate information of any camera 2D detection frame. The area of ​​the first union is obtained by subtracting the area of ​​the first intersection from the total area between the area of ​​the LiDAR 2D detection frame and the area of ​​any of the camera 2D detection frames; The ratio of the area of ​​the first intersection to the area of ​​the first union is used as the first inconsistency between the lidar and the binocular camera. And / or, the process of obtaining the second inconsistency between the two cameras in the binocular camera system by calculating the intersection and union between the first camera 2D detection frame and the second camera 2D detection frame, and based on the ratio of the intersection and the union, includes: The intersection of the first camera 2D detection frame and the second camera 2D detection frame is taken as the second intersection, and the combination of the first camera 2D detection frame and the second camera 2D detection frame is taken as the second union; Obtain the coordinates of the top left and bottom right corners of the second intersection; Calculate the area of ​​the second intersection based on the coordinates of its top-left and bottom-right corners; The area of ​​the first camera 2D detection frame is calculated based on the coordinate information of the first camera 2D detection frame, and the area of ​​the second camera 2D detection frame is calculated based on the coordinate information of the second camera 2D detection frame. The area of ​​the second union is obtained by subtracting the area of ​​the second intersection from the total area between the area of ​​the first camera 2D detection frame and the area of ​​the second camera 2D detection frame. The ratio of the area of ​​the second intersection to the area of ​​the second union is used as the second inconsistency between the two cameras in the binocular camera.

7. A sensor attack detection and location device, characterized in that, The device includes: The data acquisition module is used to acquire target detection results from multiple sensors and analyze the target detection results to obtain the target's tracking identifier, observation angle, yaw angle, and multiple detection boxes corresponding to the multiple sensors. The inconsistency calculation module is used to select the detection boxes corresponding to different sensors to calculate the inconsistency between the different sensors and obtain multiple inconsistencies. The model prediction module is used to input the tracking identifier, observation angle, yaw angle and multiple inconsistencies from the previous time step into a pre-built inconsistency estimation model based on Gaussian process to obtain the confidence space of the multiple inconsistencies at the current time step. An attack detection module is used to detect whether the various sensors are under attack by determining whether the multiple inconsistencies are within the confidence space at the current time, and to locate the attack. The multiple detection frames corresponding to the various sensors include a lidar 2D detection frame and a binocular camera 2D detection frame, wherein the camera 2D detection frame includes a first camera 2D detection frame and a second camera 2D detection frame; the inconsistency calculation module is further used for: Select the 2D detection frame of the lidar and the 2D detection frame of either the binocular camera; The first inconsistency between the lidar and the binocular camera is obtained by calculating the intersection and union between the lidar 2D detection frame and any of the camera 2D detection frames, based on the ratio of the intersection and the union. Select the first camera 2D detection frame and the second camera 2D detection frame of the binocular camera; By calculating the intersection and union of the first camera 2D detection frame and the second camera 2D detection frame, a second inconsistency between the two cameras in the binocular camera is obtained based on the ratio of the intersection to the union.

8. A computer device, characterized in that, include: A memory and a processor are interconnected, the memory storing computer instructions, and the processor executing the computer instructions to perform the sensor attack detection and localization method according to any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions for causing the computer to execute the sensor attack detection and localization method according to any one of claims 1 to 6.

Images