A sensor spoofing attack detection method, apparatus, device and medium

Abstract

Embodiments of the present application disclose a sensor spoofing attack detection method, device, equipment and medium. The method comprises: acquiring a current frame collected by a sensor in a vehicle, and at least two historical continuous frames before the current frame; determining a first key point of a target object in the current frame according to a preset directional fast rotation algorithm, and determining a second key point of the target object in each historical image frame, and converting each second key point into a three-dimensional map point; performing key point matching based on a mapping relationship between the three-dimensional map point and the first key point to obtain a key point matching error; acquiring scene data corresponding to the vehicle, and detecting spoofing attack of the sensor based on the scene data and the key point matching error. Through the above technical solution, all sensor attacks and spoofing can be defended and identified, and the detection delay can be controlled within 100 milliseconds, thereby improving the safety of autonomous driving.

Description

Technical Field

[0001] This invention relates to the field of vehicle driving safety technology, and in particular to a method, apparatus, device, and medium for detecting sensor spoofing attacks. Background Technology

[0002] Sensor spoofing attacks are quite effective in automated driving systems (ADS). However, these attacks are easily detected by users, which limits their effectiveness. This discrepancy with reality arises from a spatiotemporal contradiction between the target state of the spoofing and the actual state sensed in the real world.

[0003] Negligence by sensors can lead to modifications of the internal state estimated based on sensor data, resulting in unexpected dangerous behaviors such as rear-end collisions and falling off cliffs. These are types of deception attacks: (1) Target position alteration: The adversary induces a false position for the vehicle, thereby triggering unexpected decisions and actions; (2) Target position deviation: The accumulation of position deviations leads to incorrect predictions of the target vehicle's position; (3) Target speed instability: An angular velocity is generated in a single direction, causing the vehicle to overturn; (4) Obstacle appearance: The vehicle is deceived into believing that there is a non-existent obstacle ahead; (5) Obstacle concealment: The vehicle is deceived into not being able to detect an obstacle ahead; (6) Object classification error: The vehicle is misidentified by traffic signs or traffic signals, resulting in dangerous consequences; (7) Lane change: The vehicle is misidentified by the lane, resulting in potentially dangerous consequences.

[0004] Existing technologies can only identify certain sensor spoofing attacks. Due to their heavy reliance on DNN models and their limitation to only a few types of sensor attacks, existing methods suffer from high detection latency and a lack of interpretability. Furthermore, they require GPUs to run, which restricts their application scope. Summary of the Invention

[0005] In view of this, the present invention provides a sensor spoofing attack detection method, apparatus, device and medium, which can defend against and identify all sensor attacks, and the detection delay can be controlled within 100 milliseconds, thereby improving the safety of autonomous driving.

[0006] According to one aspect of the present invention, an embodiment of the present invention provides a method for detecting sensor spoofing attacks, the method comprising:

[0007] Acquire the current frame collected by the sensors in the vehicle, as well as at least two consecutive historical frames preceding the current frame;

[0008] The first key point of the target object in the current frame is determined according to the preset orientation fast rotation algorithm, and the second key point of the target object in each historical image frame in each historical continuous frame is determined, and each second key point is converted into a three-dimensional map point.

[0009] The key point matching error is obtained by performing key point matching based on the mapping relationship between the 3D map points and the first key point.

[0010] Acquire scene data corresponding to the vehicle, and detect sensor spoofing attacks based on the scene data and the key point matching error.

[0011] According to another aspect of the present invention, embodiments of the present invention also provide a sensor spoofing attack detection device, the device comprising:

[0012] The acquisition module is used to acquire the current frame collected by the sensors in the vehicle, as well as at least two consecutive historical frames preceding the current frame.

[0013] The key point determination module is used to determine the first key point of the target object in the current frame according to the preset orientation fast rotation algorithm, and to determine the second key point of the target object in each historical image frame in each of the historical consecutive frames, and to convert each second key point into a three-dimensional map point.

[0014] The error determination module is used to obtain the key point matching error by performing key point matching based on the mapping relationship between the three-dimensional map points and the first key point.

[0015] The detection module is used to acquire scene data corresponding to the vehicle and detect sensor spoofing attacks based on the scene data and the key point matching error.

[0016] According to another aspect of the present invention, embodiments of the present invention also provide an electronic device, the electronic device comprising:

[0017] At least one processor; and

[0018] A memory communicatively connected to the at least one processor; wherein,

[0019] The memory stores a computer program that can be executed by the at least one processor, which enables the at least one processor to perform the sensor spoofing attack detection method according to any embodiment of the present invention.

[0020] According to another aspect of the present invention, embodiments of the present invention also provide a computer-readable storage medium storing computer instructions for causing a processor to execute and implement the sensor spoofing attack detection method described in any embodiment of the present invention.

[0021] The technical solution of this invention determines the first key point of the target object in the current frame through a preset directional fast rotation algorithm, determines the second key point of the target object in each historical image frame in each historical consecutive frame, converts each second key point into a three-dimensional map point, performs key point matching based on the mapping relationship between the three-dimensional map point and the first key point to obtain the key point matching error, acquires the scene data corresponding to the vehicle, and detects sensor spoofing attacks based on the scene data and the key point matching error. It can defend against and identify attacks on all sensors, and the detection delay can be controlled within 100 milliseconds, thereby improving the safety of autonomous driving.

[0022] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of the present invention, nor is it intended to limit the scope of the invention. Other features of the invention will become readily apparent from the following description. Attached Figure Description

[0023] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0024] Figure 1 A flowchart illustrating a sensor spoofing attack detection method according to an embodiment of the present invention;

[0025] Figure 2 A flowchart of another sensor spoofing attack detection method provided in an embodiment of the present invention;

[0026] Figure 3 An architecture diagram of a spatial consistency model provided in an embodiment of the present invention;

[0027] Figure 4 The detection architecture diagram for each deception attack provided in the embodiments of the present invention;

[0028] Figure 5 This is an architecture diagram for detecting sensor spoofing attacks provided in an embodiment of the present invention;

[0029] Figure 6 A performance overhead comparison diagram of an architecture for detecting sensor spoofing attacks provided in an embodiment of the present invention;

[0030] Figure 7 This is a structural block diagram of a sensor spoofing attack detection device provided in an embodiment of the present invention;

[0031] Figure 8 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention. Detailed Implementation

[0032] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0033] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0034] In one embodiment, Figure 1 This is a flowchart of a sensor spoofing attack detection method according to an embodiment of the present invention. This embodiment is applicable to the defense and identification of all sensor spoofing attacks. The method can be executed by a sensor spoofing attack detection device, which can be implemented in hardware and / or software and can be configured in an electronic device.

[0035] like Figure 1 As shown, the sensor spoofing attack detection method in this embodiment includes the following specific steps:

[0036] S110. Obtain the current frame collected by the sensors in the vehicle, as well as at least two consecutive historical frames preceding the current frame.

[0037] The sensors used may include, but are not limited to, Global Positioning System (GPS), LiDAR, cameras, inertial measurement unit (IMU) sensors, ultrasonic sensors, and millimeter-wave radar. Of course, different sensors generally serve different purposes. For example, an IMU sensor is a combination of an accelerometer and a gyroscope sensor. It is used to detect acceleration and angular velocity to represent motion and motion intensity; cameras are generally used to capture video of a vehicle in motion, and so on. This embodiment does not impose any limitations.

[0038] In this embodiment, the current frame can be understood as the frame image at the current moment. This image frame includes multiple target objects. Of course, the target objects may include, but are not limited to, stationary or non-stationary target objects such as lane lines and obstacles. The historical continuous frames can be understood as image frames of at least two consecutive moments in history. Similarly, the historical continuous frames may include, but are not limited to, stationary or non-stationary target objects.

[0039] In this embodiment, the current frame collected by the sensors in the vehicle and at least two previous historical consecutive frames can be obtained from the video images collected by one or more sensors installed in the autonomous vehicle. In this embodiment, the at least two historical consecutive frames can form a stable continuous frame image. The current frame and the at least two historical consecutive frames both include corresponding key point information, which can characterize key features.

[0040] S120. Determine the first key point of the target object in the current frame according to the preset orientation fast rotation algorithm, and determine the second key point of the target object in each historical image frame in each historical continuous frame, and convert each second key point into a three-dimensional map point.

[0041] The preset orientation fast rotation algorithm, also known as the ORB (Oriented FAST and Rotated BRIEF) algorithm, can quickly extract and describe feature points. Essentially, the ORB algorithm uses the FAST algorithm to detect key points in the image pyramid, calculates the orientation of the key points using first-order moments, and generates feature descriptors using orientation-corrected BRIEFs. The first key point refers to the key point of the target object obtained from the current frame; the second key point refers to the key point of the target object obtained from historical image frames. In this embodiment, the historical consecutive frames include at least two historical image frames, and each historical image frame includes a corresponding second key point.

[0042] In some embodiments, the second keypoint may include global keypoints and local keypoints; global keypoints can be understood as all keypoints in the historical frame image, and local keypoints can be understood as keypoints of a specific target object in the historical frame image. Since keypoints can be extracted from multiple consecutive historical image frames, these keypoints can form stable keypoints in image frames at different times. These stable keypoints can be converted into the form of 3D map points. In this embodiment, the 3D map points may include global map points and local map points. It should be noted that local keypoints and local map points are located within bounding boxes; the bounding boxes include a first bounding box and a second bounding box.

[0043] In this embodiment, the first key point and the second key point can be characterized as pixels in the current frame whose brightness changes more than a preset pixel brightness threshold compared to surrounding pixels. The first key point of the target object in the current frame and the second key point of the target object in each historical image frame in each historical consecutive frame can be extracted according to the ORB algorithm. It can be understood that the extraction of the first key point and the second key point can be performed using the ORB algorithm. Of course, in addition to the extraction method provided in this embodiment, other methods can also be used to extract key points. This embodiment does not limit this.

[0044] In some embodiments, for the current frame and each historical image frame in a continuous historical frame, an arbitrary pixel can be selected as the center point. The brightness values ​​of a preset number of pixels within a preset distance from the center point are traversed. If the brightness values ​​of the preset number of pixels are greater than or equal to a preset pixel brightness threshold, the pixel is taken as a key point, and the next pixel is selected. The above steps are repeated until all pixels in the current frame have been judged, resulting in a key point set. This key point set includes multiple key points. Multi-scale sampling is performed on each key point to obtain multi-scale target key points. The direction of the multi-scale target key points is determined using a preset intensity centroid method, and the multi-scale target key points with the determined direction are taken as the first key points. In other embodiments, a universal scale-invariant feature transformation algorithm can also be used to detect and describe local features in the image to determine the first and second key points in the current frame and each historical image frame in a continuous historical frame. This embodiment does not impose any limitations on this.

[0045] In this embodiment, converting each second keypoint into a 3D map point can include two scenarios: one is when a historical continuous frame is identified as a key image frame, in which case each second keypoint needs to be converted into a 3D map point; the other is when a bounding box event is triggered, which can include: the first detection of a bounding box and the sudden disappearance of the bounding box, in which case each second keypoint is converted into a 3D map point. In this embodiment, when a historical continuous frame is identified as a key image frame, triangulation can be performed on each global keypoint at each historical moment to obtain the first change information of each global keypoint in the historical continuous frame image in at least two historical continuous frames, and the global map point corresponding to each historical continuous frame can be generated based on the first change information. In some embodiments, when a bounding box event is triggered, local keypoints of each target object in the bounding box at different times can be extracted, triangulation can be performed on the local keypoints to obtain the second change information of each local keypoint in the bounding box in the historical continuous frames, and the local map point corresponding to the historical continuous frame can be generated based on the second change information.

[0046] S130. Based on the mapping relationship between 3D map points and the first key point, perform key point matching to obtain the key point matching error.

[0047] The keypoint matching error can be understood as the error obtained by projecting 3D map points onto the first keypoint in 2D space for point matching. In this embodiment, the 3D map points include global map points and local map points, with local map points located within bounding boxes; the bounding boxes include a first bounding box and a second bounding box.

[0048] In this embodiment, 3D map points can be projected into the 2D space of the current scene to obtain the distribution of predicted key points at the current moment. Since the 3D map points in this embodiment include global map points and local map points, different map points will result in different predicted key point distributions at the current moment when projected into the 2D space. In some embodiments, global map points can be projected into the 2D space of the current frame to obtain the first predicted key point distribution at the current moment. The first predicted key point distribution is matched with the first key point to obtain the corresponding first offset error. In other embodiments, a first distance between the first key point and the first bounding box and the vehicle can be determined, and a second distance between the local map point and the vehicle can be determined. The local map points in the second bounding box with the second distance are mapped to the first bounding box with the first distance to obtain the second predicted key point distribution. The second key point is matched with the second predicted key point distribution to obtain the corresponding second offset error. This can be understood as follows: in order to detect the latter two types of spatial consistency violations, it is necessary to use an additional distance metric to convert the extracted key points in each bounding box into three-dimensional feature points. Subsequently, the three-dimensional feature points and the corresponding matching observation key points are nonlinearly optimized to obtain the corresponding second offset error. In this embodiment, spatial consistency violations may include target entity confusion, target semantic confusion, and target flickering.

[0049] S140. Obtain scene data corresponding to the vehicle, and detect sensor deception attacks based on the scene data and key point matching errors.

[0050] In this embodiment, the scene data may include, but is not limited to, the vehicle's position information, the bounding box information in front of the vehicle, and the type of the target object in front of the vehicle; wherein, the position information includes: the vehicle's current position information at the current moment, and the vehicle's position information at the previous moment before the current moment; the bounding box information includes: first bounding box information and second bounding box information.

[0051] In this embodiment, the positional deviation between the current position and the previous position can be obtained based on the vehicle's current position information and the vehicle's position information at the previous moment. The sensor spoofing attack can be detected based on the positional deviation and the key point matching error. In some embodiments, the sensor spoofing attack can also be detected based on the bounding box information and the key point matching error. In other embodiments, if the object type of the key point obtained by mapping the local map point in the second bounding box to the first bounding box changes, the type of the target object in the local map point can be determined, and the sensor spoofing attack can be detected based on the type of the target object and the key point matching error. This embodiment does not impose any limitations on this.

[0052] The technical solution of this invention determines the first key point of the target object in the current frame through a preset directional fast rotation algorithm, determines the second key point of the target object in each historical image frame in each historical consecutive frame, converts each second key point into a three-dimensional map point, performs key point matching based on the mapping relationship between the three-dimensional map point and the first key point to obtain the key point matching error, acquires the scene data corresponding to the vehicle, and detects sensor spoofing attacks based on the scene data and the key point matching error. It can defend against and identify attacks on all sensors, and the detection delay can be controlled within 100 milliseconds, thereby improving the safety of autonomous driving.

[0053] In one embodiment, Figure 2 This is a flowchart of another sensor spoofing attack detection method provided in an embodiment of the present invention. Based on the above embodiments, this embodiment further refines the following steps: determining the first key point of the target object in the current frame according to a preset directional fast rotation algorithm; converting each second key point into a 3D map point; performing key point matching based on the mapping relationship between the 3D map point and the first key point to obtain the key point matching error; and detecting sensor spoofing attacks based on scene data and key point matching error.

[0054] like Figure 2 As shown, the sensor spoofing attack detection method in this embodiment may specifically include the following steps:

[0055] S210. Obtain the current frame collected by the sensors in the vehicle, as well as at least two consecutive historical frames preceding the current frame.

[0056] S220. In the current frame, arbitrarily select a pixel as the current pixel, and with the current pixel as the center point, traverse the brightness values ​​of a preset number of pixels within a preset distance from the center point.

[0057] The preset distance and preset quantity are custom values ​​that can be set based on experience or according to requirements. This embodiment does not impose any restrictions on these values.

[0058] In this embodiment, any pixel can be selected in the current frame as the current pixel. Using the current pixel as the center point, the brightness values ​​of a preset number of pixels within a preset distance from the center point are traversed. This can be understood as the two-dimensional feature extraction process first selecting a pixel as the center, setting a certain radius, and searching for a preset number of pixel values ​​within that radius. For example, 16 pixels are searched on a circular pattern with a radius of 3 pixels.

[0059] S230. Determine whether the brightness value of a preset number of pixels is greater than or equal to a preset pixel brightness threshold. If yes, execute S240; otherwise, return to execute S220.

[0060] In this embodiment, it determines whether the brightness value of a preset number of pixels is greater than or equal to a preset pixel brightness threshold. If the threshold is met, the current pixel is designated as a key point, and the next pixel is selected and designated as the current pixel. The process then returns to the step of traversing the brightness values ​​of a preset number of pixels within a preset distance from the current pixel as the center point, until all pixels in the current frame have been determined. If the threshold is not met, the process returns to the step of arbitrarily selecting any pixel in the current frame as the current pixel, until the brightness value of the preset number of pixels satisfies the condition that the brightness value of the preset number of pixels is greater than or equal to the preset pixel brightness threshold. This can be understood as follows: if the brightness difference between the searched pixel and the selected pixel exceeds a predetermined threshold (e.g., 20%) for 12 consecutive pixels, the selected pixel will be considered a key point and selected as a feature. Generally, these key points are located in the corners, edges, and blocky areas of the image.

[0061] S240. Take the current pixel as the key point, select the next pixel, take the next pixel as the current pixel, and return to the step of traversing the brightness values ​​of a preset number of pixels within a preset distance from the current pixel as the center point, until all pixels in the current frame have been judged.

[0062] In this embodiment, if the brightness value of a preset number of pixels is greater than or equal to a preset pixel brightness threshold, the current pixel is taken as a key point, and the next pixel is selected. The next pixel is taken as the current pixel, and the process of traversing the brightness values ​​of a preset number of pixels within a preset distance from the current pixel as the center point is repeated until all pixels in the current frame have been judged.

[0063] S250. Use a preset multi-scale image pyramid method to perform multi-scale sampling on the extracted key points to obtain multi-scale target key points, and collect the multi-scale target key points.

[0064] In this embodiment, the preset multi-scale image pyramid method can be understood as a multi-scale signal representation method for images, where the signal or image undergoes repeated smoothing and secondary sampling. By performing multi-scale pixel sampling on the original image, N images of different resolutions are generated. The preset multi-scale image pyramid method can be used to perform multi-scale sampling on extracted key points to obtain multi-scale target key points, and to collect multi-scale target key points extracted from different levels.

[0065] S260. The orientation of the multi-scale target key points is determined by the preset intensity centroid method, and the multi-scale target key points with the determined orientation are taken as the first key points.

[0066] In this embodiment, the preset intensity centroid method can also be called the intensity centroid method. The direction of the key point can be calculated using the intensity centroid method (first moment). That is, the direction of the multi-scale target key point is determined by the intensity centroid method (first moment). The intensity centroid can be considered as the position of the average pixel intensity in a given image. It can be understood as using the fact that the intensity of the angle is its center offset to determine the direction, and the multi-scale target key point with the determined direction is taken as the first key point.

[0067] S270. Determine the second key point of the target object in each historical image frame in each historical consecutive frame.

[0068] In this embodiment, to ensure the scale and rotation invariance of the vehicle during movement, a multi-scale image pyramid and intensity centroid algorithm are used to further process the extracted key points. In this embodiment, the image pyramid represents a single image at multiple scales, with each layer being a downsampled version of the original image. All key points extracted from different layers are collected. Finally, to further distinguish the key points, a descriptor is used, which is a 256-bit binary feature vector containing only 1s and 0s. Each bit is calculated by comparing the brightness of surrounding pixels according to a pre-defined pattern.

[0069] S280, in response to the identification of historical consecutive frames as key image frames, take each global key point at each historical moment as a stable key point.

[0070] In this embodiment, a key image frame is defined as a frame in which the change of global key points corresponding to each target object in two consecutive image frames exceeds a preset range. This can be understood as a frame being determined based on whether there are significant changes in the environment surrounding the vehicle. In response to the recognition of historical consecutive frames as key image frames, each global key point at each historical moment is taken as a stable key point.

[0071] S290. Triangulation is performed on the stable key points to obtain the first change information of each global key point in the historical continuous frame image in at least two historical continuous frames.

[0072] The first change information represents the first depth information of each global key point in the historical consecutive frames.

[0073] In this embodiment, triangulation of stable key points can be used to obtain the first change information of each global key point in at least two historical consecutive frames in the historical continuous frame image. This can be understood as obtaining the depth information of pixels through continuous frame images. By obtaining multiple frames, the change of key points in the image frames can be obtained, and the distance information of unified key points in multiple frame images can be obtained. Triangulation refers to determining the distance of a point by observing the angle between the same point in two places.

[0074] S2100: Generate global map points corresponding to each historical consecutive frame based on the first change information.

[0075] In this embodiment, global map points of the environment are used to detect global spatial tearing and global spatial transfer violations. Consistent with the ORB-SLAM framework, the conversion of stable keypoints to 3D map points for a frame is only performed when the current frame is identified as a keyframe. Keyframes are typically determined based on whether there are significant changes in the vehicle's surrounding environment. This method significantly reduces the required computation frequency. Furthermore, there is no need to construct an environment map; a window mechanism is used to store only the map points generated from N consecutive keyframes. Based on this method, the storage requirements increased by map expansion are greatly reduced.

[0076] S2110. In response to the bounding box event, extract the local key points of each target object in the bounding box at different times.

[0077] In this embodiment, in response to the triggering of a bounding box event, local key points of each target object within the bounding box at different times are extracted. In this embodiment, optimization is performed only in two specific cases: (1) optimization is performed when the bounding box is first detected. This detection can identify whether an obstacle exists or whether an object is misclassified; (2) optimization is performed when the bounding box suddenly disappears or its estimated type changes. This detection can determine whether an obstacle is hidden or misclassified. In other cases, no nonlinear optimization and related operations are required, thus greatly reducing the computational overhead of the detection method. The bounding box events include: the first detection of the bounding box and the sudden disappearance of the bounding box.

[0078] S2120. Triangulation is performed on the local key points to obtain the second change information of each local key point in the bounding box in the historical continuous frames.

[0079] The second change information represents the second depth information of each local key point in the historical continuous frames.

[0080] In this embodiment, triangulation can be performed on local key points to obtain second change information of each local key point in the bounding box in historical consecutive frames.

[0081] S2130. Generate local map points corresponding to the historical continuous frames based on the second change information.

[0082] In this embodiment, to detect violations such as target entity confusion, target semantic confusion, and target flickering, additional distance metrics are used to convert the extracted keypoints in each bounding box into 3D map points. Subsequently, these feature points are nonlinearly optimized with the corresponding matching observation keypoints.

[0083] S2140. Project the global map points into the two-dimensional space of the current frame to obtain the distribution of the first predicted key points at the current moment.

[0084] The first predicted keypoint distribution refers to the predicted keypoint distribution obtained by projecting global map points into the two-dimensional space of the current frame.

[0085] In this embodiment, global map points are projected onto the two-dimensional space of the current frame to obtain the distribution of the first predicted key points at the current moment.

[0086] S2150. Traverse through the first key points to find the first matching key points that match the distribution of the first predicted key points, and obtain the corresponding first matching key pairs.

[0087] In this embodiment, the first matching key point is found by traversing through the first key point and matching the distribution of the first predicted key point, thus obtaining the corresponding first matching key pair.

[0088] S2160. Determine the first offset error corresponding to each first matching key pair, and determine the average value of the first offset error corresponding to each first matching key pair based on each first offset error, and use the average value of the first offset error as the key point matching error.

[0089] In this embodiment, the first offset error corresponding to each first matching key pair is determined, and the average value of the first offset error corresponding to the first matching key pair is determined based on each first offset error. The average value of the first offset error is used as the key point matching error.

[0090] S2170. Obtain the first key point corresponding to the current frame in the first bounding box, determine the first distance between the first key point in the first bounding box and the vehicle, and determine the second distance between the local map point in the second bounding box and the vehicle.

[0091] In this embodiment, the first key point corresponding to the current frame in the first bounding box is obtained, the first distance between the first key point in the first bounding box and the vehicle is determined, and the second distance between the local map point in the second bounding box and the vehicle is determined.

[0092] S2180. Map the local map points in the second bounding box with the second distance to the first bounding box with the first distance to obtain the second predicted key point distribution.

[0093] In this embodiment, local map points in a second bounding box with a second distance are mapped to a first bounding box with a first distance to obtain a second predicted keypoint distribution.

[0094] S2190. Traverse through the second key points to find the second matching key points that match the distribution of the second predicted key points, and obtain the corresponding second matching key pairs.

[0095] In this embodiment, the second keypoints are traversed to find the second matching keypoints that match the distribution of the second predicted keypoints, and the corresponding second matching key pairs are obtained.

[0096] S2200. Determine the second offset error corresponding to each second matching key pair, and determine the average value of the second offset error corresponding to each second matching key pair based on each second offset error. Use the average value of the second offset error as the key point matching error.

[0097] In this embodiment, the second offset error corresponding to each second matching key pair is determined, and the average value of the second offset error corresponding to the second matching key pair is determined based on each second offset error. The average value of the second offset error is used as the key point matching error.

[0098] S2210. Detect sensor deception attacks based on one of the following: vehicle location information, bounding box information in front of the vehicle, type of target object in front of the vehicle, and key point matching error.

[0099] In this embodiment, the scene data includes: vehicle position information, bounding box information in front of the vehicle, and the type of target object in front of the vehicle; wherein, the position information includes: the vehicle's current position information at the current moment, and the vehicle's position information at the previous moment; the bounding box information includes: first bounding box information and second bounding box information.

[0100] In this embodiment, sensor spoofing attacks can be detected based on any one of the following: vehicle position information, bounding box information in front of the vehicle, and the type of target object in front of the vehicle, along with the keypoint matching error. Specifically, when the scene data is vehicle position information, the positional deviation between the current position and the previous position is obtained based on the vehicle's current position and the position information of the vehicle at the previous moment. Sensor spoofing attacks are then detected based on this positional deviation and the keypoint matching error. When the scene data is bounding box information, sensor spoofing attacks can also be detected using the bounding box information and the keypoint matching error. When the scene data is the type of target object, sensor spoofing attacks are detected based on the type of target object and the keypoint matching error.

[0101] In one embodiment, detecting sensor spoofing attacks based on vehicle location information and key point matching errors includes:

[0102] Determine the positional deviation between the current positional information and the positional information at the previous moment, and compare the positional deviation with a preset positional deviation threshold to obtain the comparison result;

[0103] If the key point matching error reaches a preset error threshold, and the position deviation is greater than or equal to the preset position deviation threshold, a first sensor spoofing attack is determined to have occurred.

[0104] If the key point matching error does not reach the preset error threshold, the mapping error is determined, resulting in a second sensor spoofing attack.

[0105] The preset position deviation threshold refers to the position deviation threshold between the current position information and the position information at the previous moment. This position deviation threshold is a value defined by the user based on experience or requirements, and this embodiment does not impose any restrictions on it. The preset error threshold refers to the key point matching error threshold obtained by converting the second key point into a 3D map point and then mapping the 3D map point back to the 2D space where the first key point is located. This key point matching error threshold is also a value defined by the user based on experience or requirements, and this embodiment does not impose any restrictions on it.

[0106] In this embodiment, the positional deviation between the current position information and the previous position information is obtained by calculating the difference between the current position information and the previous position information. The positional deviation is then compared with a preset positional deviation threshold to obtain a comparison result. The comparison result includes: the positional deviation is greater than or equal to the preset positional deviation threshold, and the positional deviation is less than the preset positional deviation threshold. If the keypoint matching error reaches the preset error threshold and the positional deviation is greater than or equal to the preset positional deviation threshold, a first sensor spoofing attack is determined to occur. If the keypoint matching error does not reach the preset error threshold, a mapping error is determined, and a second sensor spoofing attack is generated. This can be understood as follows: if the keypoint matching error is relatively small and almost negligible, but the positional deviation is large, a first sensor spoofing attack is determined to occur; if the keypoint matching error is large, a mapping error is determined, and a second sensor spoofing attack is generated.

[0107] In one embodiment, detecting sensor spoofing attacks based on bounding box information in front of the vehicle and key point matching errors includes:

[0108] If the key point matching error reaches a preset error threshold, and the first or second bounding box information in the bounding box information is missing, a third sensor spoofing attack is determined to occur; wherein, the first bounding box information is the first key point contained in the first bounding box, and the second bounding box information is the local map point contained in the second bounding box;

[0109] If the key point matching error does not reach the preset error threshold, a fourth sensor spoofing attack is determined.

[0110] In this embodiment, a third sensor spoofing attack can be determined when the key point matching error reaches a preset error threshold and the first or second bounding box information in the bounding box information is absent; wherein, the first bounding box information is the first key point contained in the first bounding box, and the second bounding box information is the local map point contained in the second bounding box; if the key point matching error does not reach the preset error threshold, a fourth sensor spoofing attack can be determined.

[0111] In one embodiment, the scene data includes: the type of the target object; correspondingly, detecting sensor spoofing attacks based on the type of the target object in front of the vehicle and key point matching errors includes:

[0112] If the object type of the key point obtained by mapping the local map point in the second bounding box to the first bounding box changes, determine the type of the target object in the local map point.

[0113] A fifth sensor deception attack is determined to occur when the key point matching error reaches a preset error threshold and the type of the target object is inconsistent with the object type of the key points obtained in the first bounding box.

[0114] In this embodiment, the type of the target object in the local map point can be determined when the object type of the key point obtained by mapping the local map point in the second bounding box to the first bounding box changes; and a fifth sensor spoofing attack is determined when the key point matching error reaches a preset error threshold and the type of the target object is inconsistent with the object type of the key point obtained in the first bounding box.

[0115] The technical solution of this invention involves arbitrarily selecting a pixel in the current frame as the current pixel, using the current pixel as the center point, traversing the brightness values ​​of a preset number of pixels within a preset distance from the center point, and determining whether the brightness values ​​of the preset number of pixels are greater than or equal to a preset pixel brightness threshold. If so, the current pixel is designated as a key point, and the next pixel is selected and designated as the current pixel. The process is repeated until all pixels in the current frame have been determined. A preset multi-scale image pyramid method is used to perform multi-scale sampling on the extracted key points to obtain multi-scale target key points. These multi-scale target key points are collected and then analyzed using a preset intensity quality... The method determines the orientation of multi-scale target key points and uses these orientation-determined multi-scale target key points as the first key points. When the second key points include both global and local key points, each second key point is converted into a 3D map point. These converted 3D map points are then mapped into the 2D space containing the first key point for key point matching, thus obtaining the corresponding key point matching error. Based on one of the following: vehicle position information, bounding box information in front of the vehicle, or the type of the target object in front of the vehicle, and the key point matching error, sensor deception attacks are detected. This further enables defense and identification of all sensor attacks and deceptions, with detection latency controlled within 100 milliseconds, thereby improving autonomous driving safety.

[0116] In one embodiment, to facilitate a better understanding of the spatial consistency model in sensor spoofing attack detection, Figure 3 This is an architectural diagram of a spatial consistency model provided in an embodiment of the present invention. Figure 4 This invention provides detection schemes for each type of deception attack. For example... Figure 3 As shown, (1) Two-dimensional feature extraction: Its purpose is to extract two-dimensional feature points from the current perceived image. It is based on the orientation FAST and rotation BRIEF (ORB) algorithm because the algorithm can provide real-time performance without GPU and has good invariance to changes in viewpoint and illumination. (2) Feature prediction and generation: Its purpose is to predict three-dimensional feature points using historical environmental states and generate fake three-dimensional feature points through deceptive sensor data. (3) Match search: Its purpose is to identify key point matching pairs from the extracted two-dimensional key points and three-dimensional predicted points or fake points. To do this, we first need to project the three-dimensional map points in the physical world onto the two-dimensional coordinates in the current frame and estimate their pose. (4) Offset error calculation: Its purpose is to find suitable pose and environmental states by minimizing all error edges by traversing the error edges of each pair of matching objects. After multiple optimizations, the total average error is calculated, and matching pairs with errors exceeding the threshold are identified as outliers.

[0117] like Figure 4 As shown, various detection strategies for different spatial consistency conflicts are summarized. The common goal of these strategies is to generate the current state of an erroneous environment. 1) Global Spatial Tear: Global spatial tearing is characterized by a significant deviation between the spoofed position and the vehicle's actual position. However, the feature point offset error between the previous and current frames is very small and negligible. Therefore, the recognition conditions that must be met for this attack to occur are introduced. 2) Global Spatial Movement: The aircraft's position may shift slightly, but the resulting change in orientation is drastically different. To address this, a method is proposed to obtain the current control output by analyzing the spoofed IMU or GPS data and comparing it with the previous pose. 3) Target Entity Fusion: The vehicle either fails to detect obstacles or incorrectly identifies obstacles that do not exist ahead. To counter the obstacle appearance attack, a method is proposed that involves extracting all map points within the bounding box. These 3D points are then converted into spoofed environmental features using distances estimated by an additional objection detection module. These spoofed depth data may project incorrect levels. The error in the nonlinear optimization process will increase significantly due to the difference in distance between the real observation points and the deceptive estimated features; 4) Target semantic fusion: Vehicles may misclassify objects, incorrectly identifying one object or lane as another. To address this issue, a method similar to obstacle attack detection is proposed. However, instead of appending obstacle distances to the current map points, the distances obtained through triangulation of the original keypoints are utilized. Once the detected object type changes, the distribution of its feature points is expected to change accordingly. Therefore, a large difference will appear between the observed and predicted observations. This difference can be used to identify such attacks.

[0118] In one embodiment, Figure 5 This is an architecture diagram for detecting sensor spoofing attacks provided in an embodiment of the present invention, as shown below. Figure 5As shown, the system includes a global detector, a local detector, and a plotter: The global detector aims to identify sensor spoofing attacks caused by interference and changes in the global space. First, 2D keypoints are extracted from each frame captured by the camera. Then, these features are used to generate 3D map points and construct optimized matching pairs. Historical map points are transferred to the current moment and projected into 2D space. These projected map points search for the best features among the extracted keypoints to form the best matching pair and calculate the error. Finally, these errors are combined with contextual data to identify global spoofing attacks. The local detector aims to identify sensor spoofing attacks originating from target entities and semantic obfuscation. Similar to the global detector, it projects historical 3D map points in a bounding box (identifying a bounding box containing keypoints at a distance from the vehicle) onto real-time acquired 2D keypoints, which are also in the bounding box at a certain distance from the vehicle. It maps the 3D map points of the previous bounding box at a certain distance onto the 2D bounding box for matching, identifying attacks through errors and the distance of the bounding boxes. Then, these features are used to find the best matching points among the keypoints extracted from the bounding boxes, forming the best pair and calculating the error. Ultimately, these errors, along with the bounding boxes, are used to identify local spoofing attacks. The plotter is designed to generate global and local 3D map points from historical 2D keyframes. Global map points are constructed from keyframes extracted from two adjacent keyframes, while local map points are generated from keyframes extracted from bounding boxes of two consecutive frames. Since map points generated from keyframes are sparse in local space, we choose to use frames instead of keyframes to generate local map points. This sparsity leads to significant error fluctuations when detecting local sensor spoofing attacks.

[0119] In one embodiment, Figure 6 Table 1 presents a performance overhead comparison diagram of an architecture for detecting sensor spoofing attacks according to an embodiment of the present invention. Table 1 compares the current success rate and latency solutions provided in this embodiment; Table 2 presents an evaluation result table, which is an experimental table of real-world performance in detecting sensor spoofing attacks. Figure 6 As can be seen from Table 1, the present invention is significantly superior to the prior art.

[0120] Table 1: Comparison of Current Success Rates and Latency Solutions

[0121]

[0122] Table 2: Evaluation Results Table

[0123] Benign / 0° NO N / A N / A 0％ Benign / 30° NO N / A N / A 0％ Benign / 45° NO N / A N / A 0％ Adversarial / 0° Yes 100％ 0.2s N / A Adversarial / 30° Yes 100％ 0.1s N / A Adversarial / 45° Yes 100％ 0.1s N / A

[0124] In one embodiment, Figure 7This is a structural block diagram of a sensor spoofing attack detection device according to an embodiment of the present invention. This device is suitable for defending against and identifying all types of sensor spoofing attacks and can be implemented in hardware or software. It can be configured in an electronic device to implement a sensor spoofing attack detection method according to an embodiment of the present invention.

[0125] like Figure 7 As shown, the device includes: an acquisition module 710, a key point determination module 720, an error determination module 730, and a detection module 740.

[0126] The acquisition module 710 is used to acquire the current frame collected by the sensors in the vehicle, as well as at least two consecutive historical frames preceding the current frame.

[0127] The key point determination module 720 is used to determine the first key point of the target object in the current frame according to the preset orientation fast rotation algorithm, and to determine the second key point of the target object in each historical image frame in each historical continuous frame, and to convert each second key point into a three-dimensional map point.

[0128] The error determination module 730 is used to perform key point matching based on the mapping relationship between the three-dimensional map points and the first key point to obtain the key point matching error.

[0129] The detection module 740 is used to acquire scene data corresponding to the vehicle and detect sensor spoofing attacks based on the scene data and the key point matching error.

[0130] In this embodiment of the invention, a key point determination module determines the first key point of the target object in the current frame using a preset directional fast rotation algorithm, determines the second key point of the target object in each historical image frame in each historical consecutive frame, and converts each second key point into a 3D map point. An error determination module performs key point matching based on the mapping relationship between the 3D map points and the first key point to obtain the key point matching error. A detection module acquires the scene data corresponding to the vehicle and detects sensor deception attacks based on the scene data and the key point matching error. This allows for defense and identification of all sensor attacks, and the detection latency can be controlled within 100 milliseconds, thereby improving the safety of autonomous driving.

[0131] In one embodiment, the second key point includes: a global key point and a local key point; the 3D map point includes: a global map point and a local map point; the local key point and the local map point are located within a bounding box; the bounding box includes a first bounding box and a second bounding box.

[0132] In one embodiment, the key point determination module 720 includes:

[0133] The traversal unit is used to arbitrarily select a pixel in the current frame as the current pixel, and traverse the brightness values ​​of a preset number of pixels within a preset distance from the current pixel as the center point.

[0134] The judgment unit is used to determine whether the brightness value of the preset number of pixels is greater than or equal to the preset pixel brightness threshold.

[0135] The first determining unit is used to, if satisfied, take the current pixel as the key point, select the next pixel, take the next pixel as the current pixel, and return to the step of taking the current pixel as the center point and traversing the brightness values ​​of a preset number of pixels contained within a preset distance of the center point, until all pixels contained in the current frame have been judged.

[0136] The second determining unit is used to return to the step of arbitrarily selecting a pixel in the current frame as the current pixel if the condition is not met, until the brightness value of the preset number of pixels is greater than or equal to the preset pixel brightness threshold.

[0137] The collection unit is used to perform multi-scale sampling on the extracted key points using a preset multi-scale image pyramid method to obtain multi-scale target key points, and to collect the multi-scale target key points.

[0138] The orientation determination unit is used to determine the orientation of the multi-scale target key points using a preset intensity centroid method, and to use the multi-scale target key points with determined orientations as the first key points.

[0139] In one embodiment, the second key point is a global key point, and the 3D map point is a global map point; correspondingly, the key point determination module 720 further includes:

[0140] The identification unit is configured to, in response to the historical continuous frames being identified as key image frames, take each of the global key points at each historical moment as stable key points; wherein, the key image frame is defined as the global key points corresponding to the target object in two consecutive image frames where the change exceeds a preset change range.

[0141] The first measurement unit is used to perform triangulation on the stable key points to obtain first change information of each global key point in at least two historical consecutive frames in the historical consecutive frame image; wherein, the first change information represents the first depth information of each global key point in the historical consecutive frames.

[0142] The first generation unit is used to generate global map points corresponding to each of the historical consecutive frames based on the first change information.

[0143] In one embodiment, the second key point is a local key point, and the 3D map point is a local map point; correspondingly, the key point determination module 720 further includes:

[0144] An extraction unit is used to extract local key points of each target object within the bounding box at different times in response to a bounding box event; wherein the bounding box event includes: the first detection of the bounding box and the sudden disappearance of the bounding box;

[0145] The second measurement unit is used to perform triangulation on the local key points to obtain second change information of each local key point in the bounding box in the historical continuous frames; wherein, the second change information represents the second depth information of each local key point in the historical continuous frames.

[0146] The second generation unit is used to generate local map points corresponding to the historical continuous frames based on the second change information.

[0147] In one embodiment, the three-dimensional map points are global map points; correspondingly, the error determination module 730 includes:

[0148] The first distribution determination unit is used to project the global map points onto the two-dimensional space where the current frame is located to obtain the first predicted key point distribution at the current moment;

[0149] The search unit is used to traverse the first key points to find the first matching key points that match the distribution of the first predicted key points, and obtain the corresponding first matching key pair;

[0150] An error determination unit is used to determine the first offset error corresponding to each of the first matching key pairs, and to determine the average value of the first offset error corresponding to each of the first offset errors, and to use the average value of the first offset error as the key point matching error.

[0151] In one embodiment, the 3D map point is a local map point, and the local map point is located within a bounding box; the bounding box includes a first bounding box and a second bounding box; correspondingly, the error determination module 730 includes:

[0152] The acquisition unit is used to acquire the first key point corresponding to the current frame in the first bounding box;

[0153] The determining unit is used to determine a first distance between the first bounding box where the first key point is located and the vehicle, and to determine a second distance between the second bounding box where the local map point is located and the vehicle;

[0154] The second distribution determination unit is used to map local map points in a second bounding box with the second distance to a first bounding box with the first distance to obtain a second predicted key point distribution.

[0155] The search unit is used to traverse the second key points to find the second matching key points that match the distribution of the second predicted key points, and obtain the corresponding second matching key pair;

[0156] An error determination unit is used to determine the second offset error corresponding to each of the second matching key pairs, and to determine the average value of the second offset error corresponding to each of the second offset errors, and to use the average value of the second offset error as the key point matching error.

[0157] In one embodiment, the scene data includes: vehicle position information, bounding box information in front of the vehicle, and the type of target object in front of the vehicle; wherein, the position information includes: the vehicle's current position information at the current moment, and the vehicle's position information at the previous moment before the current moment; the bounding box information includes: first bounding box information and second bounding box information.

[0158] In one embodiment, the detection module 740 includes:

[0159] The detection unit is used to detect sensor deception attacks based on one of the following: the vehicle's position information, the bounding box information in front of the vehicle, the type of the target object in front of the vehicle, and the key point matching error.

[0160] In one embodiment, the detection unit includes:

[0161] The deviation determination unit is used to determine the position deviation between the current position information and the position information at the previous moment, and compare the position deviation with a preset position deviation threshold to obtain a comparison result.

[0162] The first determining unit is configured to determine that a first sensor spoofing attack has occurred when the key point matching error reaches the preset error threshold and the position deviation is greater than or equal to the preset position deviation threshold.

[0163] The second determining unit is used to determine a mapping error and generate a second sensor spoofing attack when the key point matching error does not reach the preset error threshold.

[0164] In one embodiment, the detection unit further includes:

[0165] The third determining unit is configured to determine that a third sensor spoofing attack has occurred when the key point matching error reaches the preset error threshold and the first or second bounding box information in the bounding box information does not exist; wherein the first bounding box information is the first key point contained in the first bounding box, and the second bounding box information is the local map point contained in the second bounding box.

[0166] The fourth determining unit is used to determine that a fourth sensor spoofing attack has occurred if the key point matching error does not reach the preset error threshold.

[0167] In one embodiment, the detection unit further includes:

[0168] A type determination unit is used to determine the type of the target object in the local map point when the object type of the local map point mapped to the key point in the first bounding box changes;

[0169] The fifth determining unit is used to determine that a fifth sensor deception attack occurs when the key point matching error reaches the preset error threshold and the type of the target object is inconsistent with the object type of the key point obtained in the first bounding box.

[0170] The sensor spoofing attack detection device provided in this embodiment of the invention can execute the sensor spoofing attack detection processing method provided in any embodiment of the invention, and has the corresponding functional modules and beneficial effects of the execution method.

[0171] In one embodiment, Figure 8 This is a schematic diagram of an electronic device provided for an embodiment of the present invention. The electronic device 10 is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices (such as helmets, glasses, watches, etc.), and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the invention described and / or claimed herein.

[0172] like Figure 8As shown, the electronic device 10 includes at least one processor 11 and a memory, such as a read-only memory (ROM) 12 or a random access memory (RAM) 13, communicatively connected to the at least one processor 11. The memory stores computer programs executable by the at least one processor. The processor 11 can perform various appropriate actions and processes based on the computer program stored in the ROM 12 or loaded from storage unit 18 into the RAM 13. The RAM 13 may also store various programs and data required for the operation of the electronic device 10. The processor 11, ROM 12, and RAM 13 are interconnected via a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.

[0173] Multiple components in electronic device 10 are connected to I / O interface 15, including: input unit 16, such as keyboard, mouse, etc.; output unit 17, such as various types of displays, speakers, etc.; storage unit 18, such as disk, optical disk, etc.; and communication unit 19, such as network card, modem, wireless transceiver, etc. Communication unit 19 allows electronic device 10 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0174] Processor 11 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. Processor 11 performs the various methods and processes described above, such as sensor spoofing attack detection methods.

[0175] In some embodiments, the sensor spoofing attack detection processing method may be implemented as a computer program tangibly contained in a computer-readable storage medium, such as storage unit 18. In some embodiments, part or all of the computer program may be loaded and / or installed on electronic device 10 via ROM 12 and / or communication unit 19. When the computer program is loaded into RAM 13 and executed by processor 11, one or more steps of the sensor spoofing attack detection method described above may be performed. Alternatively, in other embodiments, processor 11 may be configured to perform the sensor spoofing attack detection method by any other suitable means (e.g., by means of firmware).

[0176] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), payload-programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0177] Computer programs used to implement the methods of the present invention can be written in any combination of one or more programming languages. These computer programs can be provided to the processor of a general-purpose computer, a special-purpose computer, or other programmable sensor spoofing attack detection device, such that when executed by the processor, the computer programs cause the functions / operations specified in the flowcharts and / or block diagrams to be implemented. The computer programs can be executed entirely on a machine, partially on a machine, as a standalone software package partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0178] In the context of this invention, a computer-readable storage medium can be a tangible medium that may contain or store a computer program for use by or in conjunction with an instruction execution system, apparatus, or device. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination thereof. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0179] To provide interaction with a user, the systems and techniques described herein can be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the electronic device. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0180] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as data servers), or computing systems that include middleware components (e.g., application servers), or computing systems that include frontend components (e.g., user computers with graphical user interfaces or web browsers through which users can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., communication networks). Examples of communication networks include local area networks (LANs), wide area networks (WANs), blockchain networks, and the Internet.

[0181] A computing system can include clients and servers. Clients and servers are generally located far apart and typically interact through communication networks. The client-server relationship is created by computer programs running on the respective computers and having a client-server relationship with each other. The server can be a cloud server, also known as a cloud computing server or cloud host, which is a hosting product within the cloud computing service system to address the shortcomings of traditional physical hosts and VPS services, such as high management difficulty and weak business scalability.

[0182] It should be understood that the various forms of processes shown above can be used, with steps reordered, added, or deleted. For example, the steps described in this invention can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution of this invention can be achieved, and this is not limited herein.

[0183] The specific embodiments described above do not constitute a limitation on the scope of protection of this invention. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this invention should be included within the scope of protection of this invention.

Claims

1. A method for detecting sensor spoofing attacks, characterized in that, include: Acquire the current frame collected by the sensors in the vehicle, as well as at least two consecutive historical frames preceding the current frame; The first key point of the target object in the current frame is determined according to the preset orientation fast rotation algorithm, and the second key point of the target object in each historical image frame in each historical continuous frame is determined, and each second key point is converted into a three-dimensional map point. The key point matching error is obtained by performing key point matching based on the mapping relationship between the 3D map points and the first key point. Obtain scene data corresponding to the vehicle, and detect sensor spoofing attacks based on the scene data and the key point matching error; The step of determining the first key point of the target object in the current frame according to a preset directional fast rotation algorithm includes: In the current frame, any pixel is selected as the current pixel. Using the current pixel as the center point, the brightness values ​​of a preset number of pixels within a preset distance from the center point are traversed. Determine whether the brightness value of the preset number of pixels is greater than or equal to a preset pixel brightness threshold; If satisfied, the current pixel is taken as the key point, and the next pixel is selected. The next pixel is taken as the current pixel. The process of returning to the step of taking the current pixel as the center point and traversing the brightness values ​​of a preset number of pixels within a preset distance from the center point continues until all pixels in the current frame have been judged. If not satisfied, return to the step of arbitrarily selecting a pixel in the current frame as the current pixel until the brightness value of the preset number of pixels is greater than or equal to the preset pixel brightness threshold. Multi-scale target key points are obtained by multi-scale sampling of the extracted key points using a preset multi-scale image pyramid method, and the multi-scale target key points are collected. The orientation of the multi-scale target key points is determined using the preset intensity centroid method, and the multi-scale target key points with determined orientations are used as the first key points.

2. The method according to claim 1, characterized in that, The second key point is a global key point, and the 3D map point is a global map point; correspondingly, converting each of the second key points into a 3D map point includes: In response to the historical consecutive frames being identified as key image frames, each of the global key points at each historical moment is taken as a stable key point; wherein, the key image frame is one in which the change of the global key points corresponding to the target object in two consecutive image frames exceeds a preset change range. Triangulation is performed on the stable key points to obtain the first change information of each global key point in the historical continuous frame image in at least two historical continuous frames; wherein, the first change information represents the first depth information of each global key point in the historical continuous frames. Based on the first change information, global map points corresponding to each of the historical consecutive frames are generated.

3. The method according to claim 1, characterized in that, The second key point is a local key point, and the 3D map point is a local map point; correspondingly, the step of converting each of the second key points into a 3D map point further includes: In response to a bounding box event, local key points of each target object within the bounding box at different times are extracted; wherein, the bounding box event includes: the first detection of the bounding box and the disappearance of the bounding box; Triangulation is performed on the local key points to obtain the second change information of each local key point in the bounding box in the historical continuous frames; wherein, the second change information represents the second depth information of each local key point in the historical continuous frames; Based on the second change information, local map points corresponding to the historical continuous frames are generated.

4. The method according to claim 1, characterized in that, The 3D map points are global map points; correspondingly, the keypoint matching based on the mapping relationship between the 3D map points and the first keypoints to obtain the keypoint matching error includes: The global map points are projected onto the two-dimensional space of the current frame to obtain the distribution of the first predicted key points at the current moment; The first matching key point is found by traversing through the first key point and matching the first predicted key point distribution to obtain the corresponding first matching key pair. Determine the first offset error corresponding to each of the first matching key pairs, and determine the average value of the first offset error corresponding to each of the first offset errors, and use the average value of the first offset error as the key point matching error.

5. The method according to claim 1, characterized in that, The 3D map points are local map points, and the local map points are located within bounding boxes; the bounding boxes include a first bounding box and a second bounding box; correspondingly, the step of obtaining keypoint matching error based on the mapping relationship between the 3D map points and the first keypoints further includes: Obtain the first key point corresponding to the current frame within the first bounding box; Determine the first distance between the first key point located in the first bounding box and the vehicle, and determine the second distance between the local map point located in the second bounding box and the vehicle; Map local map points in a second bounding box with the second distance to a first bounding box with the first distance to obtain a second predicted keypoint distribution; Traverse the second keypoints to find the second matching keypoints that match the distribution of the second predicted keypoints, and obtain the corresponding second matching key pairs; Determine the second offset error corresponding to each of the second matching key pairs, and determine the average value of the second offset error corresponding to each of the second matching key pairs based on the second offset error. Use the average value of the second offset error as the key point matching error.

6. The method according to claim 1, characterized in that, The scene data includes: vehicle location information, bounding box information in front of the vehicle, and the type of target object in front of the vehicle; wherein, the location information includes: the vehicle's current location information at the current moment, and the vehicle's location information at the previous moment before the current moment; the bounding box information includes: first bounding box information and second bounding box information.

7. The method according to claim 6, characterized in that, Detection of sensor spoofing attacks based on the scene data and the key point matching error includes: The sensor deception attack is detected based on one of the following: the vehicle's location information, the bounding box information in front of the vehicle, the type of the target object in front of the vehicle, and the key point matching error.

8. The method according to claim 6, characterized in that, Detection of sensor spoofing attacks based on the vehicle's location information and the key point matching error includes: Determine the position deviation between the current position information and the previous position information, and compare the position deviation with a preset position deviation threshold to obtain a comparison result; If the key point matching error reaches a preset error threshold, and the position deviation is greater than or equal to the preset position deviation threshold, a first sensor spoofing attack is determined to have occurred. If the key point matching error does not reach the preset error threshold, a mapping error is determined, and a second sensor spoofing attack is generated.

9. The method according to claim 6, characterized in that, Detection of sensor spoofing attacks based on the bounding box information in front of the vehicle and the key point matching error includes: If the key point matching error reaches a preset error threshold, and the first or second bounding box information in the bounding box information is absent, a third sensor spoofing attack is determined to have occurred; wherein, the first bounding box information is the first key point contained in the first bounding box, and the second bounding box information is the local map point contained in the second bounding box; If the key point matching error does not reach the preset error threshold, a fourth sensor spoofing attack is determined to have occurred.

10. The method according to claim 6, characterized in that, Detection of sensor deception attacks based on the type of target object in front of the vehicle and the key point matching error includes: If the object type of the key point obtained by mapping a local map point in the second bounding box to the first bounding box changes, determine the type of the target object in the local map point. If the key point matching error reaches a preset error threshold, and the type of the target object is inconsistent with the object type of the key point obtained in the first bounding box, a fifth sensor spoofing attack is determined to have occurred.

11. A sensor spoofing attack detection device, characterized in that, include: The acquisition module is used to acquire the current frame collected by the sensors in the vehicle, as well as at least two consecutive historical frames preceding the current frame. The key point determination module is used to determine the first key point of the target object in the current frame according to the preset orientation fast rotation algorithm, and to determine the second key point of the target object in each historical image frame in each of the historical consecutive frames, and to convert each second key point into a three-dimensional map point. The error determination module is used to obtain the key point matching error by performing key point matching based on the mapping relationship between the three-dimensional map points and the first key point. The detection module is used to acquire scene data corresponding to the vehicle and detect sensor spoofing attacks based on the scene data and the key point matching error. The key point determination module includes: The traversal unit is used to arbitrarily select a pixel in the current frame as the current pixel, and traverse the brightness values ​​of a preset number of pixels within a preset distance from the current pixel as the center point. The judgment unit is used to determine whether the brightness value of the preset number of pixels is greater than or equal to the preset pixel brightness threshold. The first determining unit is used to, if satisfied, take the current pixel as the key point, select the next pixel, take the next pixel as the current pixel, and return to the step of taking the current pixel as the center point and traversing the brightness values ​​of a preset number of pixels contained within a preset distance of the center point, until all pixels contained in the current frame have been judged. The second determining unit is used to return to the step of arbitrarily selecting a pixel in the current frame as the current pixel if the condition is not met, until the brightness value of the preset number of pixels is greater than or equal to the preset pixel brightness threshold. The collection unit is used to perform multi-scale sampling on the extracted key points using a preset multi-scale image pyramid method to obtain multi-scale target key points, and to collect the multi-scale target key points. The orientation determination unit is used to determine the orientation of the multi-scale target key points using a preset intensity centroid method, and to use the multi-scale target key points with determined orientations as the first key points.

12. An electronic device, characterized in that, The electronic device includes: At least one processor; and A memory communicatively connected to the at least one processor; wherein, The memory stores a computer program that can be executed by the at least one processor, the computer program being executed by the at least one processor to enable the at least one processor to perform the sensor spoofing attack detection method according to any one of claims 1-10.

13. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions that, when executed by a processor, implement the sensor spoofing attack detection method according to any one of claims 1-10.

Images