Intrusion detection method and apparatus

Abstract

Embodiments of the present application provide an intrusion detection method and device. The method comprises: obtaining a first point cloud from a LiDAR sensor; obtaining first object information by applying a first perception process to the first point cloud; modifying the first point cloud to obtain a second point cloud; obtaining second object information by applying a second perception process to the second point cloud; and providing an intrusion result based on a comparison between the first object information and the second object information. The technical solution of the embodiments of the present application can identify adversarial attacks and improve the reliability of object detection.

Description

Technical Field

[0001] This disclosure relates generally to the field of object detection, and more particularly to an intrusion detection method and apparatus. Background Technology

[0002] Light detection and ranging (LiDAR) sensors play a crucial role in many fields. For example, in the field of autonomous driving, when a vehicle equipped with LiDAR sensors is in motion, the LiDAR sensors identify surrounding vehicles or obstacles to ensure the vehicle's safety.

[0003] Due to the nature of the technology, LiDAR sensors contain vulnerabilities that can be physically exploited. LiDAR sensors identify surrounding objects by emitting infrared (IR) pulses and receiving IR pulses reflected from them. The IR pulses received by the LiDAR sensor could be duplicated signals provided by an attacker. As a result, the malicious signal would lead to incorrect identification of surrounding objects.

[0004] Various intrusion detection methods have been proposed. In some implementations, the previous and current frames are compared to detect anomalies. Most normal objects can be similarly represented in almost all frames; and attacker-generated anomalous data may exist in some frames. However, due to the significant changes in information between consecutive frames during driving, anomaly detection results may be inaccurate. In other implementations, the shape of the identified object is compared to the expected shape of the target object in a ground truth database to detect anomalies. An object is considered anomalous when the correlation between the expected shape of the target object and the shape of the identified object exceeds a threshold. However, it is difficult to construct a ground truth database for all objects in the world, meaning that brute-force methods are practically inefficient. Summary of the Invention

[0005] This application provides an intrusion detection method and apparatus.

[0006] According to a first aspect, an intrusion detection method is provided. The intrusion detection method includes: obtaining a first point cloud from a LiDAR sensor; obtaining first object information by applying a first perception processing to the first point cloud; modifying the first point cloud to obtain a second point cloud; obtaining second object information by applying a second perception processing to the second point cloud; and providing an intrusion result based on a comparison between the first object information and the second object information.

[0007] In the technical solution provided in this application embodiment, adversarial attacks can be identified by comparing initial object information with modified object information. In other words, point cloud anomalies caused by attackers can be identified. Therefore, the method in this application embodiment can improve the reliability of object detection, which is beneficial for providing more accurate perception results for subsequent control strategies.

[0008] Furthermore, the technical solution provided in this application can detect intrusions on a per-object point cloud basis, rather than on a per-point basis. In other words, this technical solution can act on each object region individually, rather than detecting the entire point cloud. This technical solution targets object regions, and therefore is faster than analyzing all points in the scene.

[0009] For example, the first point cloud is the raw point cloud from a LiDAR sensor. The first point cloud includes a set of point clouds of objects. In other words, objects can be identified by sensing the first point cloud.

[0010] For example, object information includes at least one of the following: the object's position (x, y, z), the object's volume (width, height, length), or an objectivity score.

[0011] The first object information can be either identified object information or uncertain object information. Identified object information refers to object information with a high confidence score. Uncertain object information refers to object information with a low confidence score.

[0012] Optionally, modifying the first point cloud to obtain the second point cloud includes randomly modifying the first point cloud to obtain the second point cloud.

[0013] In one alternative implementation, the first perception process and the second perception process use the same perception algorithm.

[0014] In this way, no additional sensing algorithms are needed, reducing the demand for storage space; that is, no additional storage space needs to be provided for other sensing algorithms.

[0015] In one alternative implementation, modifying the first point cloud to obtain the second point cloud includes at least one of the following: adding one or more points to the first point cloud, or removing one or more points from the first point cloud.

[0016] It should be noted that the operation of modifying the first point cloud to the second point cloud can include one or more modifications.

[0017] In one alternative implementation, modifying the first point cloud to obtain the second point cloud includes: selecting an action from the following: add, remove, a combination of remove and add, a combination of reset and add, a combination of reset and remove, or a combination of reset, add and remove; locating one or more points to which the action will be performed; and performing the selected operation on the one or more points.

[0018] Optionally, an action may be selected from the following: selecting an action through deep reinforcement learning.

[0019] In this way, the action and position can be determined separately, which is conducive to obtaining more appropriate modifications and thus obtaining the test results as soon as possible.

[0020] In an alternative implementation, the method further includes locating the one or more points using deep reinforcement learning.

[0021] In one alternative implementation, providing an intrusion result based on a comparison between the first object information and the second object information includes: classifying the first object information as an intrusion when the comparison between the first object information and the second object information is equal to or greater than a first threshold.

[0022] In one alternative implementation, the method further includes at least one of the following: discarding the first object information, or sending the corrected object information to the planning module, wherein the corrected object information is obtained based on the second object information.

[0023] Sending corrected object information can be seen as a form of warning against intrusion.

[0024] For example, if the object corresponding to the first piece of information is an object added by the attacker, the first object information is discarded.

[0025] For example, when the object corresponding to the first piece of information is an object removed by an attacker, the corrected object information is provided to the planning module.

[0026] The corrected object information can be second object information.

[0027] When this method is applied to an autonomous driving system, it can improve driving safety and user experience. For example, when an object that should be identified is hidden by an attacker, the technical solution provided in this application can identify the hidden object and warn of intrusion to avoid accidents, including car crashes. For example, when an attacker creates an object that should not be identified, the technical solution provided in this application can identify the created object and discard its information to avoid freezing the vehicle or triggering emergency braking.

[0028] In one alternative implementation, providing an intrusion result based on a comparison between the first object information and the second object information includes classifying the first object information as benign when the comparison between the first object information and the second object information is less than a first threshold.

[0029] In one alternative implementation, the method further includes sending the first object information to the planning module.

[0030] In this way, information about objects classified as benign is passed to the planning module, which helps the planning module make correct decisions.

[0031] According to a second aspect, an intrusion detection device is provided. The intrusion detection device includes: an acquisition unit configured to: acquire a first point cloud from a LiDAR sensor; and acquire first object information by applying a first perception processing to the first point cloud; and a processing unit configured to modify the first point cloud to obtain a second point cloud; wherein the acquisition unit is further configured to: acquire second object information by applying a second perception processing to the second point cloud; and the processing unit is further configured to provide an intrusion result based on a comparison between the first object information and the second object information.

[0032] In the technical solution provided in this application embodiment, adversarial attacks can be identified by comparing initial object information with modified object information. In other words, point cloud anomalies caused by attackers can be identified. Therefore, the device in this application embodiment can improve the reliability of object detection, which is beneficial for providing more accurate perception results for subsequent control strategies.

[0033] Furthermore, the technical solution provided in this application can detect intrusions on a per-object point cloud basis, rather than on a per-point basis. In other words, this technical solution can act on each object region individually, rather than detecting the entire point cloud. This technical solution targets object regions, and therefore is faster than analyzing all points in the scene.

[0034] In one alternative implementation, the processing unit is specifically used to add one or more points to the first point cloud, or to remove one or more points from the first point cloud.

[0035] In an alternative implementation, the processing unit is also used to locate the one or more points through deep reinforcement learning.

[0036] In one alternative implementation, the processing unit is specifically used to classify the first object information as an intrusion when the comparison between the first object information and the second object information is equal to or greater than a first threshold.

[0037] In one alternative implementation, the processing unit is further configured to perform at least one of the following: discarding the first object information, or sending the corrected object information to the planning module, wherein the corrected object information is obtained based on the second object information.

[0038] In one alternative implementation, the processing unit is specifically used to classify the first object information as benign when the comparison between the first object information and the second object information is less than the first threshold.

[0039] In one alternative implementation, the processing unit is further configured to: send the first object information to the planning module.

[0040] According to a third aspect, an intrusion detection device is provided. The intrusion detection device includes a processor and a memory. The memory stores a computer program, and the processor executes the computer program stored in the memory and performs the method according to any one of the first aspects or possible implementations of the first aspect.

[0041] According to a fourth aspect, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores a computer program. When the computer program is executed by a processor, it implements the method according to any one of the first aspects or possible implementations of the first aspect.

[0042] According to a fifth aspect, a computer program product including instructions is provided. When the instructions are executed on a computer, the computer performs the method according to any one of the first aspects or possible implementations of the first aspect.

[0043] According to a sixth aspect, a chip is provided. The chip includes at least one processor and interface circuitry. The at least one processor obtains instructions stored in a memory through the interface circuitry and executes the method according to any one of the first aspects or possible implementations of the first aspect.

[0044] Optionally, the chip may further include a memory for storing instructions, and the processor is configured to execute the instructions stored in the memory. When the instructions are executed, the processor is configured to perform the method according to any one of the first aspect or possible implementations of the first aspect.

[0045] According to a seventh aspect, a terminal is provided. The terminal includes an intrusion detection device according to any one of the second aspect or any possible implementation thereof.

[0046] For example, the terminal could be a vehicle, a robot, or a drone.

[0047] In one alternative implementation, the terminal includes a LiDAR sensor. Attached Figure Description

[0048] Figure 1 This is a schematic diagram of the vehicle control process;

[0049] Figure 2 This is a diagram illustrating a point cloud-level attack;

[0050] Figure 3 This is a schematic block diagram of an intrusion detection module according to an embodiment of this application;

[0051] Figure 4 This is a schematic diagram of an intrusion detection method according to an embodiment of this application;

[0052] Figure 5 This is a schematic diagram of an intrusion detection method according to another embodiment of this application;

[0053] Figure 6 This is a schematic block diagram of an intrusion detection device according to an embodiment of this application; and

[0054] Figure 7 This is a schematic block diagram of an intrusion detection device according to another embodiment of this application. Detailed Implementation

[0055] The technical solution of this application will be further described in detail below with reference to the accompanying drawings and embodiments.

[0056] The technical solution of this application can be applied to fields requiring intrusion detection, such as autonomous driving, robot navigation, drone navigation, and virtual / augmented reality. The following mainly introduces the scenario of autonomous driving.

[0057] Figure 1 The data flow required to perceive the surrounding environment, formulate plans, and control the vehicle is described. In some embodiments, the vehicle is configured to be fully or partially autonomous. The vehicle relies on various sensors and perceptions to identify the surrounding environment, analyze the driving environment, and make decisions.

[0058] like Figure 1 As shown, the autonomous vehicle includes sensors, a perception module 220, a planning module 230, a control module 240, and actuators 250. One of the main sensors used in the vehicle is a LiDAR sensor 210. When the vehicle with the embedded LiDAR sensor is in motion, the LiDAR sensor identifies surrounding vehicles or obstacles to ensure the vehicle's safety.

[0059] LiDAR sensors use IR pulse signals to measure the distance, velocity, orientation, and various features of objects. Furthermore, LiDAR sensors can identify objects in the environment.

[0060] More specifically, a LiDAR sensor sends IR pulse signals to surrounding objects. These objects reflect the IR pulse signals from the LiDAR sensor, and the reflected IR pulse signals return to the receiver within the LiDAR sensor. Point clouds can then be obtained based on the reflected IR pulse signals.

[0061] like Figure 1 As shown, the perception module 220 applies perception processing to the point cloud to obtain information about the object, such as its position (x, y, z), volume (width, height, length), and objectivity score. The planning module 230 then formulates a control strategy based on the object information. For example, the planning module 230 formulates a control strategy to avoid or otherwise circumvent potential obstacles in the vehicle environment. Another example is that the planning module 230 formulates a control strategy to determine a driving path. Then, based on commands from the planning module 230, the control module 240 sends analog signals to the actuator 250.

[0062] When driving, the accuracy of object information is important for the control strategy of the planning module 230.

[0063] However, attackers can exploit the principles of LiDAR sensors to cause misidentification of surrounding objects. For example, because the threat of attack is not considered, the IR pulse signal received by the LiDAR sensor may be an attack signal. As a result, point cloud data based on the attack signal becomes offensive point cloud data, thus causing misidentification of surrounding objects.

[0064] In other words, attackers can inject or modify point clouds to produce false perception results, such as hiding existing objects or placing fake objects.

[0065] Figure 2 Two examples of this type of attack are shown. For example... Figure 2 As shown in (a), in a road scene, when an adversarial mesh is placed on surrounding vehicles, the perception module will be unable to identify the surrounding vehicles. In other words, it cannot detect the surrounding vehicles. An attacker can successfully hide a target vehicle by projecting an adversarial mesh onto it. Figure 2 As shown in (b), the attack model can modify the point cloud to hide or place objects. For example, an attacker creating a deceptive point cloud T by modifying the real point cloud might cause a real object to be hidden. An attacker creating an adversarial point cloud T' by modifying the real point cloud might cause a deceptive obstacle to be detected. Objects hidden by the attacker can lead to serious events, including car crashes. Fake objects placed by the attacker can freeze vehicles or trigger emergency braking.

[0066] Modified point clouds may affect the accuracy of perception results, leading to abnormal behavior and thus reducing vehicle reliability.

[0067] Previous methods for perceiving fake objects included adversarial training of the perception algorithm. This was only possible when various adversarial examples were available.

[0068] The technical solution of this application provides intrusion detection to protect vehicles with embedded LiDAR sensors from point cloud modification attacks, i.e., adversarial attacks. As a result, the reliability of autonomous driving is improved, which translates to enhanced safety and user experience.

[0069] Figure 1 One or more components can be installed individually or associated with the vehicle. Components can be coupled together in wired and / or wireless communication.

[0070] Optionally, the above components are just an example. In actual applications, these components can be added or removed as needed. Figure 1 This should not be construed as a limitation on the embodiments of this application.

[0071] The intrusion detection module provided in this application embodiment can be installed in devices such as vehicles, robots, or drones that are embedded with LiDAR sensors. It should be understood that the intrusion detection module can be an intrusion detection device.

[0072] For example, the intrusion detection module is located in Figure 2 Between the perception module 220 and the planning module 230. The perception module 220 sends the perception results to the intrusion detection module instead of the planning module 230.

[0073] The intrusion detection module in this application embodiment can also be called PointWiper.

[0074] like Figure 3 As shown, the intrusion detection module receives the perception results and point cloud.

[0075] The perceived outcome includes object information. For example, object information may include the object's position (x, y, z), its volume (width, height, length), or an objectivity score. The objectivity score, also known as the confidence score, represents the probability that the object is accurately identified. Figure 3 As shown, object information can be represented as (x, y, z, w, h, l, score…). Here, x, y, and z represent the object's coordinates. W represents the object's width. H represents the object's height. L represents the object's length. The score represents the object's objective score.

[0076] Optionally, the perception results include identified object information and uncertain object information with low confidence scores. Identified object information may refer to object information with high confidence scores.

[0077] For example, the point cloud can be a raw point cloud from the LiDAR sensor 210. The point cloud can be transmitted to the intrusion detection module 300 via the sensing module 220, or the point cloud can be transmitted directly from the LiDAR sensor 210 to the intrusion detection module 300.

[0078] The intrusion detection module 300 includes a modification unit 310 and a detection unit 320.

[0079] The modification unit 310 is used to modify the point cloud of an object from a LiDAR sensor to obtain a modified point cloud.

[0080] For example, modification unit 310 is used to perform at least one of the following: adding one or more points to the point cloud, or removing one or more points from the point cloud. For a detailed description, please refer to method 400 below, which will not be repeated here.

[0081] Optionally, the intrusion detection module 300 also includes a sensing unit 330. In this case, the sensing unit 330 can sense the modified point cloud.

[0082] Alternatively, the intrusion detection module 300 can send the modified point cloud to other devices to obtain the perception results of the modified point cloud. For example, the intrusion detection module 300 can send the modified point cloud to... Figure 1 The sensing module 220 in the middle. And the sensing module 220 can re-sensor the modified point cloud and return the sensing result of the modified point cloud.

[0083] In some possible embodiments, the perception algorithm used to identify point clouds from LiDAR sensors and the perception algorithm used to identify modified point clouds may be different.

[0084] In other alternative embodiments, the perception algorithm used to identify point clouds from LiDAR sensors and the perception algorithm used to identify modified point clouds can be the same.

[0085] The detection unit 320 is used to provide intrusion results by comparing the perception results based on the point cloud from the LiDAR sensor with the perception results of the modified point cloud.

[0086] In other words, the detection unit 320 is used to detect whether an object is benign or intrusive.

[0087] Optionally, the intrusion detection module 300 also includes an inference unit 340.

[0088] The inference unit 340 is used to determine whether further progress is needed, i.e. whether further modifications are needed to the currently modified point cloud.

[0089] For example, if no intrusion is detected, the inference unit 340 can be notified, and the inference unit 340 can determine whether further action is needed. If the inference unit 340 determines that further action is needed, it can notify the modification unit 310. If the inference unit 340 determines that no further action is needed, that is, the object is classified as benign, the inference unit 340 can provide the point cloud of the object from the LiDAR sensor to other devices, such as... Figure 1 Planning module 230 in the middle.

[0090] For a detailed description, please refer to Method 400 below, which will not be repeated here.

[0091] Figure 4 This is a schematic diagram of an intrusion detection method 400 according to an embodiment of this application.

[0092] In some possible implementations, method 400 can be applied to intrusion detection of LiDAR embedded in electronic devices such as vehicles or robots. Method 400 can be executed by an intrusion detection module. For example, Figure 3 The intrusion detection module shown can execute method 400. That is, method 400 can be applied to... Figure 3 Between the perception module and the planning module.

[0093] Method 400 includes the following steps.

[0094] S410, obtains the first point cloud from the LiDAR sensor.

[0095] In some possible implementations, step S410 can be performed by Figure 3 The modification unit 310 in the middle is executed.

[0096] In some possible implementations, the first point cloud belongs to the raw point cloud from the LiDAR sensor, which means that the first point cloud is the initial point cloud.

[0097] The first point cloud consists of a set of point clouds representing an object. In other words, an object can be identified by perceiving the first point cloud.

[0098] This application embodiment uses only one object as an example to illustrate the intrusion detection method. It should be understood that intrusion detection of other objects in the scene can also be implemented using the same method, which means that for each object, method 400 can be used to detect whether an attacker intends to modify the object.

[0099] S420: Obtain information about the first object by applying first perception processing to the first point cloud.

[0100] In some possible implementations, step S420 can be performed by Figure 3The detection unit 320 in the middle performs this. In this case, the first sensing process can be performed by... Figure 3 The perception module 220 in the middle is executed.

[0101] For example, the perception module 220 applies perception processing to the raw point cloud from the LiDAR sensor to obtain object information. The raw point cloud includes a first point cloud. The object information includes first object information. The object information obtained by perceiving the raw point cloud can be called initial object information. The first object information can be regarded as initial information.

[0102] For example, object information includes at least one of the following: the object's position (x, y, z), the object's volume (width, height, length), or an objectivity score. An objectivity score, also known as a confidence score, represents the probability that an object will be accurately identified by perceptual processing. Figure 3 As shown, object information can be represented as (x, y, z, w, h, l, fraction...).

[0103] The first object information can be either identified object information or uncertain object information. Identified object information refers to object information with a high confidence score. Uncertain object information refers to object information with a low confidence score.

[0104] By processing uncertain object information, that is, by using uncertain object information as the first point cloud information, it is possible to ensure that no undetected objects are missed after the attacker modifies the point cloud.

[0105] S430, modify the first point cloud to obtain the second point cloud.

[0106] In some possible implementations, step S430 can be performed by Figure 3 The modification unit 310 in the middle is executed.

[0107] To facilitate a clear description of the technical solutions in the embodiments of this application, the terms "first" and "second" are used in the embodiments of this application to distinguish the same or similar objects, which have essentially the same function and purpose. Those skilled in the art will understand that the terms "first" and "second" do not constitute a limitation on the quantity or execution order, and the terms "first" and "second" do not indicate a definite distinction.

[0108] For example, "first point cloud" and "second point cloud" are used only to distinguish the point cloud of an object before and after modification. In some possible implementations, the first point cloud can be understood as the object's initial point cloud. The object's initial point cloud belongs to the original point cloud from the LiDAR sensor. The second point cloud can be understood as the currently modified point cloud of the same object.

[0109] It should be understood that modifying the point cloud of an object can also be called modifying the object. In other words, the modified point cloud can also be called the modified object.

[0110] Optionally, step S430 includes at least one of the following: adding one or more points to the first point cloud, or removing one or more points from the first point cloud.

[0111] In other words, in step S430, the points that are to be removed or need to be created are calculated.

[0112] It should be noted that the operation of changing the first point cloud to the second point cloud can include one or more modifications.

[0113] For example, one modification includes steps S431 to S433 ( Figure 4 (Not shown in the image).

[0114] S431, select one of the following actions: add, remove, a combination of remove and add, a combination of reset and add, a combination of reset and remove, or a combination of reset, add and remove.

[0115] Resetting means clearing one or more previous modifications.

[0116] For example, when resetting means clearing the last modification, resetting can be understood as restoring the current point cloud to the point cloud before the last modification.

[0117] For example, when resetting means clearing all previous modifications, resetting can be understood as restoring the current point cloud to the first point cloud.

[0118] In a combination of reset and other actions, the reset is performed first, and then other actions are performed on the point cloud obtained after the reset.

[0119] For example, in a combination of reset, add, and remove, the reset is performed first, followed by adding and removing points from the resulting point cloud. Adding and removing can be done simultaneously or one after the other.

[0120] As previously mentioned, the operation of modifying the first point cloud into the second point cloud can include further modifications. For example, step S430 can be performed by the modification module. If the modification does not change the perception result, the modification unit knows when to reset the point cloud.

[0121] S432, locate one or more points where the action is to be performed.

[0122] In combinations that include removal and addition, one or more points to be added and one or more points to be removed can be located separately.

[0123] S433, performs a selection action on one or more points.

[0124] It should be understood that the execution order of steps S431 to S433 is not restricted. For example, they can be executed in the order of S431 to S433. Alternatively, they can be executed in the order of S432, S431, and S433. Or, S431 and S432 can be executed simultaneously.

[0125] In this way, the action and position can be determined separately, which is conducive to obtaining more appropriate modifications and thus obtaining the test results as soon as possible.

[0126] In some possible implementations, step S430 can be performed via deep reinforcement learning.

[0127] Optionally, step S431 includes: selecting an action through deep reinforcement learning.

[0128] Optionally, step S432 includes: locating one or more points that need to be modified through deep reinforcement learning.

[0129] It should also be understood that the above steps are merely examples, and the first point cloud can be modified in other ways. This application embodiment does not limit this. For example, step S430 includes: randomly modifying the first point cloud to obtain the second point cloud.

[0130] S440, by applying second perception processing to the second point cloud, obtain information about the second object.

[0131] In some possible implementations, step S440 can be performed by Figure 3 The detection unit 320 in the middle performs this. In this case, the second sensing process can be performed by... Figure 3 The perception module 220 in the middle executes, or Figure 3 The sensing unit 330 in the middle is executed.

[0132] For example, when the second sensory processing is by Figure 3 When the perception module 220 is executed, the intrusion detection module 300 can access the perception progress in the perception module 220.

[0133] Optionally, the first and second perception processes use the same perception algorithm.

[0134] In this way, no additional sensing algorithms are needed, reducing the demand for storage space; that is, no additional storage space needs to be provided for other sensing algorithms.

[0135] S450, based on the comparison between the first object information and the second object information, provides the intrusion result.

[0136] In some possible implementations, step S450 can be performed by Figure 3 The detection unit 320 in the middle is executed.

[0137] Optionally, step S450 includes: classifying the first object information as an intrusion when the comparison between the first object information and the second object information is equal to or greater than a first threshold.

[0138] The first object information is classified as an intrusion. In other words, the object corresponding to the first object information is an intrusion, meaning that the point cloud of the first object was modified by an attacker.

[0139] In other words, if the modified object information changes significantly compared to the object information before modification, the object can be considered an intrusion.

[0140] Optionally, step S450 includes: classifying the first object information as benign when the comparison between the first object information and the second object information is less than a first threshold.

[0141] In other words, if the information of two objects before and after modification is similar, then the object can be considered benign.

[0142] Adversarial examples are unstable. Therefore, even a slight modification to a well-designed adversarial example will result in a significant change to the object's outcome (i.e., object information). In contrast, benign examples cause only minor changes. An adversarial example is a point cloud of an object modified by an attacker. A benign example is a point cloud, a genuine point cloud.

[0143] Optionally, step S450 includes: when the comparison between the first object information and the second object information is less than the first threshold, modifying the second point cloud, using the modified second point cloud as the second point cloud in steps S440 to S450, and executing steps S440 to S450.

[0144] In other words, if the information of the two objects before and after the modification is similar, the point cloud being modified can be further modified.

[0145] Optionally, step S450 includes: when the comparison between the first object information and the second object information is less than a first threshold, selecting one of the following actions: classifying the first object information as benign, or modifying the second point cloud, using the modified second point cloud as the second point cloud in steps S440 to S450, and executing steps S440 to S450.

[0146] In other words, if the information of the two objects before and after modification is similar, step S450 can determine whether further progress is needed.

[0147] In other words, if no intrusion is detected, that is, if it is determined that the object is not an intrusion, step S450 can determine whether further progress is needed on the object's current modified point cloud.

[0148] For example, if no intrusion is detected, the inference unit 340 can be notified, and the inference unit 340 can determine whether further progress is needed, which means whether the currently modified point cloud needs further modification.

[0149] For example, step S450 includes: when the comparison between the first object information and the second object information is less than the first threshold, selecting an action based on the number of modifications experienced by the currently modified point cloud.

[0150] Step S450 can determine that further progress is needed when the number of modifications the currently modified point cloud has undergone is relatively small, and determine that no further progress is needed when the number of modifications the currently modified point cloud has undergone is relatively large.

[0151] For example, step S450 includes: when the comparison between the first object information and the second object information is less than the first threshold, selecting an action based on the degree to which the current modified point cloud is modified.

[0152] Step S450 can determine that further progress is needed when the difference between the current modified point cloud and the initial point cloud is small, and determine that no further progress is needed when the difference between the current modified point cloud and the initial point cloud is large.

[0153] Optionally, step S450 includes: classifying the first object information as an intrusion when the comparison between the first object information and the second object information is equal to or greater than a first threshold; classifying the first object information as benign when the comparison is equal to or less than a second threshold; and modifying the second point cloud when the comparison is greater than the second threshold and less than the first threshold, using the modified second point cloud as the second point cloud in steps S440 to S450, and executing steps S440 to S450.

[0154] It should be understood that the position of the equal sign above is merely an example. For instance, step S450 may further include: classifying the first object information as an intrusion when the comparison between the first object information and the second object information is greater than a first threshold.

[0155] In the specific implementation process, adjustments can be made as needed, and the embodiments of this application do not limit this.

[0156] Optionally, method 400 further includes step S460.

[0157] S460, when the first object information is classified as an intrusion, discard the first object information.

[0158] In some possible implementations, step S460 can be performed by Figure 3 The detection unit 320 in the middle is executed.

[0159] For example, if the object corresponding to the first piece of information is an object added by the attacker, the first object information is discarded.

[0160] The objects added by the attacker refer to objects that do not actually exist in the environment but are identified by the perception module due to the attacker's modification of the point cloud.

[0161] Optionally, method 400 further includes step S470.

[0162] S470, when the first object information is classified as an intrusion, issue a warning for the intrusion.

[0163] For example, the detection module 320 can notify the planning module of the intrusion.

[0164] For example, when the object corresponding to the first piece of information is an object removed by an attacker, corrected object information is provided.

[0165] The objects removed by the attacker refer to objects that actually exist in the environment but cannot be recognized by the perception module due to the attacker's modification of the point cloud.

[0166] The corrected object information can be obtained from the second object information.

[0167] Optionally, method 400 further includes step S480.

[0168] S480, when the first object information is classified as benign, the first object information is sent to the planning module.

[0169] In some possible implementations, step S480 can be performed by Figure 3 The detection unit 320 in the middle is executed. Or step S480 can be performed by... Figure 3 The inference unit 340 in the middle is executed.

[0170] The intrusion detection method provided in this application embodiment can be regarded as a point cloud-level intrusion detection.

[0171] In the technical solution provided in this application embodiment, adversarial attacks can be identified by comparing initial object information with modified object information. In other words, point cloud anomalies caused by attackers can be identified. Therefore, the method in this application embodiment can improve the reliability of object detection, which is beneficial for providing more accurate perception results for subsequent control strategies. For example, when method 400 is applied to an autonomous driving system, it can improve driving safety and user experience.

[0172] For example, when an object that should be identified is hidden by an attacker, the technical solution provided in this application can identify the hidden object and warn of an intrusion to avoid accidents, including car crashes. For example, when an attacker creates an object that should not be identified, the technical solution provided in this application can identify the created object and discard its information to avoid freezing the vehicle or triggering emergency braking.

[0173] Furthermore, the technical solution provided in this application can detect intrusions on a per-object point cloud basis, rather than on a per-point basis. In other words, this technical solution can act on each object region individually, rather than detecting the entire point cloud. This technical solution targets object regions, and therefore is faster than analyzing all points in the scene.

[0174] Figure 5 A flowchart of an intrusion detection process applied to an autonomous driving system, provided in an embodiment of this application, is shown. The intrusion detection process corresponding to the above method embodiment is described below. It should be understood that the description of the intrusion detection process corresponds to the description of the method embodiment. Therefore, for any content not described in detail, please refer to the above method 400. For the sake of brevity, further details are omitted here.

[0175] The intrusion detection process can be performed by the intrusion module 300. The intrusion module 300 is used between the perception module 220 and the planning module 230. Figure 5 The entire process is described. The intrusion detection method includes the following steps.

[0176] S510 receives the raw point cloud from LiDAR and the perception results from the perception module 230.

[0177] In other words, the sensing module 220 does not send the sensing results to, for example... Figure 1 Instead of the planning module 230 shown, it sends the perception results to the intrusion detection module 300. The perception results based on the raw point cloud from LiDAR can also be called the initial perception results.

[0178] The perception module 220 applies perception processing to the raw point cloud from LiDAR and returns the perception results.

[0179] A primitive point cloud consists of one or more sets of point clouds. A set of point clouds refers to the point cloud of a single object. The first point cloud belongs to the primitive point cloud.

[0180] Accordingly, the perception result includes information about one or more objects. For example, the perception result consists of identified object information and uncertain object information. The object information in the original perception result can also be called the initial object information. The first object information belongs to the original perception result.

[0181] like Figure 5As shown, information about an object can be represented as (x, y, z, w, h, l, fraction...).

[0182] S520, Modify the point cloud (an example of the first point cloud) to obtain the modified point cloud (an example of the second point cloud).

[0183] like Figure 5 As shown, the modification unit 310 modifies a set of point clouds to obtain a modified set of point clouds.

[0184] For example, the renderer generates a new set of point clouds and passes it to the perception module. This new set of point clouds can be understood as a modified point cloud. The second point cloud belongs to the modified point cloud.

[0185] Step S520 includes steps S521 to S523.

[0186] S521, Modify the unit to select one of the following actions:

[0187] 1) Add one or more points.

[0188] 2) Remove one or more points.

[0189] 3) Reset previous modifications and additions / removals.

[0190] Action option 3) can also be understood as resetting the modification, and then selecting an action from action option 1) and action option 2).

[0191] In other words, if the modification does not change the perception result, the modification unit knows when to reset the point cloud.

[0192] S522, Modify the 3D position of the unit positioning target.

[0193] The target 3D position refers to the location of one or more points that will be modified.

[0194] For example, steps S521 and S522 can be achieved through deep reinforcement.

[0195] S523, the modified unit applies the action to the target's 3D position.

[0196] S530, perceive the modified point cloud to obtain updated object information (example of second object information).

[0197] It should be understood that Figure 5 The following explanation uses point cloud recognition performed by the sensing unit 330 in the intrusion detection module 300 as an example. In practical applications, the intrusion detection module 300 can be connected to the sensing module 220 to sense the modified point cloud.

[0198] For example, the perception module 220 re-perceives the slightly modified point cloud, which means that the perception processing applied to the modified point cloud and the perception processing applied to the original point cloud use the same perception algorithm.

[0199] S540 provides intrusion detection results based on the updated object information and the initial object information.

[0200] For example, if the object information is classified as an intrusion, that is, an intrusion is detected, the intrusion detection module 300 may perform at least one of the following: issue an intrusion warning or discard the object information.

[0201] Discarding object information can mean not sending information about objects added by the attacker to the planning module.

[0202] An intrusion warning can refer to notifying the planning module of an object that has been removed by an attacker. For example, it can provide correction information for objects removed by the attacker.

[0203] Alternatively, an intrusion warning could refer to notifying the planning module that an intrusion has occurred.

[0204] If the object information is not classified as an intrusion, that is, no intrusion is detected, the inference unit 340 determines whether further progress is needed.

[0205] If the inference unit 340 classifies the object information as benign, it sends the initial object information to the planning module. If the inference unit 340 determines that further progress is needed, that is, the object is undeterminable, the inference unit 340 may notify the modification unit 310 to apply a further modification process, and then repeat steps S520 to S540.

[0206] For each object identified by the perception module, the detection module can determine whether an attacker has intentionally modified the object. For the sake of simplicity, only one object is used as an example in steps S520 to S540.

[0207] In the technical solution provided in this application embodiment, adversarial attacks can be identified by comparing initial object information with modified object information. In other words, point cloud anomalies caused by attackers can be identified. Therefore, the method in this application embodiment can improve the reliability of object detection, which is beneficial for providing more accurate perception results for subsequent control strategies. For example, when method 400 is applied to an autonomous driving system, it can improve driving safety and user experience.

[0208] Furthermore, the technical solution provided in this application can detect intrusions on a per-object point cloud basis, rather than on a per-point basis. In other words, this technical solution can act on each object region individually, rather than detecting the entire point cloud. This technical solution targets object regions, and therefore is faster than analyzing all points in the scene.

[0209] It should be understood that the technical solutions of the embodiments of this application can be applied to various electronic devices, such as robots, drones, virtual / augmented reality devices, etc., but the embodiments of this application are not limited thereto.

[0210] The method embodiments of this application have been described above. The apparatus embodiments corresponding to the above method embodiments are described below. It should be understood that the descriptions of the apparatus embodiments correspond to the descriptions of the method embodiments. Therefore, for any content not detailed above, please refer to the above method embodiments. For the sake of brevity, further details are omitted here.

[0211] like Figure 6 As shown in the figure, this application provides an intrusion detection device 600.

[0212] The intrusion detection device 600 includes the following units.

[0213] Acquisition unit 610 is configured to acquire a first point cloud from a LiDAR sensor and acquire first object information by applying a first perception processing to the first point cloud.

[0214] Processing unit 620 is used to modify the first point cloud to obtain a second point cloud.

[0215] The acquisition unit 610 is also used to: obtain second object information by applying second perception processing to the second point cloud.

[0216] The processing unit 620 is also used to provide intrusion results based on the comparison between the first object information and the second object information.

[0217] Optionally, in some embodiments, the processing unit 620 is specifically used to add one or more points to the first point cloud, or to remove one or more points from the first point cloud.

[0218] Optionally, in some embodiments, the processing unit 620 is also configured to locate the one or more points through deep reinforcement learning.

[0219] Optionally, in some embodiments, the processing unit 620 is specifically configured to classify the first object information as an intrusion when the comparison between the first object information and the second object information is equal to or greater than a first threshold.

[0220] Optionally, in some embodiments, the processing unit 620 is further configured to perform at least one of the following: discarding the first object information, or sending the corrected object information to the planning module, wherein the corrected object information is obtained based on the second object information.

[0221] Optionally, in some embodiments, the processing unit 620 is specifically configured to classify the first object information as benign when the comparison between the first object information and the second object information is less than the first threshold.

[0222] Optionally, in some embodiments, the processing unit 620 is further configured to: send the first object information to the planning module.

[0223] like Figure 7 As shown in the figure, this application embodiment also provides an intrusion detection device 700.

[0224] The device 700 includes at least one processor 720 and a communication interface 730.

[0225] Optionally, the device 700 may also include at least one of a memory 710 and a bus 740. Any two or all three of the memory 710, processor 720 and communication interface 730 may be interconnected via the bus 740.

[0226] Optionally, the memory 710 may be a read-only memory (ROM), a static storage device, a dynamic storage device, or a random access memory (RAM). The memory 710 may store programs. When the processor 720 executes the program stored in the memory 710, the processor 720 and the communication interface 730 are used to execute various steps of the intrusion detection method of this embodiment. In other words, the processor 720 can obtain stored instructions from the memory 710 through the communication interface 730 to execute various steps of the intrusion detection method of this embodiment.

[0227] Optionally, the processor 720 may employ a general-purpose CPU, microprocessor, ASIC, graphics processing unit (GPU), or one or more integrated circuits to execute relevant programs, thereby realizing the functions that each unit in the device needs to perform, or executing each step of the intrusion detection method of the embodiments of this application.

[0228] Alternatively, the processor 720 can also be an integrated circuit chip with signal processing capabilities. In implementation, each step of the intrusion detection method of this application embodiment can be completed by hardware integrated logic circuits in the processor or by software instructions.

[0229] Optionally, the processor 720 can also be a general-purpose processor, a digital signal processing (DSP), an application-specific integrated circuit (ASIC), an off-the-shelf field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. The methods, steps, and logic block diagrams disclosed in the embodiments of this application can be implemented or executed. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the methods disclosed in the embodiments of this application can be directly embodied as being executed and completed by a hardware decoding processor, or by a combination of hardware and software modules in the decoding processor. The software modules can reside in storage media that are well-established in the field, such as random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, etc. The storage medium is located in memory; the processor reads information from the memory and, in conjunction with its hardware, completes the functions required by the units included in the intrusion detection device of the embodiments of this application, or executes each intrusion detection method in the intrusion detection method of the embodiments of this application.

[0230] Optionally, the communication interface 730 can use a transceiver device (e.g., but not limited to a transceiver) to enable communication between the device and other devices or communication networks. For example, the communication interface 730 can also be an interface circuit.

[0231] Bus 740 may include paths for transferring information between various components of the device, such as memory, processor, and communication interfaces.

[0232] This application also provides a computer program product including instructions. When these instructions are executed on a computer, they cause the computer to perform the methods described in the above-described method embodiments.

[0233] This application also provides a computer-readable storage medium storing a computer program. When the computer program is executed by a processor, it implements the methods described in the above-described method embodiments.

[0234] This application also provides a chip. The chip includes at least one processor and an interface circuit. The at least one processor obtains instructions stored in a memory through the interface circuit and executes the methods described in the above-described method embodiments.

[0235] Optionally, the chip may further include a memory for storing instructions, and the processor is used to execute the instructions stored in the memory. When the instructions are executed, the processor is used to perform the method described in the above method embodiments.

[0236] This application also provides a terminal. The terminal device includes the intrusion detection device in the above-described device embodiments, for example... Figure 6 or Figure 7 The apparatus shown.

[0237] For example, the terminal could be a vehicle, a robot, or a drone.

[0238] The aforementioned intrusion detection device can be installed on the target vehicle or independently of the target vehicle.

[0239] In one alternative implementation, the terminal includes a LiDAR sensor.

[0240] For explanations and benefits regarding any of the intrusion detection devices provided above, please refer to the corresponding method embodiments provided above, which will not be repeated here.

[0241] It should also be understood that the terms "first," "second," "third," "fourth," and various numbers used in this specification are merely for ease of description and are not intended to limit the scope of the embodiments of this application.

[0242] It should be understood that the processor in the embodiments of this application can be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor can be a microprocessor or any conventional processor.

[0243] It is understood that the memory mentioned in the embodiments of this application can be volatile memory or non-volatile memory, or may include both volatile and non-volatile memory. Non-volatile memory can be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. Volatile memory can be random access memory (RAM) used as an external cache. By way of example, but not limitation, many forms of RAM are available, such as Static Random Access Memory (SRAM), Dynamic Random Access Memory (DRAM), Synchronous DRAM (SDRAM), Double Data Rate Synchronous DRAM (DDR SDRAM), Enhanced Synchronous DRAM (ESDRAM), Synchlink DRAM (SLDRAM), and Direct Rambus RAM (DR RAM).

[0244] It should be noted that when the processor is a general-purpose processor, DSP, ASIC, FPGA or other programmable logic device, discrete gate or transistor logic device, or discrete hardware component, the memory (storage module) is integrated into the processor.

[0245] It should be noted that the memory described in this specification is intended to include, but is not limited to, these memories and any other suitable type of memory.

[0246] Those skilled in the art will recognize that, based on the examples described in conjunction with the embodiments disclosed in this specification, the units and algorithm steps can be implemented using electronic hardware or a combination of computer software and electronic hardware. Whether these functions are implemented using hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this application.

[0247] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the above systems, devices and units can be found in the corresponding processes in the above method embodiments, and will not be repeated here.

[0248] In the several embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the described apparatus embodiments are merely examples. For example, the unit division is merely a logical functional division, and in actual implementation, it can be other divisions. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not performed. Furthermore, the mutual coupling or direct coupling or communication connection shown or discussed can be implemented by using some interface. Indirect coupling or communication connection between apparatuses or units can be implemented in electrical, mechanical, or other forms.

[0249] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; specifically, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected according to actual needs to achieve the purpose of this embodiment.

[0250] In addition, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0251] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This software product is stored in a storage medium and includes several instructions to instruct a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0252] The above description is merely a specific implementation of this application and is not intended to limit the scope of protection of this application. Any variations or substitutions that are readily conceived by those skilled in the art within the scope of the technology disclosed in this application should fall within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. An intrusion detection method, characterized in that, include: A first point cloud is obtained from a LiDAR sensor, the first point cloud including the point cloud of objects; By applying first perception processing to the first point cloud, first object information about the object is obtained; Modify the first point cloud to obtain a second point cloud of the object; By applying a second perception processing to the second point cloud, second object information about the object is obtained; as well as The intrusion result is provided based on the comparison result between the first object information and the second object information and a first threshold. The first object information and the second object information each include at least one of the following: the position, volume or confidence score of the object, and the comparison includes comparing the corresponding items of the first object information and the second object information.

2. The intrusion detection method according to claim 1, characterized in that, Modifying the first point cloud to obtain the second point cloud of the object includes at least one of the following: Add one or more points to the first point cloud, or Remove one or more points from the first point cloud.

3. The intrusion detection method according to claim 2, characterized in that, It also includes locating one or more points through deep reinforcement learning.

4. The intrusion detection method according to any one of claims 1 to 3, characterized in that, The intrusion result is provided by comparing the comparison result between the first object information and the second object information with a first threshold, including: When the comparison between the first object information and the second object information is equal to or greater than the first threshold, the first object information is classified as an intrusion.

5. The intrusion detection method according to claim 4, characterized in that, It also includes at least one of the following: The first object information is discarded, or the corrected object information is sent to the planning module, wherein the corrected object information is obtained based on the second object information.

6. The intrusion detection method according to any one of claims 1 to 3, characterized in that, The intrusion result is provided by comparing the comparison result of the first object information and the second object information with a first threshold, including: When the comparison between the first object information and the second object information is less than the first threshold, the first object information is classified as benign.

7. The intrusion detection method according to claim 6, characterized in that, Also includes: The information of the first object is sent to the planning module.

8. An intrusion detection device, characterized in that, include: Acquisition unit, used for: A first point cloud is obtained from a LiDAR sensor, the first point cloud including the point cloud of objects; and By applying first perception processing to the first point cloud, first object information about the object is obtained; as well as A processing unit is configured to modify the first point cloud to obtain a second point cloud of the object; The acquisition unit is further configured to: obtain second object information about the object by applying a second perception processing to the second point cloud; and The processing unit is further configured to compare the comparison result between the first object information and the second object information with a first threshold to provide an intrusion result; The first object information and the second object information each include at least one of the following: the position, volume or confidence score of the object, and the comparison includes comparing the corresponding items of the first object information and the second object information.

9. The intrusion detection device according to claim 8, characterized in that, The processing unit is specifically used for: Add one or more points to the first point cloud, or Remove one or more points from the first point cloud.

10. The intrusion detection device according to claim 9, characterized in that, The processing unit is also used to locate the one or more points through deep reinforcement learning.

11. The intrusion detection device according to any one of claims 8 to 10, characterized in that, The processing unit is specifically used for: When the comparison between the first object information and the second object information is equal to or greater than the first threshold, the first object information is classified as an intrusion.

12. The intrusion detection device according to claim 11, characterized in that, The processing unit is further configured to perform at least one of the following: The first object information is discarded, or the corrected object information is sent to the planning module, wherein the corrected object information is obtained based on the second object information.

13. The intrusion detection device according to any one of claims 8 to 10, characterized in that, The processing unit is specifically used for: When the comparison between the first object information and the second object information is less than the first threshold, the first object information is classified as benign.

14. The intrusion detection device according to claim 13, characterized in that, The processing unit is further configured to: The information of the first object is sent to the planning module.

15. An intrusion detection device, characterized in that, include: Memory, used to store computer instructions; and A processor for executing computer instructions stored in the memory, wherein when the computer instructions are executed, the processor performs the intrusion detection method according to any one of claims 1 to 7.

16. A computer-readable storage medium, characterized in that, Used to store computer programs, wherein the computer programs are used to execute instructions for the intrusion detection method according to any one of claims 1 to 7.

17. A computer program product, characterized in that, It includes one or more instructions that, when executed on a computer, cause the computer to perform the method according to any one of claims 1 to 7.

18. A terminal, characterized in that, include: The intrusion detection device according to any one of claims 8 to 15.

19. The terminal according to claim 18, characterized in that, It also includes LiDAR sensors.

Images