A Simulation Test Method and System for Lane Recognition in Autonomous Driving Based on Physics Attacks

Abstract

This invention relates to the field of lane line recognition technology, and provides a simulation testing method and system for autonomous driving lane line recognition based on physical attacks. The method includes: establishing several operating scenarios in an autonomous driving simulator, adding natural noise to these scenarios to obtain several simulated images, forming a simulation dataset; for each simulated image in the simulation dataset, generating a physical attack map using an attack algorithm, importing the physical attack map into the autonomous driving simulator for resampling, and obtaining a physical attack dataset; acquiring a lane line detection model to be tested, performing lane line recognition on the simulation dataset and the physical attack dataset respectively, comparing the recognition results, and obtaining the robustness and anti-attack performance of the lane line detection model. This method can effectively simulate real-world scenarios with a large number of dynamic traffic elements, providing a more accurate safety assessment for autonomous driving lane line detection algorithms.

Description

Technical Field

[0001] This invention belongs to the field of lane line recognition technology, and particularly relates to a simulation test method and system for autonomous driving lane line recognition based on physical attacks. Background Technology

[0002] The statements in this section are merely background information related to the present invention and do not necessarily constitute prior art.

[0003] Currently, lane detection models are mostly evaluated directly using metrics such as accuracy, recall, or F1 score. However, this method cannot assess the robustness of lane detection models against physical attacks such as obstacle occlusion.

[0004] To address these issues, some scholars have proposed testing methods for lane detection algorithms based on digital attacks. However, these methods typically focus on simulating attacks on the lane lines themselves, rarely taking into account the complex interactions of various traffic participants (such as different types of motor vehicles, non-motorized vehicles, and pedestrians in different driving states). In real-world scenarios, the presence and behavior of surrounding traffic participants affect the sensor's perception of lane lines and the algorithm's detection of them. In real-world scenarios with numerous dynamic traffic elements, the lane detection algorithm's ability to withstand digital attacks may differ significantly from its performance in simplified, relatively static test scenarios. Summary of the Invention

[0005] To address the technical problems mentioned above, this invention provides a simulation testing method and system for autonomous driving lane line recognition based on physical attacks. A simulation dataset containing operating conditions and natural noise is established based on an autonomous driving simulator. A physical attack is then performed to obtain a texture file after the physical attack. This texture file is imported into the autonomous driving simulator and resampled to obtain a physical attack dataset. This effectively simulates real-world scenarios with interference from numerous dynamic traffic elements. The lane line detection model under test performs lane line recognition on the datasets before and after the physical attack, thereby determining the performance of the lane line detection algorithm and providing a more accurate safety evaluation for autonomous driving lane line detection algorithms.

[0006] To achieve the above objectives, the present invention adopts the following technical solution:

[0007] The first aspect of the present invention provides a simulation testing method for lane line recognition in autonomous driving based on physical attacks, comprising:

[0008] Several operating scenarios were created in the autonomous driving simulator, and natural noise was added to the operating scenarios to obtain several simulation images, which constituted a simulation dataset.

[0009] For each simulation image in the simulation dataset, an attack algorithm is used to generate a physical attack map. The physical attack map is then imported into the autonomous driving simulator for resampling to obtain the physical attack dataset.

[0010] Obtain the lane detection model to be tested, perform lane recognition on the simulation dataset and the physical attack dataset respectively, compare the recognition results, and obtain the robustness and anti-attack performance of the lane detection model.

[0011] Furthermore, the evaluation steps for the anti-attack performance include:

[0012] Calculate the recognition accuracy (Precision1) of the lane line detection model on the simulation dataset;

[0013] Calculate the recognition accuracy (Precision2) of the lane line detection model on the physical attack dataset;

[0014] Calculate the anti-attack performance R = (Precision1 - Precision2) / (∑Hj), where Hj represents the entropy value of the j-th physical attack texture. Pij = nij / Nj, where nij represents the number of pixels with grayscale value i in the j-th physical attack texture, and Nj represents the total number of pixels in the j-th physical attack texture.

[0015] Furthermore, the robustness is the difference between the F1 score of the lane detection model under test for the simulation dataset and the F1 score of the lane detection model under test for the physical attack dataset.

[0016] Furthermore, the operating scenarios include single dashed line, double solid line - turning, double solid line - overtaking, double dashed line, solid and dashed double line, elevated road, highway, single solid line - uphill, downhill and / or tunnel.

[0017] Furthermore, the natural noise includes rainy days, foggy days, snowy days, backlighting, and / or midnight.

[0018] Furthermore, the step of generating a physical attack map using an attack algorithm includes:

[0019] (1) For each simulation image in the simulation dataset, initialize the texture map;

[0020] (2) After overlaying the texture map with the simulation image, input it into the lane detection model to obtain the loss value;

[0021] (3) Backpropagate the loss value, calculate the gradient, and update the texture map;

[0022] (4) Repeat steps (2)-(3) several times to optimize the texture and obtain the physical attack texture.

[0023] A second aspect of the present invention provides a simulation test system for autonomous driving lane line recognition based on physical attacks, comprising:

[0024] The simulation image creation module is configured to: create several working scenarios in the autonomous driving simulator, add natural noise to the working scenarios, obtain several simulation images, and form a simulation dataset.

[0025] The physical attack module is configured to: generate a physical attack map using an attack algorithm for each simulation image in the simulation dataset, import the physical attack map into the autonomous driving simulator for resampling, and obtain the physical attack dataset;

[0026] The simulation testing module is configured to: acquire the lane detection model to be tested, perform lane recognition on the simulation dataset and the physical attack dataset respectively, compare the recognition results, and obtain the robustness and anti-attack performance of the lane detection model.

[0027] Furthermore, the evaluation steps for the anti-attack performance include:

[0028] Calculate the recognition accuracy (Precision1) of the lane line detection model on the simulation dataset;

[0029] Calculate the recognition accuracy (Precision2) of the lane line detection model on the physical attack dataset;

[0030] Calculate the anti-attack performance R = (Precision1 - Precision2) / (∑Hj), where Hj represents the entropy value of the j-th physical attack texture. Pij = nij / Nj, where nij represents the number of pixels with grayscale value i in the j-th physical attack texture, and Nj represents the total number of pixels in the j-th physical attack texture.

[0031] A third aspect of the present invention provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps in the above-described simulation test method for lane line recognition of autonomous driving based on physical attacks.

[0032] A fourth aspect of the present invention provides a computer device including a computer-readable storage medium, a processor, and a computer program stored on the computer-readable storage medium and executable on the processor, wherein the processor executes the program to implement the steps in the above-described simulation test method for autonomous driving lane line recognition based on physical attacks.

[0033] Compared with the prior art, the beneficial effects of the present invention are:

[0034] This invention establishes a simulation dataset containing operating conditions and natural noise based on an autonomous driving simulator, and performs a physical attack to obtain the texture file after the physical attack. This texture file is then imported into the autonomous driving simulator for resampling to obtain the physical attack dataset. This dataset can effectively simulate real-world scenarios with interference from a large number of dynamic traffic elements. The lane detection model under test performs lane line recognition on the datasets before and after the physical attack, thereby judging the performance of the lane detection algorithm and providing a more accurate safety evaluation for autonomous driving lane detection algorithms.

[0035] This invention assesses the robustness of lane detection algorithms based on the magnitude of F1-score decline and evaluates the anti-attack performance of lane detection models based on the entropy value of physical attack maps, providing a comprehensive evaluation of lane detection algorithms for autonomous driving. Attached Figure Description

[0036] The accompanying drawings, which form part of this invention, are used to provide a further understanding of the invention. The illustrative embodiments of the invention and their descriptions are used to explain the invention and do not constitute an improper limitation of the invention.

[0037] Figure 1 This is a flowchart of the simulation test method for autonomous driving lane line recognition based on physical attacks according to Embodiment 1 of the present invention;

[0038] Figure 2 This is a schematic diagram of the reference dataset for autonomous driving lane line recognition in Embodiment 1 of the present invention;

[0039] Figure 3 This is a schematic diagram of the physical attack texture mapping in Embodiment 1 of the present invention;

[0040] Figure 4 This is a schematic diagram illustrating the effect of the physical attack dataset in Embodiment 1 of the present invention;

[0041] Figure 5 This is a schematic diagram of the structure of a computer device according to Embodiment 4 of the present invention. Detailed Implementation

[0042] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings.

[0043] Example 1

[0044] This embodiment provides a simulation test method for lane line recognition in autonomous driving based on physical attacks.

[0045] This embodiment provides a simulation test method for autonomous driving lane line recognition based on physical attacks. First, a simulation dataset is established based on the autonomous driving simulator Carla, including 10 working conditions such as single dashed line, double solid line - turning, double solid line - overtaking, double dashed line, solid and dashed double line, elevated road, highway, single solid line - uphill, downhill, and tunnel, as well as 6 weather conditions such as sunny day, rainy day, foggy day, snowy day, backlight, and midnight. Second, the audi.obj model file in Carla is physically attacked based on the C&W (Carlini & Wagner) attack algorithm (also known as the CW attack algorithm). The attack was performed to obtain the texture file after the physical attack, which was then imported into Carla for resampling to obtain the simulation dataset after the physical attack. Finally, the dataset was manually annotated using the Sprite Annotation Assistant software to obtain a JSON file containing lane line coordinate information. Based on the lane line detection model, lane line recognition was performed on the simulation dataset before and after the physical attack. The robustness of the lane line detection algorithm was judged based on the decrease in F1-score, and the anti-attack performance of the lane line detection model was judged based on the entropy value of the physical attack texture, providing a more accurate safety evaluation for the lane line detection algorithm of autonomous driving.

[0046] The simulation test method for autonomous driving lane line recognition based on physical attacks provided in this embodiment is as follows: Figure 1 As shown, the specific steps include:

[0047] S1: Dataset Creation: Install the Carla autonomous driving simulator on the Ubuntu 20.04 system. Create 10 driving scenarios in Carla, including single dashed line, double solid line - turning, double solid line - overtaking, double dashed line, solid and dashed double line, elevated road, highway, single solid line - uphill, downhill, and tunnel. Represent these as basic scenarios, such as... Figure 2 As shown, based on the basic scene, rainy day, foggy day, snowy day, backlight and / or midnight natural noise are added respectively to obtain several simulation images, which constitute the lane line detection simulation dataset.

[0048] S2: Manual annotation: Use the Sprite Annotation Assistant software to manually annotate the lane lines involved in the simulation dataset in S1, and obtain a JSON file containing lane line coordinate information.

[0049] S3: Benchmark Dataset Test: Based on the lane line detection model, lane line detection and recognition are performed on the simulation dataset in S1 to complete the benchmark dataset test. The recognition accuracy is obtained based on the evaluation metric F1-score.

[0050] In this embodiment, the lane detection model can use GANet.

[0051] S4: A physics attack is performed on the vehicle model audi.obj in Carla based on the C&W algorithm. The implementation principle is as follows:

[0052] (1) For each simulation image in the simulation dataset, initialize a texture map. Generally, it can start with a random noisy image, such as randomly generating an image matrix with a certain resolution and pixel values ​​within a certain range as the initial state, and its size should match the simulation image.

[0053] (2) After overlaying the texture map with the simulation image, input it into the lane detection model GANet to obtain the loss value.

[0054] In this step, the loss value is calculated using the initial regression loss from GANet's loss function:

[0055]

[0056] (3) Backpropagate the loss value in step (2) and update the texture map according to the gradient.

[0057] (4) Repeat steps (2)-(3) to continuously optimize the texture. After training, save the result to obtain the texture map after the physical attack, such as... Figure 3 As shown.

[0058] (4) Import the physical attack texture obtained in step (5) into the autonomous driving simulator CALA for resampling to obtain the lane line recognition physical attack dataset. The effect diagram is shown below. Figure 4 As shown.

[0059] S5: Physical Attack Dataset Test: Based on the lane line detection model, lane line detection and recognition tests were conducted on the physical attack dataset in S4, and the recognition accuracy was obtained based on the evaluation metric F1-score.

[0060] S6: Accuracy Comparison: Based on steps S3 and S5, compare the recognition accuracy of the lane detection model on the simulation dataset before and after the addition of physical attacks, judge the robustness of the lane detection model based on the decrease in F1-score, and evaluate the anti-attack performance of the lane detection model.

[0061] The evaluation steps for the anti-attack performance of the lane detection model include:

[0062] (1) Calculate the recognition accuracy of the lane line detection model on the simulation dataset before the attack, which is the proportion of correctly predicted positive to all correctly predicted positive. Precision = TP / (TP+FP).

[0063] (2) Calculate the recognition accuracy of the lane line detection model on the simulated dataset after the attack;

[0064] (3) Calculate the anti-attack performance: R = (Precision1 - Precision2) / (∑Hj), where Hj represents the entropy value of the j-th physical attack texture. Pij = nij / Nj, where nij represents the number of pixels with grayscale value i in the j-th physics attack texture, and Nj represents the total number of pixels in the j-th physics attack texture.

[0065] The results of the F1-score decline are shown in Table 1. As can be seen from Table 1, the recognition accuracy of the GANet model on the unattacked dataset is 72.91%, while the recognition accuracy on the dataset after physical attacks is added is 55.00%, a decrease of 17.91%. This provides a safety evaluation for the autonomous driving lane line recognition algorithm and provides an important basis for the robustness testing of lane line detection algorithms.

[0066] Table 1. Comparison of recognition accuracy of GANet network on different datasets

[0067]

[0068] Example 2

[0069] This embodiment provides a simulation test system for autonomous driving lane line recognition based on physical attacks, which specifically includes:

[0070] The simulation image creation module is configured to: create several working scenarios in the autonomous driving simulator, add natural noise to the working scenarios, obtain several simulation images, and form a simulation dataset.

[0071] The physical attack module is configured to: generate a physical attack map using an attack algorithm for each simulation image in the simulation dataset, import the physical attack map into the autonomous driving simulator for resampling, and obtain the physical attack dataset;

[0072] The simulation testing module is configured to: acquire the lane detection model to be tested, perform lane recognition on the simulation dataset and the physical attack dataset respectively, compare the recognition results, and obtain the robustness and anti-attack performance of the lane detection model.

[0073] Robustness is the difference between the F1 score of the lane detection model under test for the simulation dataset and the F1 score of the lane detection model under test for the physical attack dataset.

[0074] The evaluation steps for resistance to attacks include:

[0075] Calculate the recognition accuracy (Precision1) of the lane line detection model on the simulation dataset;

[0076] Calculate the recognition accuracy (Precision2) of the lane line detection model on the physical attack dataset;

[0077] Calculate the anti-attack performance R = (Precision1 - Precision2) / (∑Hj), where Hj represents the entropy value of the j-th physical attack texture. Pij = nij / Nj, where nij represents the number of pixels with grayscale value i in the j-th physical attack texture, and Nj represents the total number of pixels in the j-th physical attack texture.

[0078] It should be noted that each module in this embodiment corresponds one-to-one with each step in Embodiment 1, and their specific implementation processes are the same, so they will not be repeated here.

[0079] Example 3

[0080] This embodiment provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps in the simulation test method for autonomous driving lane line recognition based on physical attacks as described in Embodiment 1 above.

[0081] Example 4

[0082] This embodiment provides a computer device, such as... Figure 5 As shown, the system includes a display device, an input device, a computer-readable storage medium (volatile memory and non-volatile storage medium), a processor, a communication interface (i.e., a network interface), and a computer program stored on the computer-readable storage medium and executable on the processor. The processor, communication interface, and computer-readable storage medium can be connected via a bus or other means. The communication interface is used to receive and send data, and when the processor executes the program, it implements the steps in the simulation test method for autonomous driving lane line recognition based on physical attacks as described in Embodiment 1 above.

[0083] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0084] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0085] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A simulation test method for lane line recognition in autonomous driving based on physical attacks, characterized in that, include: Several operating scenarios were created in the autonomous driving simulator, and natural noise was added to the operating scenarios to obtain several simulation images, which constituted a simulation dataset. For each simulation image in the simulation dataset, an attack algorithm is used to generate a physical attack map. The physical attack map is then imported into the autonomous driving simulator for resampling to obtain the physical attack dataset. Obtain the lane detection model to be tested, perform lane recognition on the simulation dataset and the physical attack dataset respectively, compare the recognition results, and obtain the robustness and anti-attack performance of the lane detection model. The evaluation steps for the anti-attack performance include: calculating the recognition accuracy Precision1 of the lane line detection model on the simulation dataset; calculating the recognition accuracy Precision2 of the lane line detection model on the physical attack dataset; and calculating the anti-attack performance R=(Precision1-Precision2) / (∑Hj), where Hj represents the entropy value of the j-th physical attack texture. , nij represents the number of pixels with grayscale value i in the j-th physical attack texture. This represents the total number of pixels in the j-th physics attack texture; The robustness is the difference between the F1 score of the lane detection model under test for the simulation dataset and the F1 score of the lane detection model under test for the physical attack dataset. The steps of generating a physical attack texture using the attack algorithm include: (1) initializing the texture for each simulation image in the simulation dataset; (2) superimposing the texture with the simulation image and inputting it into the lane detection model to obtain the loss value; (3) backpropagating the loss value, calculating the gradient, and updating the texture; (4) repeating steps (2)-(3) several times to optimize the texture and obtain the physical attack texture.

2. The simulation test method for autonomous driving lane line recognition based on physical attacks as described in claim 1, characterized in that, The operating scenarios include single dashed line, double solid line - turning, double solid line - overtaking, double dashed line, solid and dashed double line, elevated road, expressway, single solid line - uphill, downhill and / or tunnel.

3. The simulation test method for autonomous driving lane line recognition based on physical attacks as described in claim 1, characterized in that, The natural noises include rainy days, foggy days, snowy days, backlighting, and / or midnight.

4. A simulation test system for autonomous driving lane line recognition based on physical attacks, characterized in that, include: The simulation image creation module is configured to: create several working scenarios in the autonomous driving simulator, add natural noise to the working scenarios, obtain several simulation images, and form a simulation dataset. The physical attack module is configured to: generate a physical attack map using an attack algorithm for each simulation image in the simulation dataset, import the physical attack map into the autonomous driving simulator for resampling, and obtain the physical attack dataset; The simulation testing module is configured to: acquire the lane detection model to be tested, perform lane recognition on the simulation dataset and the physical attack dataset respectively, compare the recognition results, and obtain the robustness and anti-attack performance of the lane detection model. The evaluation steps for the anti-attack performance include: calculating the recognition accuracy Precision1 of the lane line detection model on the simulation dataset; calculating the recognition accuracy Precision2 of the lane line detection model on the physical attack dataset; and calculating the anti-attack performance R=(Precision1-Precision2) / (∑Hj), where Hj represents the entropy value of the j-th physical attack texture. , nij represents the number of pixels with grayscale value i in the j-th physical attack texture. This represents the total number of pixels in the j-th physics attack texture; The robustness is the difference between the F1 score of the lane detection model under test for the simulation dataset and the F1 score of the lane detection model under test for the physical attack dataset. The steps of generating a physical attack texture using the attack algorithm include: (1) initializing the texture for each simulation image in the simulation dataset; (2) superimposing the texture with the simulation image and inputting it into the lane detection model to obtain the loss value; (3) backpropagating the loss value, calculating the gradient, and updating the texture; (4) repeating steps (2)-(3) several times to optimize the texture and obtain the physical attack texture.

5. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the steps in the simulation test method for autonomous driving lane line recognition based on physical attacks as described in any one of claims 1-3.

6. A computer device comprising a computer-readable storage medium, a processor, and a computer program stored on the computer-readable storage medium and executable on the processor, characterized in that, When the processor executes the program, it implements the steps in the simulation test method for autonomous driving lane line recognition based on physical attacks as described in any one of claims 1-3.

Images