Automatic driving lane line identification simulation test method and system based on physical attack

Abstract

The invention relates to the technical field of lane line identification, and provides an automatic driving lane line identification simulation test method and system based on physical attacks, and the method comprises the steps: building a plurality of working condition scenes in an automatic driving simulator, adding natural noise in the working condition scenes, obtaining a plurality of simulation images, and forming a simulation data set; for each simulation image in the simulation data set, generating a physical attack map by adopting an attack algorithm, and importing the physical attack map into the automatic driving simulator for re-sampling to obtain a physical attack data set; and obtaining a to-be-detected lane line detection model, performing lane line identification on the simulation data set and the physical attack data set, and comparing identification results to obtain robustness and anti-attack performance of the lane line detection model. A real scene interfered by a large number of dynamic traffic elements can be effectively simulated, and more accurate safety evaluation is provided for an automatic driving lane line detection algorithm.

Description

Technical Field

[0001] The present invention belongs to the technical field of lane line recognition, and in particular to a physical attack-based automatic driving lane line recognition simulation test method and system. Background Art

[0002] The statements in this section merely provide background information related to the present invention and do not necessarily constitute prior art.

[0003] At present, lane detection models are mostly evaluated directly using accuracy, recall or F1 value. However, this method cannot evaluate the robustness of lane detection models in the face of physical attacks such as obstacle occlusion.

[0004] To solve the above problems, some scholars have proposed a test method for lane detection algorithms based on digital attacks, but this method usually focuses on simulating attacks on the lanes themselves, and seldom comprehensively combines the complex interactions of various traffic participants (such as motor vehicles, non-motor vehicles, pedestrians, etc. of different types and driving states) to conduct tests. On actual roads, the presence and behavior of surrounding traffic participants will affect the sensor's perception of lanes and the algorithm's detection of them. In real scenarios with a large number of dynamic traffic elements, the lane detection algorithm's ability to cope with digital attacks may be very different from its performance in simplified, relatively static test scenarios. Summary of the invention

[0005] In order to solve the technical problems existing in the above-mentioned background technology, the present invention provides an automatic driving lane line recognition simulation test method and system based on physical attack. A simulation data set including working conditions and natural noise is established based on an automatic driving simulator, and a physical attack is performed to obtain a texture file after the physical attack, which is imported into the automatic driving simulator for re-sampling to obtain a physical attack data set, which can effectively simulate the real scene of interference from a large number of dynamic traffic elements. The lane line detection model to be tested performs lane line recognition on the data set before and after the physical attack, thereby judging the performance of the lane line detection algorithm, and providing a more accurate safety evaluation for the automatic driving lane line detection algorithm.

[0006] In order to achieve the above object, the present invention adopts the following technical solution:

[0007] A first aspect of the present invention provides a physical attack-based automatic driving lane line recognition simulation test method, which includes:

[0008] Establish several working scenarios in the autonomous driving simulator, add natural noise to the working scenarios, obtain several simulation images, and constitute a simulation data set;

[0009] For each simulation image in the simulation data set, an attack algorithm is used to generate a physical attack map, and the physical attack map is imported into the autonomous driving simulator for resampling to obtain a physical attack data set;

[0010] The lane line detection model to be tested is obtained, and lane line recognition is performed on the simulation data set and the physical attack data set respectively. The recognition results are compared to obtain the robustness and anti-attack performance of the lane line detection model.

[0011] Furthermore, the anti-attack performance evaluation step includes:

[0012] Calculate the recognition accuracy Precision1 of the lane detection model for the simulation data set;

[0013] Calculate the recognition accuracy Precision2 of the lane detection model for the physical attack dataset;

[0014] Calculate the anti-attack performance R = (Precision1-Precision2) / (∑Hj), where Hj represents the entropy value of the jth physical attack map. Pij=nij / Nj, nij represents the number of pixels with gray value i in the j-th physical attack map, and Nj represents the total number of pixels in the j-th physical attack map.

[0015] Furthermore, the robustness is the difference between the F1 value of the lane line detection model to be tested for the simulation data set and the F1 value of the lane line detection model to be tested for the physical attack data set.

[0016] Furthermore, the operating scenarios include single dashed line, double solid line-turning, double solid line-overtaking, double dashed line, real dashed double line, elevated road, expressway, single solid line-uphill, downhill and / or tunnel.

[0017] Furthermore, the natural noise includes rainy days, foggy days, snowy days, backlight and / or midnight.

[0018] Furthermore, the step of using the attack algorithm to generate a physical attack map includes:

[0019] (1) For each simulated image in the simulation data set, initialize the map texture;

[0020] (2) After superimposing the texture map and the simulated image, the map is input into the lane detection model to obtain the loss value.

[0021] (3) Back propagate the loss value, calculate the gradient, and update the map texture;

[0022] (4) Repeat steps (2)-(3) several times to optimize the map texture and obtain the physical attack map.

[0023] A second aspect of the present invention provides an autonomous driving lane line recognition simulation test system based on physical attack, which comprises:

[0024] A simulation image creation module is configured to: establish a number of operating scenarios in an autonomous driving simulator, and add natural noise to the operating scenarios to obtain a number of simulation images to form a simulation data set;

[0025] A physical attack module is configured to: for each simulation image in the simulation data set, generate a physical attack map using an attack algorithm, import the physical attack map into the autonomous driving simulator for resampling, and obtain a physical attack data set;

[0026] The simulation test module is configured to: obtain the lane line detection model to be tested, perform lane line recognition on the simulation data set and the physical attack data set respectively, compare the recognition results, and obtain the robustness and anti-attack performance of the lane line detection model.

[0027] Furthermore, the anti-attack performance evaluation step includes:

[0028] Calculate the recognition accuracy Precision1 of the lane detection model for the simulation data set;

[0029] Calculate the recognition accuracy Precision2 of the lane detection model for the physical attack dataset;

[0030] Calculate the anti-attack performance R = (Precision1-Precision2) / (∑Hj), where Hj represents the entropy value of the jth physical attack map. Pij=nij / Nj, nij represents the number of pixels with gray value i in the j-th physical attack map, and Nj represents the total number of pixels in the j-th physical attack map.

[0031] A third aspect of the present invention provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps in the physical attack-based autonomous driving lane line recognition simulation test method as described above.

[0032] The fourth aspect of the present invention provides a computer device, including a computer-readable storage medium, a processor, and a computer program stored on the computer-readable storage medium and executable on the processor, wherein when the processor executes the program, the steps in the physical attack-based autonomous driving lane line recognition simulation test method as described above are implemented.

[0033] Compared with the prior art, the present invention has the following beneficial effects:

[0034] The present invention establishes a simulation data set including working conditions and natural noise based on an autonomous driving simulator, and performs physical attacks to obtain a texture file after the physical attack, which is then imported into the autonomous driving simulator for re-sampling to obtain a physical attack data set, which can effectively simulate a real scene with interference from a large number of dynamic traffic elements. The lane line detection model to be tested performs lane line recognition on the data set before and after the physical attack, thereby judging the performance of the lane line detection algorithm, and providing a more accurate safety evaluation for the autonomous driving lane line detection algorithm.

[0035] The present invention judges the robustness of the lane line detection algorithm according to the decrease of F1-score, and judges the anti-attack performance of the lane line detection model based on the entropy value of the physical attack map, providing a comprehensive evaluation of the autonomous driving lane line detection algorithm. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] The accompanying drawings in the specification, which constitute a part of the present invention, are used to provide a further understanding of the present invention. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute improper limitations on the present invention.

[0037] Figure 1 is a flow chart of a physical attack-based automatic driving lane line recognition simulation test method according to a first embodiment of the present invention;

[0038] Figure 2 This is a schematic diagram of a benchmark data set for autonomous driving lane recognition according to Embodiment 1 of the present invention;

[0039] Figure 3 This is a schematic diagram of a physical attack texture map according to the first embodiment of the present invention;

[0040] Figure 4 This is a schematic diagram of the physical attack data set effect of the first embodiment of the present invention;

[0041] Figure 5 It is a structural diagram of a computer device according to a fourth embodiment of the present invention. DETAILED DESCRIPTION

[0042] To make the purpose, technical solution and advantages of the embodiments of the present invention more clear, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings in the embodiments of the present invention.

[0043] Embodiment 1

[0044] This embodiment provides an autonomous driving lane line recognition simulation test method based on physical attack.

[0045] The automatic driving lane line recognition simulation test method based on physical attack provided in this embodiment firstly establishes a simulation data set including 10 working conditions including single dashed line, double solid line-turning, double solid line-overtaking, double dashed line, real dashed double line, elevated road, high speed, single solid line-uphill, downhill, and tunnel, as well as 6 weather conditions including sunny day, rainy day, foggy day, snowy day, backlight, and midnight based on the automatic driving simulator Carla; secondly, the audi.obj model file in Carla is physically attacked based on the C&W (Carlini&Wagner) attack algorithm (also known as CW attack algorithm). The attack is performed to obtain the texture file after the physical attack, which is imported into Carla for resampling to obtain the simulation data set after the physical attack. Finally, the Elf Annotation Assistant software is used to manually annotate the data set to obtain a json file containing the lane line coordinate information. The lane line is recognized based on the lane line detection model for the simulation data set before and after the physical attack. The robustness of the lane line detection algorithm is judged based on the F1-score decrease, and the anti-attack performance of the lane line detection model is judged based on the entropy value of the physical attack texture, providing a more accurate safety evaluation for the autonomous driving lane line detection algorithm.

[0046] This embodiment provides a physical attack-based automatic driving lane line recognition simulation test method, such as Figure 1 As shown, the specific steps include:

[0047] S1: Establish a data set: Install the autonomous driving simulator Carla based on Ubuntu 20.04 system, and establish a total of 10 working conditions in Carla, including single dashed line, double solid line-turning, double solid line-overtaking, double dashed line, real dashed double line, elevated road, high speed, single solid line-uphill, downhill, and tunnel, and represent them as basic scenarios, such as Figure 2 As shown, based on the basic scene, rainy days, foggy days, snowy days, backlight and / or midnight natural noise are added respectively to obtain several simulated images to constitute a lane line detection simulation dataset.

[0048] S2: Manual labeling: Use the Wizard Labeling Assistant software to manually label the lane lines involved in the simulation data set in S1 and obtain a json file containing the lane line coordinate information.

[0049] S3: Benchmark data set test: Based on the lane line detection model, lane line detection and recognition are performed on the simulation data set in S1 to complete the test of the benchmark data set, and its recognition accuracy is obtained based on the evaluation indicator F1-score.

[0050] In this embodiment, the lane line detection model may adopt GANet.

[0051] S4: Based on the C&W algorithm, a physical attack is performed on the vehicle model audi.obj in Carla. The implementation principle is as follows:

[0052] (1) For each simulated image in the simulation data set, initialize a texture. Generally, it can start with a random noise image, such as randomly generating an image matrix with a certain resolution and pixel values ​​within a certain range as the initial state, and its size must match the simulated image.

[0053] (2) After superimposing the map texture and the simulated image, the map texture is input into the lane line detection model GANet to obtain the loss value.

[0054] In this step, the loss value is calculated using the starting point regression loss in the GANet loss function:

[0055]

[0056] (3) Back propagate the loss value in step (2) and update the map texture according to the gradient.

[0057] (4) Repeat steps (2)-(3) to continuously optimize the texture map. After training is completed, save the map to obtain the texture map after physical attack, such as Figure 3 shown.

[0058] (4) The physical attack map obtained by training in step (5) is imported into the autonomous driving simulator carla for resampling to obtain the lane line recognition physical attack dataset. The effect diagram is shown in the figure below. Figure 4 shown.

[0059] S5: Physical attack dataset test: Based on the lane line detection model, the physical attack dataset in S4 is tested for lane line detection and recognition, and its recognition accuracy is obtained based on the evaluation indicator F1-score.

[0060] S6: Recognition accuracy comparison: Based on step S3 and step S5, compare the recognition accuracy of the lane detection model for the simulation data set before and after adding the physical attack, and judge the robustness of the lane detection model according to the decrease in F1-score; and evaluate the anti-attack performance of the lane detection model.

[0061] Among them, the evaluation steps of the lane detection model's anti-attack performance include:

[0062] (1) Calculate the recognition accuracy Precision1 of the lane detection model for the simulation data set before the attack, that is, the proportion of correctly predicted positives to all positive predictions, Precision = TP / (TP+FP);

[0063] (2) Calculate the recognition accuracy Precision2 of the lane detection model for the simulated data set after the attack;

[0064] (3) Calculate the anti-attack performance: R = (Precision1-Precision2) / (∑Hj), where Hj represents the entropy value of the jth physical attack map. Pij=nij / Nj, nij represents the number of pixels with gray value i in the j-th physical attack map, and Nj represents the total number of pixels in the j-th physical attack map.

[0065] Among them, the comparison results of the F1-score decrease are shown in Table 1. It can be seen from Table 1 that the recognition accuracy of the GANet model on the non-attack data set is 72.91%, and the recognition accuracy on the data set after adding physical attacks is 55.00%, a decrease of 17.91%, which provides a safety evaluation for the autonomous driving lane line recognition algorithm and provides an important basis for the robustness detection of the lane line detection algorithm.

[0066] Table 1. Comparison of GANet network recognition accuracy under different data sets

[0067]

[0068] Embodiment 2

[0069] This embodiment provides an automatic driving lane line recognition simulation test system based on physical attack, which specifically includes:

[0070] A simulation image creation module is configured to: establish a number of operating scenarios in an autonomous driving simulator, and add natural noise to the operating scenarios to obtain a number of simulation images to form a simulation data set;

[0071] A physical attack module is configured to: for each simulation image in the simulation data set, generate a physical attack map using an attack algorithm, import the physical attack map into the autonomous driving simulator for resampling, and obtain a physical attack data set;

[0072] The simulation test module is configured to: obtain the lane line detection model to be tested, perform lane line recognition on the simulation data set and the physical attack data set respectively, compare the recognition results, and obtain the robustness and anti-attack performance of the lane line detection model.

[0073] The robustness is the difference between the F1 value of the lane line detection model to be tested for the simulation data set and the F1 value of the lane line detection model to be tested for the physical attack data set.

[0074] The evaluation steps of anti-attack performance include:

[0075] Calculate the recognition accuracy Precision1 of the lane detection model for the simulation data set;

[0076] Calculate the recognition accuracy Precision2 of the lane detection model for the physical attack dataset;

[0077] Calculate the anti-attack performance R = (Precision1-Precision2) / (∑Hj), where Hj represents the entropy value of the jth physical attack map. Pij=nij / Nj, nij represents the number of pixels with gray value i in the j-th physical attack map, and Nj represents the total number of pixels in the j-th physical attack map.

[0078] It should be noted here that each module in this embodiment corresponds to each step in Example 1 one by one, and the specific implementation process is the same, which will not be repeated here.

[0079] Embodiment 3

[0080] This embodiment provides a computer-readable storage medium having a computer program stored thereon. When the program is executed by a processor, the steps in the physical attack-based autonomous driving lane line recognition simulation test method as described in the above-mentioned embodiment 1 are implemented.

[0081] Embodiment 4

[0082] This embodiment provides a computer device, such as Figure 5 As shown, it includes a display device, an input device, a computer-readable storage medium (volatile memory and non-volatile storage medium), a processor, a communication interface (i.e., a network interface), and a computer program stored on the computer-readable storage medium and executable on the processor, wherein the processor, the communication interface, and the computer-readable storage medium may be connected via a bus or other means. The communication interface is used to receive and send data, and when the processor executes the program, the steps in the automatic driving lane line recognition simulation test method based on physical attack as described in the first embodiment above are implemented.

[0083] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0084] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0085] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A simulation test method for autonomous driving lane line recognition based on physical attack, characterized in that: include: Establish several working scenarios in the autonomous driving simulator, add natural noise to the working scenarios, obtain several simulation images, and constitute a simulation data set; For each simulation image in the simulation data set, an attack algorithm is used to generate a physical attack map, and the physical attack map is imported into the autonomous driving simulator for resampling to obtain a physical attack data set; The lane line detection model to be tested is obtained, and lane line recognition is performed on the simulation data set and the physical attack data set respectively. The recognition results are compared to obtain the robustness and anti-attack performance of the lane line detection model.

2. The automatic driving lane line recognition simulation test method based on physical attack according to claim 1, characterized in that: The anti-attack performance evaluation steps include: Calculate the recognition accuracy Precision1 of the lane detection model for the simulation data set; Calculate the recognition accuracy Precision2 of the lane detection model for the physical attack dataset; Calculate the anti-attack performance R = (Precision1-Precision2) / (∑Hj), where Hj represents the entropy value of the jth physical attack map. Pij=nij / Nj, nij represents the number of pixels with gray value i in the j-th physical attack map, and Nj represents the total number of pixels in the j-th physical attack map.

3. The automatic driving lane line recognition simulation test method based on physical attack according to claim 1, characterized in that: The robustness is the difference between the F1 value of the lane line detection model to be tested for the simulation data set and the F1 value of the lane line detection model to be tested for the physical attack data set.

4. The automatic driving lane line recognition simulation test method based on physical attack according to claim 1, characterized in that: The operating scenarios include single dashed line, double solid line-turning, double solid line-overtaking, double dashed line, real dashed double line, elevated road, expressway, single solid line-uphill, downhill and / or tunnel.

5. The automatic driving lane line recognition simulation test method based on physical attack as claimed in claim 1, characterized in that: The natural noises include rainy days, foggy days, snowy days, backlight and / or midnight.

6. The automatic driving lane line recognition simulation test method based on physical attack according to claim 1, characterized in that: The step of using the attack algorithm to generate a physical attack map comprises: (1) For each simulated image in the simulation data set, initialize the map texture; (2) After superimposing the texture map and the simulated image, the map is input into the lane detection model to obtain the loss value. (3) Back propagate the loss value, calculate the gradient, and update the map texture; (4) Repeat steps (2)-(3) several times to optimize the map texture and obtain the physical attack map.

7. The automatic driving lane line recognition simulation test system based on physical attack is characterized by: include: A simulation image creation module is configured to: establish a number of operating scenarios in an autonomous driving simulator, and add natural noise to the operating scenarios to obtain a number of simulation images to form a simulation data set; A physical attack module is configured to: for each simulation image in the simulation data set, generate a physical attack map using an attack algorithm, import the physical attack map into the autonomous driving simulator for resampling, and obtain a physical attack data set; The simulation test module is configured to: obtain the lane line detection model to be tested, perform lane line recognition on the simulation data set and the physical attack data set respectively, compare the recognition results, and obtain the robustness and anti-attack performance of the lane line detection model.

8. The automatic driving lane line recognition simulation test system based on physical attack as claimed in claim 7, characterized in that: The anti-attack performance evaluation steps include: Calculate the recognition accuracy Precision1 of the lane detection model for the simulation data set; Calculate the recognition accuracy Precision2 of the lane detection model for the physical attack dataset; Calculate the anti-attack performance R = (Precision1-Precision2) / (∑Hj), where Hj represents the entropy value of the jth physical attack map. Pij=nij / Nj, nij represents the number of pixels with gray value i in the j-th physical attack map, and Nj represents the total number of pixels in the j-th physical attack map.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the steps in the physical attack-based autonomous driving lane line recognition simulation test method as described in any one of claims 1 to 6 are implemented.

10. A computer device comprising a computer-readable storage medium, a processor, and a computer program stored in the computer-readable storage medium and executable on the processor, characterized in that: When the processor executes the program, the steps in the physical attack-based autonomous driving lane line recognition simulation test method as described in any one of claims 1-6 are implemented.

Images