A method and system for generating hidden adversarial samples of lane lines

Abstract

The present invention discloses a method and system for generating hidden adversarial samples of lane lines, the method comprising: obtaining target image data with lane lines; performing edge detection on the target image data to obtain edge image data; converting the target image data from RGB space to HSV space to obtain HSV image data; inputting the edge image data and HSV image data into a model to obtain optimal perturbation, including: processing the edge image data and HSV image data using a DCNN network to obtain lane line feature data; obtaining optimal perturbation with the goal of minimizing an objective function; in the optimization process, processing the gradient of the objective function based on a first condition and a second condition to update the perturbation; adding the optimal perturbation to the target image data to obtain perturbed image data. The present invention can generate low-perturbation lane line adversarial samples in real time, and makes it possible for the model to not find that there is a problem with the perturbed image data.

Description

Technical Field

[0001] The present invention relates to the field of electronic digital data, and more particularly to a method and system for generating hidden adversarial samples of lane lines. Background Art

[0002] Lane detection is to determine the lane the vehicle is in by analyzing the images captured by the camera. With the development of deep learning technology, lane detection methods are increasingly relying on neural network models. Although these models perform well in many cases, they may be deceived by adversarial samples, which may affect the safety of autonomous driving systems. Therefore, lane adversarial samples are crucial for testing model robustness, improving model performance, and testing and improving the safety of autonomous driving lane detection systems.

[0003] There are three main methods for generating adversarial samples for lane detection: optimization-based methods, learning-based methods, and gradient-based methods. Optimization-based methods usually define an objective function and then generate adversarial samples by optimizing this function. Although this method can produce high-quality adversarial samples, it has high computational cost and is not suitable for real-time applications. Learning-based methods generate adversarial samples by training an adversarial generator. This method has a fast generation speed, but requires a large amount of training data, and the generated adversarial samples may deviate from the real scene. Gradient-based methods are simple to calculate, fast, and have good timeliness, but this method often adds noise to the entire image, which causes a large disturbance to the image.

[0004] Therefore, there is an urgent need for an adversarial sample generation method to solve the problems of high computational cost, large amount of training data required, and large disturbance in the existing technology. Summary of the invention

[0005] The present invention is proposed based on the above-mentioned requirements of the prior art. The technical problem to be solved by the present invention is to provide a method and system for generating hidden adversarial samples of lane lines to generate hidden perturbed image data in real time.

[0006] In order to solve the above problems, the present invention is implemented by adopting the following technical solutions:

[0007] A method for generating hidden adversarial samples of lane lines is provided, the method comprising: acquiring target image data with lane lines; performing edge detection on the target image data to obtain edge image data; converting the target image data from the RGB space to the HSV space to obtain the HSV image data; inputting the edge image data and the HSV image data into a model to obtain an optimal perturbation, comprising: processing the edge image data and the HSV image data using a DCNN network to obtain lane line feature data; obtaining an optimal perturbation with the goal of minimizing an objective function, wherein the objective function is the difference between the perturbed image data and the target image data; in an optimization process, processing the gradient of the objective function based on a first condition and a second condition to update the perturbation; the first condition comprising substituting the coordinates of the corresponding positions in the lane line feature data and the target image data into a Gaussian kernel function to obtain a mask value, and limiting the mask value to be less than a preset first value; the second condition is limiting the infinite norm of the perturbation to be less than a preset second value; adding the optimal perturbation to the target image data to obtain the perturbed image data as an adversarial sample.

[0008] Optionally, edge detection is performed on the target image data to obtain edge image data, including: using a Gaussian filter to smooth the target image data to obtain first image data; performing first-order partial derivative calculations on the first image data to obtain second image data; performing non-maximum suppression on the second image data to obtain third image data; performing double threshold processing on the third image data to divide pixel points into strong edge points, weak edge points and non-edge points; and connecting weak edge points with surrounding strong edge points to obtain edge image data.

[0009] Optionally, the target image data is converted from the RGB space to the HSV space to obtain HSV image data, including: , , , where H represents the hue value in the HSV image data, G represents the pixel value in the green channel, B represents the pixel value in the blue channel, R represents the pixel value in the red channel, S represents the saturation value in the HSV image data, and V represents the brightness value in the HSV image data.

[0010] Optionally, the DCNN network is trained using a cross entropy loss function.

[0011] Optionally, the objective function is expressed as: ,in, represents the objective function, represents the target image data, represents disturbance, represents the L2 norm.

[0012] Optionally, the first condition includes: ,in, M ( x,y ) represents the mask value, x Represents the horizontal axis coordinate value of the target image data, y Represents the vertical axis coordinate value of the target image data, P x Indicates the lane feature data The corresponding coordinate values ​​are P y Indicates the lane feature data y The corresponding coordinate values ​​are represents the standard deviation of the Gaussian kernel.

[0013] Optionally, processing the gradient of the objective function based on the first condition and the second condition to update the disturbance includes: ,in, Indicates t +1 iteration of perturbation, Indicates t The perturbation of the iteration, Indicates the preset step size, Represents the gradient of the objective function.

[0014] Optionally, it also includes: calculating the number of pixels in the disturbed image data that are correctly predicted as lane lines as a first value; calculating the number of pixels in the disturbed image data that are correctly predicted as non-lane lines as a second value; calculating the number of pixels in the disturbed image data that are erroneously predicted as lane lines as a third value; calculating the number of pixels in the disturbed image data that are erroneously predicted as lane lines as a fourth value; and obtaining the accuracy of the model based on the proportion of the sum of the first value and the second value in the sum of the first value, the second value, the third value and the fourth value to evaluate the performance difference between the target image data and the disturbed image data.

[0015] Optionally, the expression for calculating the accuracy is: , , , , , where TP represents the first value, represents the position in the adversarial perturbation The pixel value of Indicates the location in the target image data Lane line label, TN represents the second value, FP represents the third value, FN represents the fourth value, and Accuracy represents the accuracy.

[0016] A hidden adversarial sample generation system for lane lines is provided, the system comprising: an acquisition module, acquiring target image data with lane lines; a detection module, performing edge detection on the target image data to obtain edge image data; a first processing module, converting the target image data from the RGB space to the HSV space to obtain the HSV image data; a second processing module, inputting the edge image data and the HSV image data into a model to obtain an optimal perturbation, comprising: processing the edge image data and the HSV image data using a DCNN network to obtain lane line feature data; obtaining an optimal perturbation with the goal of minimizing an objective function, wherein the objective function is the difference between the perturbed image data and the target image data; in an optimization process, processing the gradient of the objective function based on a first condition and a second condition to update the perturbation; the first condition comprises substituting the coordinates of the corresponding positions in the lane line feature data and the target image data into a Gaussian kernel function to obtain a mask value, and limiting the mask value to be less than a preset first value; the second condition is limiting the infinite norm of the perturbation to be less than a preset second value; an output module, adding the optimal perturbation to the target image data to obtain perturbed image data, and outputting the perturbed image data as an adversarial sample.

[0017] Compared with the prior art, the present invention provides a method and system for generating hidden adversarial samples of lane lines, which generate low-disturbance lane line adversarial samples in real time, making it impossible for the model to detect problems with the perturbed image data as much as possible, and evaluating the robustness of the lane line detection algorithm to cope with the risk of adversarial attacks, thereby helping to improve the robustness of the lane line detection algorithm. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] In order to more clearly illustrate the embodiments of this specification or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in the embodiments of this specification. For ordinary technicians in this field, other drawings can also be obtained based on these drawings.

[0019] Figure 1 This is a flowchart of a method for generating hidden adversarial samples of lane lines provided in this embodiment. DETAILED DESCRIPTION

[0020] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0021] To facilitate understanding of the embodiments of the present invention, further explanation will be given below with reference to specific embodiments in conjunction with the accompanying drawings. The embodiments do not constitute a limitation on the protection scope of the present invention.

[0022] Example 1

[0023] This embodiment provides a method for generating hidden adversarial samples of lane lines, and the process is as follows: Figure 1 As shown, including:

[0024] S1 obtains target image data with lane lines.

[0025] This embodiment acquires target image data ,in, Indicates the height of the target image data. Indicates the width of the target image data. Indicates the number of channels of the target image data.

[0026] S2 performs edge detection on the target image data to obtain edge image data.

[0027] S200 uses a Gaussian filter to smooth the grayscaled target image data to obtain first image data.

[0028] The image data is convolved with a Gaussian filter to reduce the impact of noise on edge detection results.

[0029] S210 performs first-order partial derivative calculation on the first image data to obtain second image data.

[0030] The Sobel operator is used to calculate the gradient values ​​of the first image data in the horizontal and vertical directions to obtain the gradient intensity and direction of each pixel.

[0031] S220 performs non-maximum suppression on the second image data to obtain third image data.

[0032] In this step, non-maximum suppression processing is performed on each pixel in the second image data, including comparing the gradient strength of the current pixel with the two adjacent pixels along the positive and negative gradient directions. If the gradient strength of the current pixel is greater than that of the other two pixels, the pixel is retained as an edge point, otherwise the pixel will be suppressed.

[0033] S230 performs double threshold processing on the third image data to divide the pixel points into strong edge points, weak edge points and non-edge points.

[0034] Two thresholds are preset, namely a low threshold and a high threshold; if the gradient value of an edge point is higher than the high threshold, the point is a strong edge point; if the gradient value of an edge point is higher than the low threshold and lower than the high threshold, the point is a weak edge point; if the gradient value of an edge point is less than the low threshold, the pixel value of the point is set to 0.

[0035] S240 connects the weak edge point with the surrounding strong edge points to obtain edge image data.

[0036] S3 converts the target image data from the RGB space to the HSV space to obtain HSV image data.

[0037] By converting the RGB color space into the HSV space, the color information of the target image data can be extracted. The expression is:

[0038]

[0039]

[0040]

[0041] Among them, H represents the hue value in the HSV image data, G represents the pixel value in the green channel, B represents the pixel value in the blue channel, R represents the pixel value in the red channel, S represents the saturation value in the HSV image data, and V represents the brightness value in the HSV image data.

[0042] S4 inputs the edge image data and the HSV image data into the model to obtain the optimal perturbation.

[0043] S400 uses the DCNN network to process edge image data and HSV image data to obtain lane line feature data.

[0044] The DCNN network consists of multiple convolutional layers, activation function layers, and pooling layers connected in sequence.

[0045] The expression of the convolutional layer is:

[0046]

[0047] in, Represents the convolution kernel k At the location of the input image data The output data at L represents the height of the convolution kernel, M represents the width of the convolution kernel, represents the position on the convolution kernel, C represents the channel of the convolution kernel, c Indicates the identification number of the convolution kernel channel, represents the input image data, represents the parameters of the convolution kernel, represents the first bias term.

[0048] The fully connected layer maps the features extracted by the convolutional layer to the label space, and its expression is:

[0049]

[0050] in, represents the label value, softmax() represents the softmax function, W represents the weight parameter of the fully connected layer, flatten() represents the function of expanding into one-dimensional data, Represents the output data of the last convolutional layer, b represents the second bias term.

[0051] Use cross entropy loss function as the loss function of DCNN Loss , whose expression is:

[0052]

[0053] in, N represents the number of samples, n Indicates n samples, represents the true label, Represents the predicted label.

[0054] The back propagation algorithm and optimization algorithm are used to update the DCNN network parameters to minimize the loss function.

[0055] The lane line feature data is obtained through network forward propagation.

[0056] S410 aims to minimize the objective function and obtain the optimal disturbance.

[0057] The objective function is constructed by the difference between the perturbed image data and the target image data , ,in, represents disturbance, represents the L2 norm.

[0058] The adversarial perturbation is generated through the objective function to minimize the impact on the lane line detection results near the lane line feature data.

[0059] The objective function is limited to the vicinity of the lane line feature data according to the first condition. The first condition includes substituting the coordinates of the corresponding positions in the lane line feature data and the target image data into the Gaussian kernel function to obtain a mask value, and limiting the mask value to be less than a preset first value. Preferably, the preset first value is 1, and the objective function is limited to the area where the mask function is 1.

[0060]

[0061] in, M ( x,y ) represents the mask value, x Represents the horizontal axis coordinate value of the target image data, y Represents the vertical axis coordinate value of the target image data, P x Indicates the lane feature data The corresponding coordinate values ​​are P y Indicates the lane feature data y The corresponding coordinate values ​​are represents the standard deviation of the Gaussian kernel.

[0062] The range of the generated disturbance is limited according to the second condition to avoid excessive disturbance. The second condition is to limit the infinite norm of the disturbance to be less than a preset second value to control the amplitude of the disturbance.

[0063]

[0064] in, Indicates the preset second value.

[0065] The perturbation is updated using the gradient descent algorithm until the optimization target is reached or a preset number of iterations is reached.

[0066] During the optimization process, the gradient of the objective function is processed based on the first condition and the second condition, and the disturbance is updated.

[0067]

[0068] in, Indicates t +1 iteration of perturbation, Indicates t The perturbation of the iteration, Indicates the preset step size, Represents the gradient of the objective function.

[0069] S5 adds the optimal disturbance to the target image data to obtain the disturbed image data.

[0070] The expression for this step is:

[0071]

[0072] This embodiment generates adversarial perturbations near the lane line feature positions through the above method, and uses the generated perturbed image data as adversarial samples.

[0073] Furthermore, evaluation metrics will be used to measure the performance difference of the model on the target image data and adversarial samples, specifically:

[0074] The number of pixels correctly predicted as lane lines in the disturbed image data is calculated as the first value.

[0075]

[0076] Wherein, TP represents the first value, represents the position in the adversarial perturbation The pixel value of Indicates the location in the target image data Lane labels, The value is 0 or 1.

[0077] The number of pixels correctly predicted as non-lane lines in the disturbed image data is calculated as the second value.

[0078]

[0079] Here, TN represents the second value.

[0080] The number of pixels in the disturbed image data that are incorrectly predicted as lane lines and that are non-lane lines is calculated as the third value.

[0081]

[0082] Here, FP represents the third value.

[0083] The number of pixels in the disturbed image data that incorrectly predict the lane line as a non-lane line is calculated as the fourth value.

[0084]

[0085] Here, FN represents the fourth value.

[0086] According to the proportion of the sum of the first value and the second value in the sum of the first value, the second value, the third value and the fourth value, the accuracy of the model is obtained to evaluate the performance difference between the target image data and the perturbation image data.

[0087]

[0088] Here, Accuracy represents the accuracy, which is characterized by the ratio of the number of correctly predicted lane line pixels to the total number of lane line pixels.

[0089] This embodiment provides a method for generating hidden adversarial samples of lane lines, which generates low-disturbance lane line adversarial samples in real time, making it impossible for the model to detect problems with the perturbed image data as much as possible, and evaluating the robustness of the lane line detection algorithm to cope with the risk of adversarial attacks, thereby helping to improve the robustness of the lane line detection algorithm.

[0090] Example 2

[0091] This embodiment provides a system for generating hidden adversarial samples of lane lines, which implements the method for generating hidden adversarial samples of lane lines described in Embodiment 1, including:

[0092] The acquisition module obtains the target image data with lane lines.

[0093] The detection module performs edge detection on the target image data to obtain edge image data.

[0094] The first processing module converts the target image data from the RGB space to the HSV space to obtain the HSV image data.

[0095] The second processing module inputs the edge image data and the HSV image data into the model to obtain the optimal disturbance, including: using the DCNN network to process the edge image data and the HSV image data to obtain the lane line feature data; taking minimizing the objective function as the goal to obtain the optimal disturbance, the objective function is the difference between the disturbed image data and the target image data; in the optimization process, the gradient of the objective function is processed based on the first condition and the second condition to update the disturbance; the first condition includes substituting the coordinates of the corresponding positions in the lane line feature data and the target image data into the Gaussian kernel function to obtain the mask value, and limiting the mask value to be less than a preset first value; the second condition is to limit the infinite norm of the disturbance to be less than a preset second value.

[0096] The output module adds the optimal perturbation to the target image data to obtain the perturbed image data, and outputs the perturbed image data as an adversarial sample.

[0097] The specific implementation methods described above further illustrate the objectives, technical solutions and beneficial effects of the present invention in detail. It should be understood that the above description is only a specific implementation method of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A method for generating hidden adversarial samples of lane lines, characterized in that: include: Obtain target image data with lane lines; Performing edge detection on target image data to obtain edge image data; Convert the target image data from the RGB space to the HSV space to obtain the HSV image data; Inputting edge image data and HSV image data into the model to obtain optimal perturbation includes: using a DCNN network to process edge image data and HSV image data to obtain lane feature data; minimizing an objective function to obtain optimal perturbation, wherein the objective function is the difference between the perturbed image data and the target image data; in the optimization process, processing the gradient of the objective function based on a first condition and a second condition to update the perturbation; the first condition includes substituting the coordinates of the corresponding positions in the lane feature data and the target image data into a Gaussian kernel function to obtain a mask value, and limiting the mask value to be less than a preset first value; the second condition is to limit the infinite norm of the perturbation to be less than a preset second value; The optimal perturbation is added to the target image data to obtain the perturbed image data as an adversarial sample.

2. The method for generating hidden adversarial samples of lane lines according to claim 1, characterized in that: Perform edge detection on the target image data to obtain edge image data, including: Use a Gaussian filter to smooth the target image data to obtain first image data; Performing first-order partial derivative calculation on the first image data to obtain second image data; Performing non-maximum suppression on the second image data to obtain third image data; Performing double threshold processing on the third image data to divide the pixel points into strong edge points, weak edge points and non-edge points; Connect the weak edge points with the surrounding strong edge points to obtain edge image data.

3. The method for generating hidden adversarial samples of lane lines according to claim 1, characterized in that: Convert the target image data from RGB space to HSV space to obtain HSV image data, including: Among them, H represents the hue value in the HSV image data, G represents the pixel value in the green channel, B represents the pixel value in the blue channel, R represents the pixel value in the red channel, S represents the saturation value in the HSV image data, and V represents the brightness value in the HSV image data.

4. The method for generating hidden adversarial samples of lane lines according to claim 1, characterized in that: Also includes: The DCNN network is trained using the cross entropy loss function.

5. The method for generating hidden adversarial samples of lane lines according to claim 1, characterized in that: The expression of the objective function is: in, represents the objective function, represents the target image data, represents disturbance, represents the L2 norm.

6. The method for generating hidden adversarial samples of lane lines according to claim 1, characterized in that: The first condition includes: in, M ( x,y ) represents the mask value, x Represents the horizontal axis coordinate value of the target image data, y Represents the vertical axis coordinate value of the target image data, P x Indicates the lane feature data The corresponding coordinate values ​​are P y Indicates the lane feature data y The corresponding coordinate values ​​are represents the standard deviation of the Gaussian kernel.

7. The method for generating hidden adversarial samples of lane lines according to claim 1, characterized in that: The gradient of the objective function is processed based on the first condition and the second condition, and the disturbance is updated, including: in, Indicates t +1 iteration of perturbation, Indicates t The perturbation of the iteration, Indicates the preset step size, Represents the gradient of the objective function.

8. The method for generating hidden adversarial samples of lane lines according to claim 1, characterized in that: Also includes: Calculate the number of pixels correctly predicted as lane lines in the disturbed image data as a first value; Calculate the number of pixels correctly predicted as non-lane lines in the disturbed image data as a second value; Calculate the number of pixels in the disturbed image data where the non-lane line is incorrectly predicted as the lane line, as the third value; Calculate the number of pixels in the disturbed image data where the lane line is incorrectly predicted as a non-lane line, as a fourth value; According to the proportion of the sum of the first value and the second value in the sum of the first value, the second value, the third value and the fourth value, the accuracy of the model is obtained to evaluate the performance difference between the target image data and the perturbation image data.

9. The method for generating hidden adversarial samples of lane lines according to claim 8, characterized in that: The expression for calculating the accuracy is: Wherein, TP represents the first value, represents the position in the adversarial perturbation The pixel value of Indicates the location in the target image data Lane line label, TN represents the second value, FP represents the third value, FN represents the fourth value, and Accuracy represents the accuracy.

10. A system for generating hidden adversarial samples of lane lines, characterized in that: include: An acquisition module, which acquires target image data with lane lines; A detection module performs edge detection on target image data to obtain edge image data; The first processing module converts the target image data from the RGB space to the HSV space to obtain the HSV image data; The second processing module inputs the edge image data and the HSV image data into the model to obtain the optimal disturbance, including: using the DCNN network to process the edge image data and the HSV image data to obtain the lane line feature data; taking minimizing the objective function as the goal to obtain the optimal disturbance, the objective function is the difference between the disturbance image data and the target image data; in the optimization process, the gradient of the objective function is processed based on the first condition and the second condition to update the disturbance; the first condition includes substituting the coordinates of the corresponding positions in the lane line feature data and the target image data into the Gaussian kernel function to obtain the mask value, and limiting the mask value to be less than the preset first value; the second condition is to limit the infinite norm of the disturbance to be less than the preset second value; The output module adds the optimal perturbation to the target image data to obtain the perturbed image data, and outputs the perturbed image data as an adversarial sample.

Images