Lane line detection improvement method for improving expected function safety
By improving the lane line detection model laneATT, adding PGD module and classification network Ladv_x, and conducting confrontation training, the problem of inaccurate lane line detection in the existing technology is solved, and the robustness and detection effect of the model are improved.
Patent Information
- Application Number
- CN202411457240.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-10-18
- Publication Date
- 2025-02-14
AI Technical Summary
The existing lane line detection technology is susceptible to light, occlusion and damage, resulting in inaccurate detection and reduced detection effect when data distribution changes.
Improve the lane line detection model laneATT, and perform adversarial training to learn robust features by adding PGD module and new classification network Ladv_x, and add adversarial sample generation and iterative update processes to improve the detection capability of the model.
The detection effect of lane line detection in the face of occlusion, damage and light changes is improved, and the robustness and adaptability of the model are enhanced.
Smart Images
Figure CN119445514A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of lane line detection, and in particular to a lane line detection improvement method for improving the safety of expected functions. Background Art
[0002] Lane detection is one of the key technologies for autonomous driving and is used in lane departure warning, lane change, and forward collision to ensure vehicle safety. Lane departure warning issues a warning when the vehicle accidentally deviates from the current lane. Lane change provides auxiliary or automated support when the vehicle changes lanes. Forward collision detects potential collision risks by detecting the road ahead and the position of other vehicles. Usually, lane detection uses computer vision technology and deep learning to identify lanes. Cameras are usually used for lane detection. Lane detection is easily affected by light, occlusion, and damage, resulting in inaccurate lane detection.
[0003] Traditional lane line detection technology is usually adjusted manually, which is labor-intensive and has poor robustness. The lane line detection currently used is usually based on deep learning and uses computer vision to detect. It usually uses a camera to capture image data and a visual algorithm to identify and process it. Therefore, the lane line detection of the visual solution is easily affected by occlusion, light, and no visual index. The shape of the lane line is thin and long, and it is difficult for the visual algorithm to successfully detect it. In addition, the lane line changes in various forms, such as straight lines, turns, and dotted lines, making it difficult to detect accurately. At the same time, the lane line is also damaged, causing the shape of the lane line to change. Although lane line detection has achieved good results in the data set, when the data distribution changes, the detection effect will be reduced, and most of these distribution changes are caused by lane line occlusion, damage, and light changes. Therefore, in actual use, the effect of lane line detection based on visual algorithms is greatly affected by the environment. Summary of the invention
[0004] In view of the deficiencies of the prior art, the present invention provides an improved lane line detection method that enhances the expected functional safety; when the lane line is blocked, damaged or the light changes, the detection effect is improved.
[0005] A lane line detection improvement method for improving expected functional safety includes the following steps:
[0006] Step 1: Use a camera to collect lane line scene data of different road sections, and divide the collected data into training set, validation set and test set according to the set ratio;
[0007] Step 2: Improve the existing public model laneATT lane detection model. Specifically, add the PGD module to the network framework and adopt a new classification network Ladv_x To control the domain information of the learned adversarial samples, the collected image data is used for training and testing to obtain the final model;
[0008] Step 2.1: The collected image data is first passed through the PDG layer to generate adversarial samples and adversarial sample labels:
[0009] Suppose the model input sample is x, the existing public model laneATT is function f, and the output after the PDG layer operation is the adversarial sample x adv ,The loss function of the existing public model laneATT is L. After the input sample x enters the PGD layer, an initial perturbation δ0 is first added to it, as shown in the following formula:
[0010] x adv =x+δ0 (1)
[0011] The adversarial sample x of the output obtained by formula (1) adv To crop and scale:
[0012] x adv =Π x+S (x adv ) (2)
[0013] Get a new adversarial sample x adv , where S represents the range;
[0014] Step 2.2: Iterative update of generated adversarial samples:
[0015] Based on formula (2), the cropped sample x adv Input into the existing public model laneATT, and get f(x adv ), calculate the loss function L on the adversarial sample x adv The corresponding gradient under the condition is obtained, and the gradient value grad is:
[0016]
[0017] The gradient value grad passes through the sign function sgn(), multiplied by the set single-step perturbation size Iter_eps, added to the original perturbation to update the perturbation, and added to the adversarial sample x generated in the previous step. adv middle:
[0018] δ=δ0+Iter_eps·sgn(grad) (4)
[0019] Update sample:
[0020] x adv =x adv +δ (5)
[0021] After t iterations, the final adversarial sample is obtained Each iteration will generate an adversarial sample. After t iterations, t adversarial samples are obtained. The adversarial sample that maximizes the loss function L is found among the adversarial samples as the final output:
[0022] x adv =maxL(x adv t ) (6)
[0023] Step 2.3, perform lane line detection;
[0024] Step 2.3.1 In the process of lane line detection, anchor points need to be used to project the anchor points onto the features extracted from the input data through the backbone network. The specific process is as follows:
[0025] When performing anchor projection, the image is first divided into different rows. Assuming there are N pts -1 row, forming a set of ordinate points, where Y represents the ordinate, and the calculation method of the i-th coordinate point is as follows:
[0026]
[0027] Among them, H I Indicates height; assuming the horizontal axis is h i , the horizontal coordinate set is H h , then all horizontal coordinates are set to:
[0028]
[0029] Each horizontal axis corresponds to a vertical axis.
[0030] The anchor point consists of a boundary and a tilt angle θ. The boundary is defined as follows, where O represents the starting point and h orig is the horizontal coordinate of the starting point, y orig The vertical coordinate of the starting point is:
[0031] O=(h orig ,y orig ) (9)
[0032] Therefore, the horizontal coordinate of the anchor point is obtained by the following formula:
[0033]
[0034] Where θ represents the tilt angle, y j Represents a set of vertical coordinates, h j Indicates the horizontal coordinate corresponding to each y, δ back Represents the global step size, the scaling of the entire image after sampling;
[0035] Step 2.3.2 After obtaining the anchor point, extract local features and global features;
[0036] Extract local feature vectors through anchor boxes;
[0037]
[0038] in Represents the i-th local feature, which belongs to the feature map with C channels F The feature map height is H F In the feature space;
[0039] Next, the local feature vector is input into the attention mechanism to obtain the global feature vector, which is shown in the following formula:
[0040]
[0041] Where L att Represents the attention mechanism layer. When the local feature i and other features j are equal, the attention weight is 0. When the two are not equal, the local feature passes through the attention mechanism layer L. att After calculation, the obtained value is converted to the range of 0 to 1 through the softmax activation function to obtain the attention weight; then all local features are According to the attention weight W i Multiply and add to get the global features Enter the prediction part for training.
[0042]
[0043] Step 2.3.3 performs prediction and loss function:
[0044] The local features and global features are predicted through three FC layers. The first two FC layers are used to predict the classification and offset, length. The third FC layer is used as a classification network to control the domain information D during training. The loss function is specifically defined as follows:
[0045]
[0046] where p i Indicates the type of the i-th prediction point, r i Indicates the predicted value of offset and length, from 0 to N pts After the -1 prediction point is input into the loss function L, its loss is calculated. Indicates the calculation of the predicted point p i The actual predicted point The loss between , is taken as the classification loss. Indicates the calculation of the predicted point ri and the true prediction point r i * The loss is used as the prediction loss of offset and length. Represents the prediction domain information D i With real domain information The loss between , λ represents the weight.
[0047] Step 2.4: Train the existing public model laneATT and the improved laneATT model separately to obtain the final model;
[0048] Step 2.4.1: Train the existing public model laneATT; input the input data x that has not been processed by the adversarial sample PGD layer into the existing public model laneATT model for training, update the model parameters, and obtain the trained feature extraction model;
[0049] Step 2.4.2: Train the classifier; use the trained feature extraction model and the improved laneATT for training, only allowing the added classification network to update parameters, and obtain the classification network model;
[0050] Step 2.4.3: Repeat the training; use the improved laneATT for training, only allow the improved laneATT model network to update parameters, and prohibit the classification network from updating parameters to obtain the final model.
[0051] Step 3: Perform lane line detection:
[0052] The original test data that has not been processed by the adversarial sample PGD layer is input into the network, and the existing public model LaneATT and the improved laneATT model obtained through training are used for prediction to obtain the final result.
[0053] The beneficial effects of adopting the above technical solution are:
[0054] The present invention provides an improved lane line detection method for improving the expected functional safety. The method adopts an adversarial training method to assist the model in training, and at the same time adds a classification network to reduce the influence of domain information, so as to learn more robust features and improve the model's detection effect in the face of lane line occlusion, damage, and light change. BRIEF DESCRIPTION OF THE DRAWINGS
[0055] Figure 1 This is a diagram of the overall network structure of an embodiment of the present invention. DETAILED DESCRIPTION
[0056] The specific implementation of the present invention is further described in detail below in conjunction with the accompanying drawings and examples. The following examples are used to illustrate the present invention, but are not intended to limit the scope of the present invention.
[0057] An improved lane detection method to improve the safety of expected functions, such as Figure 1 As shown, the following steps are included:
[0058] Step 1: Use a camera to collect lane line scene data of different road sections, and divide the collected data into training set, validation set and test set according to the set ratio;
[0059] In this embodiment, a GoPro 10Black camera is used to collect lane line data on different road sections in Jiangsu Province, Liaoning Province, Beijing and other regions in China, covering scenes of normal roads, crowded roads, night roads, worn lane lines, shadows, arrow signs, reflections, curves, and crossroads. The collected data is divided into a training set, a validation set, and a test set in a ratio of 7:2:1;
[0060] Step 2: Improve the existing public model laneATT lane detection model, add the PGD module to the network framework for data enhancement, and use a new classification network L adv_x To control the domain information of the learned adversarial samples, these measures are used to improve the model's ability to detect lane damage, occlusion, and light changes. Then the collected image data is used for training and testing to obtain the final model;
[0061] Step 2.1: The collected image data is first passed through the PDG layer to generate adversarial samples and adversarial sample labels:
[0062] Suppose the model input sample is x, the existing public model laneATT is function f, and the output after the PDG layer operation is the adversarial sample x adv ,The loss function of the existing public model laneATT is L. After the input sample x enters the PGD layer, an initial perturbation δ0 is first added to it, as shown in the following formula:
[0063] x adv =x+δ0 (15)
[0064] The adversarial sample x of the output obtained by formula (1) adv To crop and scale:
[0065] x adv =Π x+S (x adv ) (16)
[0066] Get a new adversarial sample x adv , where S represents the range, and the cropping and scaling operations ensure that the new adversarial sample x adv In the x+S range, ensure that the disturbance is not too large;
[0067] Step 2.2: Iterative update of generated adversarial samples:
[0068] Based on formula (2), the cropped sample x adv Input into the existing public model laneATT, and get f(x adv ), calculate the loss function L on the adversarial sample x adv The corresponding gradient under the condition is obtained, and the gradient value grad is:
[0069]
[0070] The gradient value grad passes through the sign function sgn(), multiplied by the set single-step perturbation size Iter_eps, added to the original perturbation to update the perturbation, and added to the adversarial sample x generated in the previous step. adv middle:
[0071] δ=δ0+Iter_eps·sgn(grad) (18)
[0072] Update sample:
[0073] x adv =x adv +δ (19)
[0074] After t iterations, the final adversarial sample is obtained Each iteration will generate an adversarial sample. After t iterations, t adversarial samples are obtained. The adversarial sample that maximizes the loss function L is found among the adversarial samples as the final output:
[0075] x adv =maxL(x adv t ) (20)
[0076] Step 2.3, perform lane line detection;
[0077] Step 2.3.1 In the process of lane line detection, anchor points need to be used to project the anchor points onto the features extracted from the input data through the backbone network. The specific process is as follows:
[0078] When performing anchor projection, the image is first divided into different rows. Assuming there are N pts -1 row, forming a set of ordinate points, where Y represents the ordinate, and the calculation method of the i-th coordinate point is as follows:
[0079]
[0080] Among them, H I Indicates height; assuming the horizontal axis is h i , the horizontal coordinate set is Hh , then all horizontal coordinates are set to:
[0081]
[0082] Each horizontal axis corresponds to a vertical axis.
[0083] The anchor point consists of a boundary and a tilt angle θ. The boundary is defined as follows, where O represents the starting point and h orig is the horizontal coordinate of the starting point, y orig The vertical coordinate of the starting point is:
[0084] O=(h orig ,y orig ) (twenty three)
[0085] Therefore, the horizontal coordinate of the anchor point is obtained by the following formula:
[0086]
[0087] Where θ represents the tilt angle, y j Represents a set of vertical coordinates, h j Indicates the horizontal coordinate corresponding to each y, δ back Represents the global step size, the scaling of the entire image after sampling;
[0088] Step 2.3.2 After obtaining the anchor point, extract local features and global features;
[0089] Extract local feature vectors through anchor boxes;
[0090]
[0091] in Represents the i-th local feature, which belongs to the feature map with C channels F The feature map height is H F In the feature space;
[0092] Next, the local feature vector is input into the attention mechanism to obtain the global feature vector, which is shown in the following formula:
[0093]
[0094] Where L att Represents the attention mechanism layer. When the local feature i and other features j are equal, the attention weight is 0. When the two are not equal, the local feature passes through the attention mechanism layer L. att After calculation, the obtained value is converted to the range of 0 to 1 through the softmax activation function to obtain the attention weight; then all local features are According to the attention weight W iMultiply and add to get the global features Enter the prediction part for training.
[0095]
[0096] Step 2.3.3 performs prediction and loss function:
[0097] The local features and global features are predicted through three FC layers. The first two FC layers are used to predict the classification and offset, length. The third FC layer is used as a classification network to control the domain information D during training. The loss function is specifically defined as follows:
[0098]
[0099] where p i Indicates the type of the i-th prediction point, r i Indicates the predicted value of offset and length, from 0 to N pts After the -1 prediction point is input into the loss function L, its loss is calculated. Indicates the calculation of the predicted point p i The actual predicted point The loss between , is taken as the classification loss. Indicates the calculation of the predicted point r i and the true prediction point r i * The loss is used as the prediction loss of offset and length. Represents the prediction domain information D i Real domain information The loss between , λ represents the weight.
[0100] Step 2.4: Train the existing public model laneATT and the improved laneATT model separately to obtain the final model;
[0101] Step 2.4.1: Train the existing public model laneATT; input the input data x that has not been processed by the adversarial sample PGD layer into the existing public model laneATT model for training, update the model parameters, and obtain the trained feature extraction model;
[0102] Step 2.4.2: Train the classifier; use the trained feature extraction model and the improved laneATT for training, only allowing the added classification network to update parameters, and obtain the classification network model;
[0103] Step 2.4.3: Repeat the training; use the improved laneATT for training, only allow the improved laneATT model network to update parameters, and prohibit the classification network from updating parameters to obtain the final model.
[0104] Step 3: Perform lane line detection:
[0105] The original test data that has not been processed by the adversarial sample PGD layer is input into the network, and the existing public model LaneATT and the improved laneATT model obtained through training are used for prediction to obtain the final result.
[0106] The above description is only a preferred embodiment of the present disclosure and an explanation of the technical principles used. Those skilled in the art should understand that the scope of the invention involved in the embodiments of the present disclosure is not limited to the technical solutions formed by a specific combination of the above-mentioned technical features, but should also cover other technical solutions formed by any combination of the above-mentioned technical features or their equivalent features without departing from the above-mentioned inventive concept. For example, the above-mentioned features are replaced with the technical features with similar functions disclosed in the embodiments of the present disclosure (but not limited to) to form a technical solution.
Claims
1. A lane line detection improvement method for improving expected functional safety, characterized in that: The following steps are involved: Step 1: Use a camera to collect lane line scene data of different road sections, and divide the collected data into training set, validation set and test set according to the set ratio; Step 2: Improve the existing public model laneATT lane detection model. Specifically, add the PGD layer to the network framework and adopt a new classification network L adv_x To control the domain information of the learned adversarial samples, the collected image data is used for training and testing to obtain the improved laneATT model; Step 3: Perform lane line detection; The original test data that has not been processed by the adversarial sample PGD layer is input into the network, and the existing public model LaneATT and the improved laneATT model obtained through training are used for prediction to obtain the final result.
2. According to claim 1, a lane line detection improvement method for improving expected functional safety is characterized in that: The step 2 comprises the following steps: Step 2.1: The collected image data is first passed through the PDG layer to generate adversarial samples and adversarial sample labels: Step 2.2: Iterative update of generated adversarial samples: Step 2.3, perform lane line detection; Step 2.4: Train the existing public model laneATT and the improved laneATT model separately to obtain the final model.
3. The lane line detection improvement method for improving the expected functional safety according to claim 2, characterized in that: Step 2.1 is as follows: Suppose the model input sample is x, the existing public model laneATT is function f, and the output after the PDG layer operation is the adversarial sample x adv ,The loss function of the existing public model laneATT is L. After the input sample x enters the PGD layer, an initial perturbation δ0 is first added to it, as shown in the following formula: x adv =x+δ0 (1) The adversarial sample x of the output obtained by formula (1) adv To crop and scale: x adv =P x+S (x adv ) (2) Get a new adversarial sample x adv , where S represents the range.
4. The lane line detection improvement method for improving expected functional safety according to claim 3, characterized in that: Step 2.2 includes the following steps: Based on formula (2), the cropped sample x adv Input into the existing public model laneATT, and get f(x adv ), calculate the loss function L on the adversarial sample x adv The corresponding gradient under the condition is obtained, and the gradient value grad is: The gradient value grad passes through the sign function sgn(), multiplied by the set single-step perturbation size Iter_eps, added to the original perturbation to update the perturbation, and added to the adversarial sample x generated in the previous step. adv middle: δ=δ0+Iter_eps·sgn(grad) (4) Update sample: x adv =x adv +δ (5) After t iterations, the final adversarial sample is obtained Each iteration will generate an adversarial sample. After t iterations, t adversarial samples are obtained. The adversarial sample that maximizes the loss function L is found among the adversarial samples as the final output: x adv =maxL(x adv t )。 (6) 5. The lane line detection improvement method for improving expected functional safety according to claim 3, characterized in that: Step 2.3 includes the following steps: Step 2.3.1 In the process of lane line detection, anchor points need to be used to project the anchor points onto the features extracted from the input data through the backbone network. The specific process is as follows: When performing anchor projection, the image is first divided into different rows. Assuming there are N pts -1 row, forming a set of ordinate points, where Y represents the ordinate, and the calculation method of the i-th coordinate point is as follows: Among them, H I Indicates height; assuming the horizontal axis is h i , the horizontal coordinate set is H h , then all horizontal coordinates are set to: Each horizontal axis corresponds to a vertical axis; The anchor point consists of a boundary and a tilt angle θ. The boundary is defined as follows, where O represents the starting point and h orig is the horizontal coordinate of the starting point, y orig The vertical coordinate of the starting point is: O=(h orig ,y orig ) (9) Therefore, the horizontal coordinate of the anchor point is obtained by the following formula: Where θ represents the tilt angle, y j Represents a set of vertical coordinates, h j Indicates the horizontal coordinate corresponding to each y, δ back Represents the global step size, the scaling of the entire image after sampling; Step 2.3.2 After obtaining the anchor point, extract local features and global features; Extract local feature vectors through anchor boxes; in Represents the i-th local feature, which belongs to the feature map with C channels F The feature map height is H F In the feature space; Next, the local feature vector is input into the attention mechanism to obtain the global feature vector, which is shown in the following formula: Where L att Represents the attention mechanism layer. When the local feature i and other features j are equal, the attention weight is 0. When the two are not equal, the local feature passes through the attention mechanism layer L. att After calculation, the obtained value is converted to the range of 0 to 1 through the softmax activation function to obtain the attention weight; then all local features are According to the attention weight W i Multiply and add to get the global features Enter the prediction part for training; Step 2.3.3 performs prediction and loss function: The local features and global features are predicted through three FC layers. The first two FC layers are used to predict the classification and offset, length. The third FC layer is used as a classification network to control the domain information D during training. The loss function is specifically defined as follows: where p i Indicates the type of the i-th prediction point, r i Indicates the predicted value of offset and length, from 0 to N pts -1 prediction point is input into the loss function L and its loss is calculated; Indicates the calculation of the predicted point p i The actual predicted point The loss between them is used as the classification loss; Indicates the calculation of the predicted point r i and the true prediction point r i * The loss is used as the prediction loss of offset and length; Represents the prediction domain information D i Real domain information The loss between , λ represents the weight.
6. The lane line detection improvement method for improving expected functional safety according to claim 3, characterized in that: Step 2.4 includes the following steps: Step 2.4.1: Train the existing public model laneATT; input the input data x that has not been processed by the adversarial sample PGD layer into the existing public model laneATT model for training, update the model parameters, and obtain the trained feature extraction model; Step 2.4.2: Train the classifier; use the trained feature extraction model and the improved laneATT for training, only allowing the added classification network to update parameters, and obtain the classification network model; Step 2.4.3: Repeat the training; use the improved laneATT for training, only allow the improved laneATT model network to update parameters, and prohibit the classification network from updating parameters to obtain the final model.