Lane line detection method, device, unmanned equipment and storage medium

Abstract

The present invention discloses a lane line detection method, device, unmanned equipment and storage medium. Obtain a current frame image collected by an unmanned equipment; determine a first detection result of the current frame image based on the current frame image, the first detection result includes a first probability value that the first curve in the current frame image is a lane line; obtain a second detection result of the previous frame image, the second detection result includes a second probability value that the second curve in the previous frame image is a lane line; calculate the distance between the first curve and the second curve; when the distance between the first curve and the second curve is less than a distance threshold, select the curve corresponding to the larger value of the first probability value and the second probability value as the lane line. Using the detection result of the previous frame as an input in the current frame detection model improves the stability and robustness of the lane line detection, has strong interpretability and high precision; has strong versatility, so that the detection method is highly flexible, the model training difficulty is low, the calculation amount is small, and the detection time is reduced.

Description

Technical Field

[0001] The present invention relates to the field of autonomous driving technology, and in particular to a lane line detection method, device, unmanned equipment and storage medium. Background Art

[0002] Lane detection is a very important task in the automated driving assistance system. The performance of the automated driving assistance system can be improved by detecting lane lines and reverse lane keeping, lane departure monitoring and warning, automatic lane change, semantic map construction and other functional tasks.

[0003] At present, the main method of lane line detection is to use multiple sensors to fuse to detect lane lines, but the use of sensors for lane line detection is greatly affected by the environment. There is also the use of a single image for lane line detection, which is easily affected by the presence of occlusions or sudden changes in light in the image and cannot meet the actual lane line detection needs. The real-time performance is low and the detection result accuracy is poor. Summary of the invention

[0004] The present invention provides a lane line detection method, device, unmanned equipment and storage medium, which are suitable for improving the real-time and universality of lane line detection, reducing the amount of calculation and reducing time consumption.

[0005] According to one aspect of the present invention, a lane line detection method is provided, comprising:

[0006] Get the current frame image collected by the unmanned equipment;

[0007] Determine a first detection result of the current frame image based on the current frame image, the first detection result comprising a first probability value that a first curve in the current frame image is a lane line;

[0008] Acquire a second detection result of the previous frame image, wherein the second detection result includes a second probability value that a second curve in the previous frame image is a lane line;

[0009] calculating a distance between the first curve and the second curve;

[0010] When the distance between the first curve and the second curve is less than a distance threshold, a curve corresponding to a larger value of the first probability value and the second probability value is selected as the lane line.

[0011] According to another aspect of the present invention, there is provided a lane line detection device, comprising:

[0012] The current frame acquisition module is used to obtain the current frame image collected by the unmanned equipment;

[0013] A first detection result determination module, configured to determine a first detection result of the current frame image based on the current frame image, wherein the first detection result includes a first probability value that a first curve in the current frame image is a lane line;

[0014] A second detection result determination module, used to obtain a second detection result of a previous frame image, wherein the second detection result includes a second probability value that a second curve in the previous frame image is a lane line;

[0015] A distance calculation module, used for calculating the distance between the first curve and the second curve;

[0016] The lane line detection module is used to select a curve corresponding to a larger value of a first probability value and a second probability value as a lane line when the distance between the first curve and the second curve is less than a distance threshold.

[0017] According to another aspect of the present invention, there is provided an unmanned device, the unmanned device comprising:

[0018] at least one processor; and

[0019] a memory communicatively connected to the at least one processor; wherein,

[0020] The memory stores a computer program that can be executed by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the lane line detection method described in any embodiment of the present invention.

[0021] According to another aspect of the present invention, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the lane line detection method described in any embodiment of the present invention when executed.

[0022] The technical solution of the embodiment of the present invention is to obtain the current frame image collected by the unmanned equipment; determine the first detection result of the current frame image based on the current frame image, the first detection result includes the first probability value that the first curve in the current frame image is the lane line; obtain the second detection result of the previous frame image, the second detection result includes the second probability value that the second curve in the previous frame image is the lane line; calculate the distance between the first curve and the second curve; when the distance between the first curve and the second curve is less than the distance threshold, select the curve corresponding to the larger value of the first probability value and the second probability value as the lane line; the technical solution of the embodiment of the present invention uses the detection result of the previous frame as one of the inputs in the current frame detection model, and the lane line detection result is more accurate. Any lane detection model can be used to detect the lane line in the current frame image, so that the lane line detection method is highly flexible, thereby reducing the time consumption of lane line detection.

[0023] It should be understood that the contents described in this section are not intended to identify the key or important features of the embodiments of the present invention, nor are they intended to limit the scope of the present invention. Other features of the present invention will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0025] Figure 1 is a flow chart of a lane line detection method provided according to Embodiment 1 of the present invention;

[0026] Figure 2a is a flow chart of a lane line detection method provided according to Embodiment 2 of the present invention;

[0027] Figure 2b It is a schematic diagram of evenly dividing a fitting curve in the horizontal direction into a plurality of sampling points with the same horizontal coordinate distance between adjacent sampling points according to the second embodiment of the present invention;

[0028] Figure 2c is a schematic diagram of evenly dividing a fitting curve in the vertical direction into a plurality of sampling points with the same ordinate distance between adjacent sampling points according to the second embodiment of the present invention;

[0029] Figure 2d It is a schematic diagram of evenly dividing a fitting curve in the curve direction into a plurality of sampling points with the same length between adjacent sampling points according to the second embodiment of the present invention;

[0030] Figure 2e is a schematic diagram of obtaining a corrected second curve by correcting the position of each sampling point based on the offset value of each sampling point according to the second embodiment of the present invention;

[0031] Figure 3 is a flow chart of a lane line detection method provided according to Embodiment 3 of the present invention;

[0032] Figure 4 is a structural schematic diagram of a lane line detection device provided according to a fourth embodiment of the present invention;

[0033] Figure 5 is a schematic structural diagram of an unmanned device for implementing the lane line detection method of an embodiment of the present invention; DETAILED DESCRIPTION

[0034] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0035] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0036] Embodiment 1

[0037] Figure 1 A flow chart of a lane line detection method is provided for the first embodiment of the present invention. This embodiment can be applied to improve the real-time and universality of lane line detection, and reduce the amount of calculation and time consumption. The method can be executed by a lane line detection device, which can be implemented in the form of hardware and / or software. The lane line detection device can be configured in a computer device, such as a server, workstation, mobile terminal, vehicle, or sweeping robot. Figure 1 As shown, the method includes:

[0038] S101, obtaining a current frame image collected by an unmanned device.

[0039] The unmanned equipment is equipped with a camera at a certain location, and the camera of the unmanned equipment continuously collects images by video or timed shooting, and obtains the image of the current frame collected by the unmanned equipment to perform real-time lane line detection. For example, the unmanned equipment can be an unmanned vehicle, a logistics robot, etc., which is only an example and not limited to the present invention.

[0040] It should be noted that in some embodiments of the present invention, each frame captured in the video can be used as an image collected by the unmanned equipment. In some embodiments of the present invention, some images can also be regularly extracted from the video as images collected by the unmanned equipment, so as to further reduce the amount of calculation for lane line detection prediction while ensuring the accuracy of lane line detection. The embodiments of the present invention are only examples and are not limited thereto.

[0041] S102: Determine a first detection result of the current frame image based on the current frame image, where the first detection result includes a first probability value that a first curve in the current frame image is a lane line.

[0042] The current frame image is input into the lane line detection model to obtain a first detection result corresponding to the current frame image. The first detection result includes a first curve representing a lane line detected in the current frame image and a first probability value that the first curve is a lane line. Specifically, the first curve may not exist in the current frame image, or there may be several curves.

[0043] In some embodiments of the present invention, several point sequences including discrete point coordinates may be used to represent the first curve. Exemplarily, if the first curve does not exist in the current frame image, a point sequence in which all discrete points are special values ​​such as (-1, -1) is set to represent the first curve. If there are several first curves in the current frame image, several point sequences are used to represent different first curves, and several first probability values ​​corresponding to the first curves are obtained.

[0044] In the embodiment of the present invention, using a point sequence to represent the first curve is an exemplary description of the present invention. In other embodiments of the present invention, other methods may be used to represent the first curve. The embodiment of the present invention is only an example and is not limiting.

[0045] S103: Obtain a second detection result of the previous frame image, where the second detection result includes a second probability value that the second curve in the previous frame image is a lane line.

[0046] The second detection result includes a second curve representing a lane line detected in the previous frame image, and a second probability value that the second curve is a lane line.

[0047] Exemplarily, the lane line detection device combines the previous frame image and the lane lines detected by the previous frame image as the second detection result of the previous frame image. The lane lines and their probability values ​​detected by the lane line detection device in the previous frame image are obtained as the second curves and the second probability values ​​corresponding to the second curves.

[0048] In other embodiments of the present invention, the previous frame image may be input into the lane detection line to obtain a second detection result corresponding to the previous frame image, wherein the second detection result includes a plurality of second curves and corresponding second probability values.

[0049] The embodiment of the present invention only provides an example of how to obtain the second detection result of the previous frame image, and does not provide a limitation thereto.

[0050] S104: Calculate the distance between the first curve and the second curve.

[0051] The distances between the obtained first curves and the obtained second curves are calculated to remove the curves with a smaller probability from the curves representing the lane lines detected in the current frame image, so as to achieve a deduplication effect.

[0052] Exemplarily, to calculate the distance between the first curve and the second curve, a number of corresponding points on the two curves may be obtained, the distance between the corresponding points of the two curves may be calculated, and the average of the obtained distances may be used as the distance between the two curves.

[0053] The calculation of the distance between the first curve and the second curve in the embodiment of the present invention is for illustrative purposes only. In other embodiments of the present invention, weights may be set for the distances of different corresponding points, and the obtained distances may be multiplied by the average of the weights to obtain the distance between the two curves. The present invention is for illustrative purposes only and is not intended to be limiting.

[0054] S105. When the distance between the first curve and the second curve is less than a distance threshold, select a curve corresponding to a larger value between the first probability value and the second probability value as a lane line.

[0055] The distance between the first curve and the second curve is compared with a distance threshold, where the distance threshold is pre-set to select several curves that may represent the same lane line, including the first curve and the second curve, and according to the first probability values ​​of the several first curves and the second probability values ​​of the several second curves, the corresponding curve with the larger probability value is selected as the lane line.

[0056] It should be noted that the rules for determining lane lines in the embodiments of the present invention are for illustrative purposes only. Other rules may be used in other embodiments of the present invention. The embodiments of the present invention are only for illustrative purposes and are not intended to be limiting.

[0057] Some existing technical solutions currently fuse the independent detection results of multiple frames of images in 3D space (Three-Dimensional). The integration of time series information is at the bottom of the entire process, which is difficult to affect the inference process of the model and has limited improvement effect. Another technical solution uses LSTM (Long Short-Term Memory) network to use two state variables based on matrix algorithm in the structure to transmit time series information. The training of this model is difficult, the internal state variables are easily confused, and the interpretability is poor. The embodiment of the present invention realizes the integration of time series information upstream of the detection model by using the detection result of the previous frame as one of the inputs in the current frame detection model, thereby improving the accuracy of the lane line detection results. In addition, the embodiment of the present invention can use any lane detection model to detect the lane line in the current frame image, so that the lane line detection method is highly flexible and reduces the model training cost.

[0058] The technical solution of the embodiment of the present invention is to obtain the current frame image collected by the unmanned equipment; determine the first detection result of the current frame image based on the current frame image, the first detection result includes the first probability value that the first curve in the current frame image is the lane line; obtain the second detection result of the previous frame image, the second detection result includes the second probability value that the second curve in the previous frame image is the lane line; calculate the distance between the first curve and the second curve; when the distance between the first curve and the second curve is less than the distance threshold, select the curve corresponding to the larger value of the first probability value and the second probability value as the lane line; the technical solution of the embodiment of the present invention uses the detection result of the previous frame as one of the inputs in the current frame detection model, and the lane line detection result is more accurate. The present invention can use any lane detection model to detect the lane line in the current frame image, so that the lane line detection method is highly flexible and reduces the model training cost.

[0059] Embodiment 2

[0060] Figure 2a Flow chart of a lane line detection method provided in Embodiment 2 of the present invention. This embodiment is a refinement of the above embodiments. Figure 2a As shown, the method includes:

[0061] S201. Obtain a current frame image collected by an unmanned device.

[0062] S202: Determine a first detection result of the current frame image based on the current frame image, where the first detection result includes a first probability value that a first curve in the current frame image is a lane line.

[0063] The current frame image is input into a pre-trained lane line detection model for processing to obtain a first detection result of the current frame image.

[0064] The lane detection model is trained using images from the sample library. It can perform lane line detection on the input image and obtain several curves representing the lane lines.

[0065] The current frame image is input into a pre-trained lane line detection model to detect the lane line, and a first detection result of the current frame image is obtained, including a plurality of first curves representing the lane lines and a plurality of first probability values ​​corresponding to the plurality of first curves.

[0066] It should be noted that the lane line detection model is not limited in the embodiment of the present invention. In the embodiment of the present invention, the YOLO algorithm model (You Only Look Once), the SSD algorithm model (Single Shot MultiBox Detector), the CNN algorithm model (Convolutional Neural Network), etc. can be used as the lane line detection model. In other embodiments of the present invention, any other detection model can also be used as the lane line detection model. The embodiment of the present invention is only an example and is not limited.

[0067] S203: Obtain a second detection result of the previous frame image, where the second detection result includes a second probability value that the second curve in the previous frame image is a lane line.

[0068] S204: Correct the deviation of the second detection result caused by the movement of the vehicle to obtain a corrected second detection result.

[0069] Since the movement of the vehicle may cause the images captured by the previous frame image and the current frame image to shift and change, when the second detection result of the previous frame image is used for lane detection, the second detection result of the previous frame image needs to be offset and corrected to obtain the corrected second detection result, which can be used as the candidate result of lane line detection at the current moment, thereby improving the precision and accuracy of lane line detection and preventing detection errors caused by the time interval between frames. The corrected second detection result includes the corrected second curve and the second probability value corresponding to the second curve.

[0070] It should be noted that the embodiment of the present invention does not limit the method for correcting the deviation of the second detection result caused by the movement of the vehicle.

[0071] In some embodiments of the present invention, S204 includes:

[0072] The second detection result is input into the trained deviation correction model for processing to obtain the offset value of the second detection result in time series as the time series information. The offset value is used to correct the second curve in the second detection result to increase the reliability of the lane line detection result.

[0073] In some embodiments of the present invention, S204 includes:

[0074] S421, down-sampling the point sequence to obtain a plurality of sampling points.

[0075] The continuous point sequence representing the second curve in the second detection result is downsampled to compress the point sequence to reduce the amount of calculation, prevent overfitting, and increase the receptive field. The downsampled point sequence is further sampled to obtain a number of sampling points on the downsampled point sequence that can represent the second curve in the second detection result.

[0076] Exemplarily, a feature map of size H×W×C is downsampled to a feature map of size N×M×C, where H is the height of the feature map, W is the width of the feature map, C is the number of channels of the feature map, N is the maximum number of lane lines preset in advance, M is the number of sampling points in the pre-set point sequence, and C is the number of channels of the feature map. Then, the point sequence based on the feature map of size N×M×C is further sampled to obtain a number of sampling points.

[0077] It should be noted that in the embodiment of the present invention, the downsampling processing of the continuous point sequence representing the second curve in the second detection result is an exemplary description. In other embodiments of the present invention, the continuous point sequence representing the first curve in the first detection result can also be merged and downsampled together. The present invention is only an example and not limited.

[0078] In addition, the downsampling method in the embodiment of the present invention is only used as an example and is not limited thereto.

[0079] In some embodiments of the present invention, S421 includes:

[0080] S211, performing curve fitting on the point sequence to obtain a fitting curve.

[0081] The curve fitting equation is used on the downsampled point sequence to fit the lane line to form a continuous curve, which transforms each discrete point in the point sequence into a continuous curve. The corresponding position of the continuous curve on the feature map includes the feature information of the lane line, so that the feature information of the lane line can be connected in series.

[0082] Exemplarily, a polynomial curve or a Bezier curve is used to perform curve fitting on the downsampled point sequence to obtain a lane line fitting curve.

[0083] It should be noted that the use of polynomial curves or Bezier curves for curve fitting in the embodiments of the present invention is for exemplary purposes only. In other embodiments of the present invention, different curve fitting equations may be used to perform curve fitting on point sequences according to actual needs. The present invention is only for example and not limited thereto.

[0084] S212, uniformly sampling the fitting curve to obtain a plurality of sampling points.

[0085] The fitting curve is uniformly sampled according to preset rules to obtain a number of regular sampling points, so as to clearly display the characteristic information of the lane line.

[0086] The embodiments of the present invention are only examples and are not limiting.

[0087] In one embodiment of the present invention, S212 may be to evenly divide the fitting curve in the horizontal direction into a plurality of sampling points with the same horizontal coordinate distance between adjacent sampling points.

[0088] Figure 2b is a schematic diagram of evenly dividing a fitting curve in the horizontal direction into a plurality of sampling points with the same horizontal coordinate distance between adjacent sampling points according to the second embodiment of the present invention, such as Figure 2b As shown, the fitting curve is evenly divided into a number of sampling points in the horizontal direction, that is, along the direction of the X-coordinate axis, and the distances between the horizontal coordinates of these sampling points are the same.

[0089] Exemplarily, the horizontal coordinate distance between adjacent sampling points may be a preset equal-width distance value.

[0090] The embodiments of the present invention are only examples and are not limiting. In other embodiments of the present invention, the fitting curve may be evenly divided into a preset number of sampling points in the horizontal direction, and the horizontal coordinate distances between adjacent sampling points are the same.

[0091] In another embodiment of the present invention, S212 may be to evenly divide the fitting curve in the vertical direction into a plurality of sampling points with the same longitudinal coordinate distance between adjacent sampling points.

[0092] Figure 2c is a schematic diagram of evenly dividing a fitting curve in the vertical direction into a plurality of sampling points with the same ordinate distance between adjacent sampling points according to the second embodiment of the present invention, such as Figure 2c As shown, the fitting curve is evenly divided into a number of sampling points along the vertical direction, that is, along the direction of the Y coordinate, and the distances between the vertical coordinates of these sampling points are the same.

[0093] Exemplarily, the ordinate distance between adjacent sampling points may be a preset equal-height distance value.

[0094] The embodiments of the present invention are only examples and are not limiting. In other embodiments of the present invention, the fitting curve may be evenly divided into a preset number of sampling points in the vertical direction, and the vertical coordinate distances between adjacent sampling points are the same.

[0095] In another embodiment of the present invention, S212 may be to evenly divide the fitting curve in the curve direction into a plurality of sampling points with the same length between adjacent sampling points.

[0096] Figure 2d is a schematic diagram of evenly dividing a fitting curve into a plurality of sampling points with the same length between adjacent sampling points in the direction of the curve according to the second embodiment of the present invention, such as Figure 2d As shown, the fitting curve is evenly divided into a number of sampling points according to the curve direction, and the lengths of the curve segments of these sampling points are the same.

[0097] Exemplarily, the length of the curve segment between adjacent sampling points may be a preset length value.

[0098] The embodiments of the present invention are only examples and are not limiting. In other embodiments of the present invention, the fitting curve may be evenly divided into a preset number of sampling points along the curve direction, and the lengths of the curve segments between adjacent sampling points are the same.

[0099] S422: For each sampling point, extract the local features of the local area where the sampling point is located from the previous frame image, and obtain a feature vector composed of the local features of several sampling points.

[0100] For each sampling point obtained by uniform sampling, the local features in the local area of ​​fixed size corresponding to each sampling point are extracted from the feature map after downsampling of the previous frame image, and the local features of several sampling points in the feature map are connected to obtain a characteristic vector containing all the feature information of the entire lane line. The lane line in the image is prone to unclear local visual features due to occlusion or illumination changes. The embodiment of the present invention uses a feature vector composed of local features of several sampling points, which gathers the local feature information of all sampling points, can better deal with the problem caused by the narrow and long lane line structure, and alleviates the problem caused by the inability of the local receptive field to capture the global information of the lane line.

[0101] For example, there are 5 sampling points, each of which corresponds to a 3D vector f i , where i = 0..4, the features of the 5 sampling points are connected into a 5×3-dimensional vector (f0, f1, f2, f3, f4) in a preset order, such as from top to bottom.

[0102] It should be noted that the connection into a 5×3-dimensional vector in the embodiment of the present invention is for exemplary purposes. In other embodiments of the present invention, it can be connected into feature vectors of other dimensions according to actual scenarios. The embodiment of the present invention is only an example and is not limiting.

[0103] S423: Classify and regress the feature vector to obtain a probability value that the second curve is a lane line as a corrected second probability value, and an offset value of each sampling point.

[0104] After obtaining the feature vector containing all the feature information of the entire lane line, the feature vector is classified and regressed to obtain the probability value that the second curve is the lane line, so as to further correct the second probability value corresponding to the second curve and determine whether the lane line is included in the previous frame image to avoid false detection. In addition, the offset value of each sampling point is obtained, and the range of the offset value is any real value, so that the offset value of each sampling point can be used to correct the lane line offset caused by vehicle movement.

[0105] Exemplarily, after classification and regression processing of the feature vector, (1+2×M) dimensional information is obtained, where M is the number of sampling points; one dimension is the corrected second probability value, and the 2×M dimensions are the horizontal and vertical offset values ​​of each of the M sampling points.

[0106] It should be noted that in other embodiments of the present invention, after the feature vectors are classified and regressed, the information of (1+2×M) dimensions is obtained as an exemplary illustration of the present invention. In other embodiments of the present invention, information of other dimensions can also be obtained according to changes in actual needs. The embodiments of the present invention are only examples and are not limited.

[0107] In some embodiments of the present invention, S423 includes:

[0108] S231. Input the feature vector into the first fully connected layer to obtain a fully connected vector.

[0109] S232. Input the fully connected vector into an activation function for classification, and obtain a probability value that the second curve is a lane line.

[0110] The first fully connected layer is used to classify whether there is a lane line in the feature vector. The feature vector containing the feature information of the entire lane line is input into the first fully connected layer to obtain a fully connected vector. The fully connected vector obtained after processing the first fully connected layer is input into an activation function to convert the fully connected vector into a probability value in the range of 0 to 1 as the probability value of the second curve being a lane line.

[0111] For example, the feature vector is input into the first fully connected layer to obtain a fully connected vector, and the fully connected vector is input into the Sigmoid activation function to obtain a probability value indicating whether the feature vector is a lane line. In information science, due to the monotonic increasing properties of the Sigmoid function and the monotonic increasing properties of the inverse function, the Sigmoid function is often used as an activation function of a neural network to map variables between 0 and 1, making the result smooth and easy to derive.

[0112] It should be noted that, in the embodiment of the present invention, the Sigmoid function is used as an activation function for exemplary explanation. In other embodiments of the present invention, other functions may be used as activation functions for classification. The embodiment of the present invention is only an example and is not limited.

[0113] S233: Input the feature vector into the second fully connected layer for regression processing to obtain the offset value of each sampling point on the second curve.

[0114] The second fully connected layer is used to regress the offset value of each sampling point in the feature vector. The feature vector is input into the second fully connected layer for regression processing to obtain the horizontal and vertical offset values ​​of each sampling point on the second curve. Exemplarily, when there are M sampling points on the second curve, the output of the second fully connected layer has a total of 2×M dimensions.

[0115] It should be noted that, in the embodiment of the present invention, the horizontal and vertical offset values ​​of each sampling point on the second curve obtained in the fully connected layer are for exemplary purposes only. In other embodiments of the present invention, other dimensions related to the sampling point offset may also be used. The embodiment of the present invention is only an example and is not limited thereto.

[0116] In some embodiments of the present invention, the first fully connected layer and the second fully connected layer may be the same fully connected layer. Classification of whether a lane line exists in a feature vector and regression of the offset value of each sampling point in the feature vector may be completed in the same fully connected layer. The present invention is for example only and is not limited thereto.

[0117] The method for classifying and regressing feature vectors in the embodiment of the present invention is only an example and is not limiting.

[0118] S424: Correct the position of each sampling point based on the offset value of each sampling point to obtain a corrected second curve.

[0119] Figure 2e FIG. 1 is a schematic diagram of obtaining a corrected second curve by correcting the position of each sampling point based on the offset value of each sampling point according to the second embodiment of the present invention, as shown in FIG. Figure 2e As shown, according to the lateral offset value and longitudinal offset value of each sampling point obtained by regression, the coordinates of each sampling point are corrected to correct the lateral and longitudinal coordinate offsets caused by the movement of the vehicle.

[0120] In some embodiments of the present invention, S424 includes:

[0121] S241 . Add the horizontal coordinate of the sampling point to the horizontal offset value in the offset value to obtain a corrected horizontal coordinate of the sampling point.

[0122] S242: Add the longitudinal offset value in the offset value to the longitudinal coordinate of the sampling point to obtain a corrected longitudinal coordinate of the sampling point.

[0123] For example, the coordinates of the sampling points are (x p ,y p ), the second fully connected layer regresses the horizontal and vertical offset values ​​to obtain △x and △y respectively. p Add the horizontal offset value △x in the offset value to change the vertical coordinate y of the sampling point p Add the longitudinal offset value △y in the offset value, and the corrected sampling point coordinates are (x p +△x,y p +△y).

[0124] The embodiments of the present invention are only examples and are not limiting.

[0125] S205: Calculate the distance between the first curve and the second curve.

[0126] To calculate the distance between the first curve and the second curve, a number of corresponding points on the two curves may be taken, the distance between the corresponding points of the two curves may be calculated, and the average of the obtained distances may be taken as the distance between the two curves.

[0127] For example, the coordinates of several sampling points on the first curve are (x qi ,y qi ), the coordinates of several sampling points on the modified second curve are (x pi +△x,y pi +△y), i=1..m, m is the number of sampling points, and the Euclidean formula is used to calculate the coordinates (x qi ,y qi ) and the coordinates on the second curve (x pi +△x,y pi +△y), as the i-th sub-distance between the first curve and the second curve, calculate i sub-distances in sequence, and take the average of the i sub-distances as the distance between the first curve and the second curve.

[0128] It should be noted that the embodiment of the present invention is an exemplary description of calculating the distance between the first curve and the second curve. In other embodiments of the present invention, weights may be set for the distances of different corresponding points, and the obtained distances are multiplied by the average value of the weights to be used as the distance between the two curves. The present invention is only an example and is not limited. At the same time, the first curve may be the first curve after the deviation is corrected, which is not limited in the embodiment of the present invention.

[0129] S206: When the distance between the first curve and the second curve is less than the distance threshold, select a curve corresponding to a larger value between the first probability value and the second probability value as the lane line.

[0130] An average value of i sub-distances between i sampling points on the first curve and the corrected second curve is used as the distance between the first curve and the second curve and compared with the distance threshold. When the distance between the first curve and the second curve is less than the preset distance threshold, a curve with a larger probability value is selected as the lane line according to the first probability values ​​of several first curves and the second probability values ​​of several second curves that are less than the distance threshold range.

[0131] Exemplarily, there is a first curve and a second curve within a range less than a distance threshold, wherein a first probability value corresponding to the first curve is 89%, and a second probability value corresponding to the second curve is 93%, then the second curve with a larger probability value is selected as the lane line for output.

[0132] It should be noted that the rules for determining lane lines in the embodiments of the present invention are for illustrative purposes only. Other rules may be used in other embodiments of the present invention. The embodiments of the present invention are only for illustrative purposes and are not intended to be limiting.

[0133] The technical solution of the embodiment of the present invention is to obtain the current frame image collected by the unmanned equipment; determine the first detection result of the current frame image based on the current frame image, the first detection result includes the first probability value that the first curve in the current frame image is the lane line; obtain the second detection result of the previous frame image, the second detection result includes the second probability value that the second curve in the previous frame image is the lane line; calculate the distance between the first curve and the second curve; when the distance between the first curve and the second curve is less than the distance threshold, select the curve corresponding to the larger value of the first probability value and the second probability value as the lane line. The technical solution of the embodiment of the present invention extracts the local features of the lane line in the previous frame image by using the detection result of the previous frame as one of the inputs in the current frame detection model, better handles the problem of the narrow and long structure of the lane line, further corrects the detection, improves the stability and robustness of the lane line detection, has strong interpretability, and the result accuracy of the lane line detection is higher; and the deviation correction model, as a pluggable module, can be used in conjunction with any lane line detection model, has strong versatility, and is more stable and reliable, without the need to perform special parameter adjustment according to a specific lane line detection model. Any lane detection model can be used to detect the lane lines in the current frame image, so that the lane line detection method is highly flexible and the detection effect is relatively stable. The training difficulty of the deviation correction model is low, the volume is small, and the calculation amount is small, which further reduces the time consumption of lane line detection.

[0134] Embodiment 3

[0135] Figure 3A flowchart of a lane line detection method provided in Embodiment 3 of the present invention is provided. This embodiment is a refinement of the deviation correction model construction for correcting the deviation of the second detection result in the above embodiment to obtain the corrected second detection result, such as Figure 3 As shown, the method includes:

[0136] S301. Select several curves from a sample library as sample curves.

[0137] The sample library contains a number of images containing lane lines, and a number of curves representing lane lines are selected from the sample library as sample curves for training the deviation correction model.

[0138] The embodiments of the present invention are only examples and are not limiting.

[0139] S302: uniformly sample the sample curve to obtain a number of sample points.

[0140] The selected sample curves are sampled evenly and equidistantly, and a number of sampling points are obtained on each sample curve.

[0141] Exemplarily, the sample curve is evenly divided in the horizontal direction into a plurality of sample sampling points with the same horizontal coordinate distance between adjacent sample sampling points.

[0142] It should be noted that the uniform sampling method in the embodiment of the present invention is an exemplary description. In other embodiments of the present invention, the sample curve can be evenly divided in the vertical direction into a number of sample sampling points with the same ordinate distance between adjacent sample sampling points, or the sample curve can be evenly divided in the curve direction into a number of sample sampling points with the same length between adjacent sample sampling points. The embodiment of the present invention is only an example and is not limited thereto.

[0143] S303: Calculate the sub-distance between each sample sampling point and the corresponding standard point on the standard curve, where the standard curve is a curve in the standard library.

[0144] The standard library contains several standard curves that calibrate the standard positions of lane lines. The standard curves serve as learning targets for sample curves.

[0145] The standard curve is also uniformly sampled to obtain a number of standard points, and the sub-distance between each sample sampling point and the corresponding standard point on the standard curve is calculated. For example, the coordinates of the sample sampling point are (x qi ,y qi ), where i = 1...M, M is the number of sample points, and the coordinates of the standard point are (x mi ,y mi), where i = 1...M, M is the number of standard points; the sub-distances between the sample sampling points and the standard points with the same subscripts are calculated, and a total of M sub-distances are calculated.

[0146] It should be noted that the embodiment of the present invention does not limit the uniform sampling method.

[0147] S304: Calculate the average value of the sub-distances corresponding to all sample points on the sample curve as the distance between the sample curve and the standard curve.

[0148] The average value of the sub-distances between all sample sampling points and the corresponding standard points on a sample curve is calculated as the distance between the sample curve and the standard curve. Exemplarily, the average value of the sub-distances between M sample sampling points and the corresponding standard points on the sample curve is calculated to obtain the distance between the sample curve and the standard curve.

[0149] The present invention is only for example and not for limitation.

[0150] S305: taking the sample curves whose distance is less than a preset distance threshold as positive samples.

[0151] S306: taking the sample curves whose distance is not less than the preset distance threshold as negative samples.

[0152] The sample curve whose distance between the sample curve and the standard curve is less than the preset distance threshold is taken as a positive sample, and the offset value of each sample sampling point on the current sample curve is calculated, including the lateral offset value and the longitudinal offset value, for subsequent offset value prediction learning; the sample curve whose distance between the sample curve and the standard curve is not less than the preset distance threshold is taken as a negative sample.

[0153] It should be noted that in the embodiment of the present invention, the classification of positive samples and negative samples according to the preset distance threshold is an exemplary description of the embodiment of the present invention. In other embodiments of the present invention, other determination methods can also be used to distinguish positive samples from negative samples. The embodiment of the present invention is only an example and is not explained.

[0154] In some embodiments of the present invention, before S307, the method further includes:

[0155] S3001. Determine whether the ratio of the number of positive samples to the number of negative samples is less than a preset sample ratio threshold; if the ratio of the number of positive samples to the number of negative samples is not less than the preset sample ratio threshold, execute S3002.

[0156] Since the quality of the lane lines in the sample library is relatively high, the number of negative samples may be relatively small. Therefore, it is necessary to determine whether the ratio of the number of positive samples to negative samples meets the training requirements, that is, to determine whether the ratio reaches the preset sample ratio threshold; if the ratio of the number of positive samples to the number of negative samples is not less than the preset sample ratio threshold, it means that the current number of negative samples is small and does not meet the training requirements. At this time, it is necessary to execute S3002 to collect several more negative samples according to the preset rules to balance the ratio of positive samples to negative samples, so as to make the prediction effect of the deviation correction model better.

[0157] For example, the preset sample ratio threshold is positive sample: negative sample = 1:3. When the calculated number of positive samples and the number of negative samples are not less than This means that there are too many positive samples and the ratio is unbalanced. S3002 needs to be executed to randomly collect a number of negative samples until the number of positive samples and the number of negative samples are less than

[0158] It should be noted that the random collection of a number of negative samples in the embodiment of the present invention is for exemplary purposes only. In other embodiments of the present invention, a number of negative samples may be collected according to other rules. The present invention is only for example and not for limitation.

[0159] S3002. Collect several negative samples according to preset rules.

[0160] S307: Use positive samples and negative samples to train a correction model to obtain a deviation correction model.

[0161] The positive samples and negative samples are input into the correction model for model training, wherein the positive samples also include the offset values ​​of each sample sampling point on the sample curve. After training, a deviation correction model is obtained, which can be used to predict the probability value of the curve being a lane line and the offset value of each sampling point on the curve, so as to correct the offset of the curve sampling point.

[0162] It should be noted that the embodiment of the present invention does not limit the structure and form of the deviation correction model.

[0163] The embodiment of the present invention distinguishes positive samples from negative samples, controls the ratio between positive samples and negative samples, and uses positive and negative samples with balanced ratios to train a deviation correction model, thereby improving the correction effect of the deviation correction model and improving the accuracy of lane line detection results. The deviation correction model predicts the offset of the image in the time series based on the input image, and uses the predicted offset to correct the position offset of the detection result in the input image to achieve the effect of time series enhancement, improve the real-time performance of lane line detection, and improve the stability, reliability, and interpretability of the final lane line detection result, with significant improvement effects.

[0164] Embodiment 4

[0165] Figure 4 This is a schematic diagram of the structure of a lane line detection device provided in Embodiment 3 of the present invention. Figure 4 As shown, the device comprises:

[0166] The current frame acquisition module 401 is used to acquire the current frame image collected by the unmanned equipment;

[0167] A first detection result determination module 402, configured to determine a first detection result of the current frame image based on the current frame image, wherein the first detection result includes a first probability value that a first curve in the current frame image is a lane line;

[0168] A second detection result determination module 403 is used to obtain a second detection result of a previous frame image, wherein the second detection result includes a second probability value that a second curve in the previous frame image is a lane line;

[0169] A distance calculation module 404, configured to calculate the distance between the first curve and the second curve;

[0170] The lane line detection module 405 is configured to select a curve corresponding to a larger value between a first probability value and a second probability value as a lane line when the distance between the first curve and the second curve is less than a distance threshold.

[0171] Optionally, the lane line detection device further includes:

[0172] The second detection result correction module corrects the deviation of the second detection result caused by the movement of the vehicle to obtain a corrected second detection result.

[0173] Optional, deviation correction processing submodule, including:

[0174] A downsampling unit, used for downsampling the point sequence to obtain a plurality of sampling points;

[0175] A feature vector extraction unit, used for extracting, for each sampling point, local features of a local area where the sampling point is located from a previous frame of image, to obtain a feature vector composed of local features of a plurality of sampling points;

[0176] a classification and regression unit, configured to perform classification and regression processing on the feature vector, and obtain a probability value that the second curve is a lane line as a corrected second probability value, and an offset value of each of the sampling points;

[0177] The sampling point correction unit is used to correct the position of each sampling point based on the offset value of each sampling point to obtain a corrected second curve.

[0178] Optional, downsampling unit, including:

[0179] A curve fitting subunit, used for performing curve fitting on the point sequence to obtain a fitting curve;

[0180] The uniform sampling subunit is used to uniformly sample the fitting curve to obtain a number of sampling points, including:

[0181] The fitting curve is evenly divided in the horizontal direction into a plurality of adjacent sampling points with the same horizontal coordinate distance between the adjacent sampling points, or,

[0182] The fitting curve is evenly divided in the vertical direction into a plurality of adjacent sampling points with the same ordinate distance between the adjacent sampling points, or

[0183] The fitting curve is evenly divided into a plurality of sampling points with the same length between adjacent sampling points in the curve direction.

[0184] Optional, classification and regression unit, including:

[0185] A fully connected vector acquisition subunit, used for inputting the feature vector into the first fully connected layer to obtain a fully connected vector;

[0186] a classification subunit, configured to input the fully connected vector into an activation function for classification, and obtain a probability value of the second curve being a lane line;

[0187] The regression subunit is used to input the feature vector into the second fully connected layer for regression processing to obtain the offset value of each sampling point on the second curve.

[0188] Optional, sampling point correction unit, including:

[0189] A horizontal coordinate correction module, used for adding the horizontal coordinate of the sampling point to the horizontal offset value in the offset value to obtain the corrected horizontal coordinate of the sampling point;

[0190] The longitudinal coordinate correction module is used to add the longitudinal offset value in the offset value to the longitudinal coordinate of the sampling point to obtain the corrected longitudinal coordinate of the sampling point.

[0191] Optional, bias correction model building submodule, including:

[0192] A sample curve selection unit, used for selecting a number of curves from a sample library as sample curves;

[0193] A sample uniform sampling unit, used for uniformly sampling the sample curve to obtain a plurality of sample sampling points;

[0194] A sub-distance calculation unit, used to calculate the sub-distance between each of the sample sampling points and the corresponding standard point on the standard curve, wherein the standard curve is a curve in the standard library;

[0195] A distance calculation unit, used to calculate an average value of sub-distances corresponding to all the sample sampling points on the sample curve as the distance between the sample curve and the standard curve;

[0196] A positive sample selection unit, configured to select the sample curve whose distance is less than a preset distance threshold as a positive sample;

[0197] A negative sample selection unit, configured to select the sample curve whose distance is not less than the preset distance threshold as a negative sample;

[0198] A model training unit is used to train a correction model using the positive samples and the negative samples to obtain the deviation correction model.

[0199] Optional, bias correction model building submodule, including:

[0200] A sample quantity determination unit is used to determine whether the ratio of the number of positive samples to the number of negative samples is less than a preset sample ratio threshold;

[0201] The negative sample collection unit is used to collect a number of negative samples according to preset rules.

[0202] The lane line detection device provided in the embodiment of the present invention can execute the lane line detection method provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of the execution method.

[0203] Embodiment 5

[0204] Figure 5 A schematic diagram of an unmanned device 10 that can be used to implement an embodiment of the present invention is shown. Unmanned devices are intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. Unmanned devices can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices (such as helmets, glasses, watches, etc.) and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present invention described and / or required herein.

[0205] like Figure 5As shown, the unmanned device 10 includes at least one processor 11, and a memory connected to the at least one processor 11, such as a read-only memory (ROM) 12, a random access memory (RAM) 13, etc., wherein the memory stores a computer program that can be executed by at least one processor, and the processor 11 can perform various appropriate actions and processes according to the computer program stored in the read-only memory (ROM) 12 or the computer program loaded from the storage unit 18 to the random access memory (RAM) 13. In the RAM 13, various programs and data required for the operation of the unmanned device 10 can also be stored. The processor 11, the ROM 12, and the RAM 13 are connected to each other through a bus 14. The input / output (I / O) interface 15 is also connected to the bus 14.

[0206] Multiple components in the unmanned device 10 are connected to the I / O interface 15, including: an input unit 16, such as a keyboard, a mouse, etc.; an output unit 17, such as various types of displays, speakers, etc.; a storage unit 18, such as a disk, an optical disk, etc.; and a communication unit 19, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 19 allows the unmanned device 10 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0207] The processor 11 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any appropriate processor, controller, microcontroller, etc. The processor 11 performs the various methods and processes described above, such as the lane line detection method.

[0208] In some embodiments, the lane detection method may be implemented as a computer program, which is tangibly contained in a computer-readable storage medium, such as a storage unit 18. In some embodiments, part or all of the computer program may be loaded and / or installed on the unmanned device 10 via the ROM 12 and / or the communication unit 19. When the computer program is loaded into the RAM 13 and executed by the processor 11, one or more steps of the lane detection method described above may be performed. Alternatively, in other embodiments, the processor 11 may be configured to perform the lane detection method in any other appropriate manner (e.g., by means of firmware).

[0209] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0210] Computer programs for implementing the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that when the computer program is executed by the processor, the functions / operations specified in the flow chart and / or block diagram are implemented. The computer program may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0211] In the context of the present invention, a computer-readable storage medium may be a tangible medium that may contain or store a computer program for use by or in combination with an instruction execution system, device or equipment. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices or equipment, or any suitable combination of the foregoing. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0212] To provide interaction with a user, the systems and techniques described herein may be implemented on an unmanned device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and a pointing device (e.g., a mouse or trackball) through which the user can provide input to the unmanned device. Other types of devices may also be used to provide interaction with the user; for example, the feedback provided to the user may be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form (including acoustic input, voice input, or tactile input).

[0213] The systems and techniques described herein may be implemented in a computing system that includes backend components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes frontend components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such backend components, middleware components, or frontend components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), a blockchain network, and the Internet.

[0214] A computing system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The client and server relationship is generated by computer programs running on the corresponding computers and having a client-server relationship with each other. The server may be a cloud server, also known as a cloud computing server or cloud host, which is a host product in the cloud computing service system to solve the defects of difficult management and weak business scalability in traditional physical hosts and VPS services.

[0215] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps described in the present invention can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solution of the present invention can be achieved, and this document does not limit this.

[0216] The above specific implementations do not constitute a limitation on the protection scope of the present invention. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A lane line detection method, characterized in that: include: Get the current frame image collected by the unmanned equipment; Determine a first detection result of the current frame image based on the current frame image, the first detection result comprising a first probability value that a first curve in the current frame image is a lane line; Acquire a second detection result of the previous frame image, wherein the second detection result includes a second probability value that a second curve in the previous frame image is a lane line; calculating a distance between the first curve and the second curve; When the distance between the first curve and the second curve is less than a distance threshold, selecting a curve corresponding to a larger value of the first probability value and the second probability value as the lane line; After obtaining the second detection result of the previous frame image, the method further includes: Correcting the deviation of the second detection result to obtain a corrected second detection result; The second detection result also includes a point sequence representing the second curve, and correcting a deviation of the second detection result to obtain a corrected second detection result includes: Downsampling the point sequence to obtain a plurality of sampling points; For each of the sampling points, extract the local features of the local area where the sampling point is located from the previous frame image to obtain a feature vector composed of the local features of several sampling points; Classifying and regressing the feature vector to obtain a probability value that the second curve is a lane line as a corrected second probability value, and an offset value of each of the sampling points; The position of each sampling point is corrected based on the offset value of each sampling point to obtain a corrected second curve.

2. The method according to claim 1, characterized in that The point sequence is downsampled to obtain a number of sampling points, including: Performing curve fitting on the point sequence to obtain a fitting curve; The fitting curve is uniformly sampled to obtain a plurality of sampling points.

3. The method according to claim 2, characterized in that The fitting curve is uniformly sampled to obtain a plurality of sampling points, including: The fitting curve is evenly divided in the horizontal direction into a plurality of adjacent sampling points with the same horizontal coordinate distance between the adjacent sampling points, or, The fitting curve is evenly divided in the vertical direction into a plurality of adjacent sampling points with the same ordinate distance between the adjacent sampling points, or The fitting curve is evenly divided into a plurality of sampling points with the same length between adjacent sampling points in the curve direction.

4. The method according to claim 2, characterized in that: The classifying and regressing the feature vector to obtain a probability value of the second curve being a lane line as a corrected second probability value and an offset value of each sampling point includes: Inputting the feature vector into a first fully connected layer to obtain a fully connected vector; Inputting the fully connected vector into an activation function for classification to obtain a probability value of the second curve being a lane line; The feature vector is input into the second fully connected layer for regression processing to obtain the offset value of each sampling point on the second curve.

5. The method according to claim 4, characterized in that The first fully connected layer and the second fully connected layer are the same fully connected layer.

6. The method according to claim 1, characterized in that The step of correcting the position of each sampling point based on the offset value of each sampling point to obtain a corrected second curve includes: Adding the horizontal coordinate of the sampling point to the horizontal offset value in the offset value to obtain a corrected horizontal coordinate of the sampling point; The longitudinal coordinate of the sampling point is added to the longitudinal offset value in the offset value to obtain a corrected longitudinal coordinate of the sampling point.

7. The method according to any one of claims 1 to 6, characterized in that: The deviation correction model construction method for correcting the deviation of the second detection result to obtain a corrected second detection result includes: Selecting several curves from the sample library as sample curves; Uniformly sampling the sample curve to obtain a number of sample points; Calculating the sub-distance between each of the sample sampling points and the corresponding standard point on the standard curve, wherein the standard curve is a curve in the standard library; Calculating an average value of the sub-distances corresponding to all the sample sampling points on the sample curve as the distance between the sample curve and the standard curve; The sample curve whose distance is less than a preset distance threshold is taken as a positive sample; Taking the sample curve whose distance is not less than the preset distance threshold as a negative sample; The positive samples and the negative samples are used to train a correction model to obtain the deviation correction model.

8. The method according to claim 7, characterized in that Before training the positive sample and the negative sample to match the sub-model to obtain the deviation correction model, the method further includes: Determine whether the ratio of the number of positive samples to the number of negative samples is less than a preset sample ratio threshold; If the ratio of the number of positive samples to the number of negative samples is not less than a preset sample ratio threshold, a number of negative samples are collected according to a preset rule.

9. A lane line detection device, characterized in that: include: The current frame acquisition module is used to obtain the current frame image collected by the unmanned equipment; A first detection result determination module, configured to determine a first detection result of the current frame image based on the current frame image, wherein the first detection result includes a first probability value that a first curve in the current frame image is a lane line; A second detection result determination module, used to obtain a second detection result of a previous frame image, wherein the second detection result includes a second probability value that a second curve in the previous frame image is a lane line; A distance calculation module, used for calculating the distance between the first curve and the second curve; A lane line detection module, configured to select a curve corresponding to a larger value of a first probability value and a second probability value as a lane line when the distance between the first curve and the second curve is less than a distance threshold; The lane line detection device further includes: A second detection result correction module, which corrects the deviation of the second detection result to obtain a corrected second detection result; The second detection result also includes a point sequence representing the second curve, and the second detection result correction module includes: A downsampling unit, used for downsampling the point sequence to obtain a plurality of sampling points; A feature vector extraction unit, used for extracting, for each sampling point, a local feature of a local area where the sampling point is located from a previous frame image, to obtain a feature vector composed of the local features of a plurality of sampling points; a classification and regression unit, configured to perform classification and regression processing on the feature vector, and obtain a probability value that the second curve is a lane line as a corrected second probability value, and an offset value of each of the sampling points; The sampling point correction unit is used to correct the position of each sampling point based on the offset value of each sampling point to obtain a corrected second curve.

10. An unmanned device, characterized in that: The unmanned equipment includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein, The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the lane line detection method described in any one of claims 1-8.

11. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the lane line detection method according to any one of claims 1 to 8 when executed.

Images