Lane detection method, device, computer equipment and storage medium

By using lightweight neural networks and traditional detection methods on vehicle terminals, combined with deep learning and traditional methods, the applicability problem of the deep learning lane line detection method in case of limited computing capabilities is solved, and the accuracy on vehicle terminals is achieved Lane line detection.

CN112580565BActive Publication Date: 2024-11-12GUANGZHOU YAMEI ZHIZAO TECH CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202011566863.7
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2020-12-25
Publication Date
2024-11-12
Estimated Expiration
2040-12-25

AI Technical Summary

Technical Problem

The lane line detection method based on deep learning cannot be applied to on-board terminals with limited computing capabilities due to the large number of parameters and low detection speed of neural networks.

Method used

Lightweight neural networks are used to combine deep learning and traditional detection methods to obtain original lane line points by taking lane line images and inputting lightweight neural networks, determine the search interval, search and relocate lane line points, and finally obtain the lane line in the lane line image. .

Benefits of technology

Accurate lane line detection is achieved on on-board terminals with limited computing capabilities, and combined with deep learning and traditional methods to improve detection speed and accuracy.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN112580565B_ABST
    Figure CN112580565B_ABST
Patent Text Reader

Abstract

The present application relates to a lane line detection method, device, computer equipment and storage medium. The method comprises: photographing a lane line image; inputting the lane line image into a preset lightweight neural network to obtain the original lane line points output by the lightweight neural network for the lane line image; determining a search interval on the lane line image based on the original lane line points; the search interval comprises pixel points; searching for relocated lane line points of the original lane line points in the search interval based on the pixel values ​​of each pixel point in the search interval; obtaining lane lines in the lane line image based on the relocated lane line points. The use of this method enables a vehicle-mounted terminal with limited computing power to perform lane line detection using a lane line detection method based on deep learning.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the field of deep learning technology, and in particular to a lane line detection method, apparatus, computer equipment, storage medium and computer program product. Background Art

[0002] Assisted autonomous driving technology is becoming more and more popular among users. The realization of assisted autonomous driving depends on the accurate detection of lane lines. At present, lane line detection methods mainly include detection methods based on traditional technology and detection methods based on deep learning. Compared with detection methods based on traditional technology, detection methods based on deep learning have higher accuracy.

[0003] However, the detection method based on deep learning can only be realized through a neural network composed of a large number of parameters. Due to the large number of parameters and low detection speed of the neural network, the detection method based on deep learning cannot be applied to vehicle terminals with limited computing power.

[0004] Therefore, the current lane line detection method based on deep learning has the problem of not being applicable to vehicle-mounted terminals with limited computing power. Summary of the invention

[0005] Based on this, it is necessary to provide a lane line detection method, device, computer equipment, storage medium and computer program product that can be applicable to vehicle-mounted terminals to address the above technical problems.

[0006] A lane line detection method, the method comprising:

[0007] Capture lane line images;

[0008] Inputting the lane line image into a preset lightweight neural network to obtain original lane line points output by the lightweight neural network for the lane line image;

[0009] Determine a search interval on the lane line image according to the original lane line point; the search interval includes pixel points;

[0010] Searching for a relocated lane line point of the original lane line point in the search interval according to a pixel value of each pixel point in the search interval;

[0011] According to the relocated lane line points, the lane lines in the lane line image are obtained.

[0012] In one embodiment, the original lane line point has a corresponding fitted lane line point, and obtaining the lane line in the lane line image according to the relocated lane line point includes:

[0013] Clustering the fitted lane line points and the relocated lane line points to obtain a plurality of lane line point classes;

[0014] Determine the number of line points of each lane line point class, and determine the shape characteristics of each lane line point class;

[0015] Determine a target lane line point class from the plurality of lane line point classes according to the line point quantity feature and the shape feature;

[0016] According to the target lane line point class, a lane line in the lane line image is obtained.

[0017] In one embodiment, searching for the relocated lane line point of the original lane line point in the search interval according to the pixel value of each pixel point in the search interval includes:

[0018] In the search interval, determine the pixel points whose pixel values ​​are greater than a preset pixel value threshold to obtain positioning reference pixels;

[0019] Counting the sum of pixel coordinates of the positioning reference pixels to obtain a pixel coordinate sum, and counting the number of the positioning reference pixels to obtain a pixel point sum;

[0020] Calculating the ratio of the sum of the pixel coordinates to the sum of the pixel points to obtain the relocated pixel coordinates;

[0021] A pixel point at the relocated pixel coordinates is determined as the relocated lane line point.

[0022] In one embodiment, determining the line point quantity feature of each lane line point class includes:

[0023] Counting the total number of lane line points in the lane line point class to obtain the lane point quantity feature;

[0024] The determining of the shape features of each lane line point class includes:

[0025] Determine a lane shape formed by lane line points in the lane line point class;

[0026] Calculating the aspect ratio of the lane shape to obtain the shape feature;

[0027] The step of determining a target lane line point class from among the plurality of lane line point classes according to the line point quantity feature and the shape feature comprises:

[0028] Determine a lane line point class whose aspect ratio of the lane shape is less than a preset aspect ratio threshold, and obtain a plurality of candidate lane line point classes;

[0029] Among the plurality of candidate lane line point classes, a candidate lane line point class having the largest total number of lane line points in the lane line point class is determined as the target lane line point class.

[0030] In one embodiment, the original lane line points include left lane line points and right lane line points, and determining the search interval on the lane line image according to the original lane line points includes:

[0031] Among the left lane line points, screened left lane line points are selected; the screened left lane line points include a first left lane line point and a second left lane line point; the pixel ordinate of the first left lane line point is greater than the pixel ordinate of the second left lane line point; the pixel abscissa of the first left lane line point is greater than the pixel abscissa of the second left lane line point;

[0032] Filter out filtered right lane line points from the right lane line points; the filtered right lane line points include a first right lane line point and a second right lane line point; the pixel ordinate of the first right lane line point is greater than the pixel ordinate of the second right lane line point; the pixel abscissa of the first right lane line point is less than the pixel abscissa of the second right lane line point;

[0033] The filtered left lane line point and the filtered right lane line point are used as filtered lane line points;

[0034] A search interval on the lane line image is determined according to the filtered lane line points.

[0035] In one embodiment, determining the search interval on the lane line image according to the filtered lane line points includes:

[0036] Get multiple historical cache lane line points;

[0037] Counting the errors between the pixel horizontal coordinates of the filtered lane line points and the pixel horizontal coordinates of each of the historical cached lane line points to obtain line point coordinate errors;

[0038] When the line point coordinate error is less than a preset error threshold, caching the filtered lane line point;

[0039] A search interval on the lane line image is determined based on the plurality of historical cached lane line points and the cached filtered lane line points.

[0040] In one embodiment, determining the search interval on the lane line image according to the plurality of historical cached lane line points and the filtered lane line points includes:

[0041] Performing curve fitting on the plurality of historical cache lane line points and the filtered lane line points to obtain fitted lane line points;

[0042] Generate the search interval; the search interval is an area on the lane line image that is centered on the fitted lane line point and meets preset pixel coordinate conditions.

[0043] In one embodiment, it also includes:

[0044] When the line point coordinate error is greater than a preset error threshold, a search interval on the lane line image is determined according to a plurality of the historical cached lane line points.

[0045] In one embodiment, the plurality of historical cached lane line points respectively have corresponding historical lane line images, and the historical lane line images have a cache time. When the line point coordinate error is less than a preset error threshold, after caching the filtered lane line point, the method further includes:

[0046] Counting the number of current lane line images; the number of current lane line images is the sum of the number of the multiple historical lane line images and the number of the lane line images;

[0047] When the number of the current lane line images exceeds a preset number threshold, determining the historical lane line image with the earliest cache time;

[0048] Among the multiple historical cached lane line points, the historical cached lane line point corresponding to the historical lane line image with the earliest cache time is cleared.

[0049] In one embodiment, when the number of the current lane line images is less than a preset number threshold, the method further includes:

[0050] Take an initialization image;

[0051] Inputting the initialization image into the lightweight neural network to obtain the initialization lane line points output by the lightweight neural network for the initialization image;

[0052] Performing straight line fitting on the initialized lane line points to obtain an initialized left lane line and an initialized right lane line;

[0053] Determine a lane line intersection point of the initialized left lane line and the initialized right lane line;

[0054] When the lane line intersection point does not match the vanishing point of the initialization image, returning to the step of capturing the initialization image;

[0055] When the lane line intersection point matches the vanishing point of the initialization image, the initialization lane line point is cached as the historical cache lane line point, and the process returns to the step of capturing the lane line image.

[0056] A lane line detection device, the device comprising:

[0057] An image shooting module, used for shooting lane line images;

[0058] A network input module, used to input the lane line image into a preset lightweight neural network to obtain original lane line points output by the lightweight neural network for the lane line image;

[0059] An interval determination module, used to determine a search interval on the lane line image according to the original lane line point; the search interval includes pixel points;

[0060] A search module, configured to search for a relocated lane line point of the original lane line point in the search interval according to a pixel value of each pixel point in the search interval;

[0061] A lane line module is used to obtain the lane line in the lane line image according to the relocated lane line point.

[0062] A computer device comprises a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the following steps are implemented:

[0063] Capture lane line images;

[0064] Inputting the lane line image into a preset lightweight neural network to obtain original lane line points output by the lightweight neural network for the lane line image;

[0065] Determine a search interval on the lane line image according to the original lane line point; the search interval includes pixel points;

[0066] Searching for a relocated lane line point of the original lane line point in the search interval according to a pixel value of each pixel point in the search interval;

[0067] According to the relocated lane line points, the lane lines in the lane line image are obtained.

[0068] A computer-readable storage medium stores a computer program, which, when executed by a processor, implements the following steps:

[0069] Capture lane line images;

[0070] Inputting the lane line image into a preset lightweight neural network to obtain original lane line points output by the lightweight neural network for the lane line image;

[0071] Determine a search interval on the lane line image according to the original lane line point; the search interval includes pixel points;

[0072] Searching for a relocated lane line point of the original lane line point in the search interval according to a pixel value of each pixel point in the search interval;

[0073] According to the relocated lane line points, the lane lines in the lane line image are obtained.

[0074] A computer program product, comprising a computer program, wherein when the computer program is executed by a processor, the following steps are implemented:

[0075] Capture lane line images;

[0076] Inputting the lane line image into a preset lightweight neural network to obtain original lane line points output by the lightweight neural network for the lane line image;

[0077] Determine a search interval on the lane line image according to the original lane line point; the search interval includes pixel points;

[0078] Searching for a relocated lane line point of the original lane line point in the search interval according to a pixel value of each pixel point in the search interval;

[0079] According to the relocated lane line points, the lane lines in the lane line image are obtained.

[0080] The lane line detection method, apparatus, computer equipment, storage medium and computer program product described above first input the lane line image into a lightweight neural network to obtain the original lane line points output by the lightweight neural network for the lane line image, and then determine the search interval of the lane line image based on the original lane line points, search for the relocated lane line points in the search interval according to the pixel values ​​of each pixel point in the search interval, and finally use the relocated lane line points to obtain the lane lines in the lane line image. Thus, through the combination of deep learning detection methods and traditional detection methods, even if a lightweight neural network suitable for vehicle-mounted terminals is used for lane line detection, the accuracy of lane line detection can still be guaranteed, so that vehicle-mounted terminals with limited computing power can also use the lane line detection method based on deep learning for lane line detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0081] Figure 1 A diagram showing an application environment of a lane line detection method in an embodiment;

[0082] Figure 2 1 is a flow chart of a lane line detection method in one embodiment;

[0083] Figure 3 is a schematic diagram of a process of performing lane line detection based on a pruned UFLD network according to an embodiment;

[0084] Figure 4 is a schematic diagram of outputting a final lane line according to an embodiment;

[0085] Figure 5 is a structural block diagram of a lane line detection device in an embodiment;

[0086] Figure 6 FIG. 4 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0087] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0088] The lane line detection method provided in this application can be applied to Figure 1 In the application environment shown, the vehicle terminal 102 may be a terminal on the vehicle 104 for controlling the vehicle camera 106 to capture a lane line image in front of the vehicle, identify lane lines in the lane line image, and perform assisted driving according to the identified lane lines.

[0089] In one embodiment, Figure 2 As shown, a lane line detection method is provided, which is applied to Figure 1 Taking the vehicle-mounted terminal 102 in the example as an example, the method includes the following steps:

[0090] Step 202: capture a lane line image.

[0091] The lane line image may be an image including lane lines captured by the on-board camera 106 of the vehicle 104 of the road in front of the vehicle.

[0092] Specifically, the user can submit an assisted driving instruction through the vehicle terminal 102, and the vehicle terminal 102 can control the vehicle camera 106 to start the shooting function according to the assisted driving instruction to obtain the above-mentioned lane line image.

[0093] Step 204: input the lane line image into a preset lightweight neural network to obtain original lane line points output by the lightweight neural network for the lane line image.

[0094] The lightweight neural network may be a pruned neural network adapted to the computing power of the vehicle-mounted terminal 102. The lightweight neural network has the characteristics of less parameters and less required computing power.

[0095] The original lane line point may be a lane line point output by a lightweight neural network after recognizing a lane line image. In practical applications, the lane line point may be expressed by pixel coordinates. For example, the original lane line point output by the lightweight neural network is Lp1(X1, Y1).

[0096] Specifically, the user may pre-load the lightweight neural network into the vehicle-mounted terminal 102. The user may perform tailoring on the basis of the existing neural network to form the above-mentioned lightweight neural network.

[0097] In practical applications, users can use the neural network constructed by the method of directly predicting lane lines based on regression (Ultra Fast Structure-aware Deep Lane Detection, UFLD) (hereinafter referred to as UFLD neural network) as the basis for trimming to obtain a lightweight neural network.

[0098] It should be noted that compared with the neural network based on the semantic segmentation method, the UFLD neural network directly regresses some lane line points on the lane line, thereby avoiding a large number of operations such as classifying each pixel in the image in the semantic segmentation method. It has greatly improved the detection speed and the number of model parameters, but it still cannot be applied to terminals with limited computing power.

[0099] Therefore, in order to make the UFLD neural network suitable for the vehicle terminal 102, the user can tailor the image resolution, row anchor, gridding_num (convolution kernel degradation amount), backbone (front-end feature extraction network) and other aspects of the input lane line image to obtain the above-mentioned lightweight neural network. For example, the front-end feature extraction network of the current UFLD neural network mainly adopts the resnet18 network (a convolutional neural network framework), and the user can modify it to the resnet10 network; for another example, the fully connected layers (fully connected layers) of the regression output lane line are removed, and the front-end feature extraction network is directly connected to the output layer (output layer).

[0100] Of course, those skilled in the art can perform trimming based on other neural networks. The above-mentioned trimming method of the UFLD neural network is only for illustration and does not limit the lightweight neural network of the present application.

[0101] After capturing the lane line image, the vehicle terminal 102 may first input the lane line image into the lightweight neural network, and the lightweight neural network outputs a plurality of lane line points based on the image features in the lane line image. For the purpose of distinguishing the description, the lane line points output by the lightweight neural network are named as original lane line points.

[0102] Step 206: Determine a search interval on the lane line image based on the original lane line points; the search interval includes pixel points.

[0103] The search interval may be an area in the lane line image used to search for real lane line points.

[0104] First of all, it should be noted that the pruned lightweight neural network has the problem of unstable lane line detection results. More specifically, since the lightweight neural network is compressed and the number of parameters is greatly reduced, when the lightweight neural network performs lane line recognition on the lane line image, the complex light in the environment may interfere with the recognition, causing the lightweight neural network to recognize the pixels with high contrast near the real lane line points as lane line points, causing the detected lane line points to deviate from the real lane line points.

[0105] To address the above problem, the vehicle-mounted terminal 102 may reposition the original lane line points based on the original lane line points output by the lightweight neural network.

[0106] Specifically, the search interval in the lane line image can be determined first according to the original lane line point, and there are many ways to determine the search interval. For example, the original lane line point can be used as the midpoint, and a certain interval around the midpoint (for example, plus or minus 10 pixels) can be used as the above search interval. The original lane line point can also be used as the center, and a certain radius interval around the center (for example, with a radius of 10 pixels) can be used as the above search interval.

[0107] When the original lane line point does not match the actual lane line point, the actual lane line point is usually near the original lane line point. Therefore, by determining the search interval based on the original lane line point, the actual lane line point can be quickly searched out.

[0108] Step 208: searching for the relocated lane line point of the original lane line point in the search interval according to the pixel value of each pixel point in the search interval.

[0109] The relocated lane line point may be a lane line point searched in the search interval and obtained after relocation.

[0110] Specifically, the vehicle-mounted terminal 102 may first determine a plurality of pixel points whose pixel values ​​are greater than a preset pixel value threshold in the search interval. Then, the sum of the pixel coordinates of the plurality of pixel points is counted to obtain the sum of the pixel coordinates. In addition, the number of the plurality of pixel points may be counted to obtain the sum of the pixel points. Then, the ratio of the sum of the pixel coordinates to the sum of the pixel points is calculated as the relocated pixel coordinates. Finally, the pixel points on the lane line image at the relocated pixel coordinates are determined as the above-mentioned relocated lane line points. The vehicle-mounted terminal 102 may also first determine the pixel points whose pixel values ​​are greater than the pixel values ​​of the original lane line points in the search interval, and then count the sum of the pixel coordinates of the plurality of pixel points to obtain the sum of the pixel coordinates. In addition, the number of the plurality of pixel points may be counted to obtain the sum of the pixel points. Then, the ratio of the sum of the pixel coordinates to the sum of the pixel points is calculated as the relocated pixel coordinates.

[0111] Based on the characteristic that pixel points with pixel values ​​within a certain range in the search interval are associated with the real lane line points, technicians in this field can search for the relocated lane line points of the original lane line points in the search interval in a variety of ways to relocate the lane line points, and ultimately detect lane lines that are close to the real lane lines.

[0112] Step 210: Obtain the lane line in the lane line image according to the relocated lane line point.

[0113] Specifically, after obtaining a plurality of relocated lane line points, a lane line composed of the plurality of relocated lane line points can be obtained and displayed to the user through the vehicle-mounted terminal 102. There are many ways to generate the final lane line according to the relocated lane line points. For example, the plurality of relocated lane line points can be fitted into the lane line. Those skilled in the art can adopt various specific implementation methods according to actual needs.

[0114] In the above lane line detection method, the lane line image is first input into a lightweight neural network to obtain the original lane line points output by the lightweight neural network for the lane line image, and then the search interval of the lane line image is determined based on the original lane line points, and the relocated lane line points are searched out in the search interval according to the pixel values ​​of each pixel point in the search interval, and finally the lane lines in the lane line image are obtained by using the relocated lane line points. Therefore, through the combination of deep learning detection methods and traditional detection methods, even if a lightweight neural network suitable for vehicle-mounted terminals is used for lane line detection, the accuracy of lane line detection can still be guaranteed, so that vehicle-mounted terminals with limited computing power can also use the lane line detection method based on deep learning for lane line detection.

[0115] In one embodiment, the original lane line point has a corresponding fitted lane line point, and step 210 may specifically include:

[0116] The fitted lane line points and the relocated lane line points are clustered to obtain a plurality of lane line point classes; a line point quantity feature of each lane line point class is determined, and a shape feature of each lane line point class is determined; a target lane line point class is determined from the plurality of lane line point classes based on the line point quantity feature and the shape feature; and a lane line in the lane line image is obtained based on the target lane line point class.

[0117] The fitted lane line points may be lane line points obtained by fitting the original lane line points.

[0118] The lane line point class may be a set of lane line points with specific features.

[0119] The lane point quantity feature may be a feature reflecting the number of lane line points included in the lane line point class.

[0120] The shape feature may be a feature reflecting the shape formed by the lane line points included in the lane line point class.

[0121] First of all, it should be noted that due to the complex driving environment of the vehicle, the lane line points output by the lightweight neural network may actually be the guide line in the road, the white line on the road, or the hood of the car, etc., which affects the accuracy of lane line detection. To address this problem, the present application can improve the lane line points by clustering them.

[0122] Specifically, a lightweight neural network can be used to fit the original lane line points output by the continuous multiple frames of lane line images captured to obtain fitted lane line points. Then, a clustering method such as k-means (K-means clustering algorithm) is used to cluster the fitted lane line points and the relocated lane line points to form multiple lane line point sets, each of which is a lane line point class.

[0123] For each lane line point class, determine its line point quantity feature. For example, the number of lane line points contained in each lane line point class can be counted as the line point quantity feature; the number of lane line points with relatively dense distribution in each lane line point class can also be counted as the line point quantity feature.

[0124] For each lane point class, its shape feature is determined. For example, the shape formed by the lane points contained in the lane point class can be determined first, and then the aspect ratio of the shape can be calculated as the shape feature; or the lane point with the largest and smallest horizontal and vertical coordinates in the lane point class can be determined first, and the distance between the pixel coordinates of the lane point can be used as the shape feature.

[0125] Then, the lane line point class whose line point quantity characteristics and shape characteristics meet specific conditions can be used as the above-mentioned target lane line point class. According to the lane line points contained in the target lane line point class, the final lane line can be further obtained. For example, the lane line point class with the largest number of line points and a width-to-height ratio less than a preset threshold can be used as the target lane line point class by taking advantage of the fact that the lane line has a large number of lane line points and a small width-to-height ratio.

[0126] Finally, according to the lane line points in the target lane line point class, the least squares method can be used to perform quadratic curve fitting, and the fitting result is used as the lane line in the lane line image.

[0127] The lane line detection method described above clusters the lane line points obtained after fitting the original lane line points and the relocated lane line points, and divides the lane line points with similar characteristics into multiple lane line point classes. Then, based on the number characteristics and shape characteristics of the lane line points of each lane line point class, a target lane line point class that can reflect the real lane line is determined. Therefore, even in a complex driving environment, the lane line detection result of the lightweight neural network can be improved.

[0128] In one embodiment, step 208 may specifically include:

[0129] In the search interval, pixel points whose pixel values ​​are greater than a preset pixel value threshold are determined to obtain positioning reference pixels; the sum of the pixel coordinates of the positioning reference pixels is counted to obtain the total pixel coordinates, and the number of the positioning reference pixels is counted to obtain the total pixel points; the ratio of the total pixel coordinates to the total pixel points is calculated to obtain the repositioning pixel coordinates; and the pixel point at the repositioning pixel coordinates is determined as the repositioning lane line point.

[0130] In practical applications, corresponding search methods can be used for lane line images in different color modes. For example, for lane line images in RGB mode, assuming that the left lane line l leftTake an original lane line point as p(x, y), and traverse all pixels in the search interval [(x-10, y), (x+10, y)] with it as the center and ±10 pixels to the left and right. If the pixel value of a certain pixel satisfies the conditions of R(x', y')>200, G(x', y')>200, and B(x', y')>200 on the three color channels of R, G, and B, then record the pixel as the above-mentioned positioning reference pixel. Count the number of recorded positioning reference pixels to obtain the above-mentioned sum of pixel points. In addition, the pixel coordinates of all positioning reference pixels can be accumulated to obtain the above-mentioned sum of pixel coordinates. Finally, divide the sum of pixel coordinates by the sum of pixel points to obtain the average, and obtain a new coordinate value (x`, y`). According to the coordinate value, the pixel point is located, and the relocated lane line point p`(x`, y`) of the original lane line point p(x, y) can be obtained.

[0131] The above lane line detection method averages the pixel coordinates of multiple pixel points whose pixel values ​​are greater than a preset threshold with the total number of the multiple pixel points. Thus, the relocated lane line points that can be obtained through simpler calculation processing are closer to the real lane line points, thereby improving the detection speed while ensuring the quality of lane line detection.

[0132] In one embodiment, determining the line point quantity feature of each lane line point class includes:

[0133] Counting the total number of lane line points in the lane line point class to obtain the lane point quantity feature;

[0134] The determining of the shape features of each lane line point class includes:

[0135] Determine a lane shape formed by lane line points in the lane line point class; calculate the aspect ratio of the lane shape to obtain the shape feature;

[0136] The step of determining a target lane line point class from among the plurality of lane line point classes according to the line point quantity feature and the shape feature comprises:

[0137] Determine a lane line point class whose aspect ratio of the lane shape is less than a preset aspect ratio threshold, and obtain multiple candidate lane line point classes; among the multiple candidate lane line point classes, determine a candidate lane line point class with the largest total number of lane line points in the lane line point class as the target lane line point class.

[0138] Specifically, the total number of lane line points in the lane line point class can be counted as the above-mentioned lane line point quantity feature. In addition, the lane shape formed by each lane line point in the lane line point class can be determined, and then the aspect ratio of the lane shape can be calculated to obtain the above-mentioned shape feature. Then, from multiple lane line point classes, a lane line point class whose lane shape has a width and height less than a preset threshold can be selected, and from the selected classes, the lane line point class with the largest total number of lane line points can be further determined as the above-mentioned target lane line point class.

[0139] The above lane line detection method selects, from multiple lane line point classes, a lane line point class whose lane shape aspect ratio is less than a preset aspect ratio threshold and whose total number of lane line points is the largest as the target lane line point class. Thus, lane line points that can accurately reflect the real lane line can be obtained with less computational effort, thereby improving the speed of lane line detection.

[0140] In one embodiment, the original lane line points include left lane line points and right lane line points. Step 206 may specifically include:

[0141] Among the left lane line points, the filtered left lane line points are screened out; the filtered left lane line points include a first left lane line point and a second left lane line point; the pixel ordinate of the first left lane line point is greater than the pixel ordinate of the second left lane line point; the pixel abscissa of the first left lane line point is greater than the pixel abscissa of the second left lane line point; among the right lane line points, the filtered right lane line points are screened out; the filtered right lane line points include a first right lane line point and a second right lane line point; the pixel ordinate of the first right lane line point is greater than the pixel ordinate of the second right lane line point; the pixel abscissa of the first right lane line point is less than the pixel abscissa of the second right lane line point; the filtered left lane line points and the filtered right lane line points are used as filtered lane line points; and the search interval on the lane line image is determined according to the filtered lane line points.

[0142] First of all, it should be noted that the lane line points output by the pruned lightweight neural network may be offset, that is, deviate from the real lane line points. To address this problem, the present application can improve it by screening the original lane line points.

[0143] It should be further explained that, for the left lane line points, as the ordinate increases, the pixel abscissa should also increase. Conversely, for the right lane line points, as the ordinate increases, the pixel abscissa should decrease. Therefore, by using the change characteristics of the ordinate and abscissa of the left and right lane lines, the lane line points that meet the characteristics can be quickly screened out from the original lane line points, thereby filtering out the lane line points that are offset by the output of the lightweight neural network.

[0144] Specifically, the lane line points belonging to the left lane line or the right lane line in the original lane line points can be distinguished first, and for the purpose of distinguishing the descriptions, they are named as the left lane line points and the right lane line points, respectively.

[0145] For the left lane line points, the pixel ordinate and pixel abscissa of any left lane line point can be first determined, and then compared with the pixel ordinate and pixel abscissa of another left lane line point. When the pixel ordinate and pixel abscissa are both greater than the other left lane line point, the two line points can be screened out as the first left lane line point and the second left lane line point mentioned above.

[0146] For the right lane line point, the pixel ordinate and pixel abscissa of any right lane line point can be first determined, and then compared with the pixel ordinate and pixel abscissa of another right lane line point. When the pixel ordinate is greater than that of the other right lane line point and the pixel abscissa is less than that of the other right lane line point, the two line points can be screened out as the first right lane line point and the second right lane line point mentioned above.

[0147] Of course, in practical applications, those skilled in the art can use the change relationship between the pixel coordinates of the left and right lane line points to obtain the above-mentioned filtered left lane line points and filtered right lane line points in other ways. For example, for the right lane line points, multiple right lane line points whose pixel vertical coordinates increase successively and whose pixel horizontal coordinates decrease successively can be selected as the above-mentioned filtered right lane line points.

[0148] The filtered left lane line point and the filtered right lane line point are used as filtered lane line points. Finally, the search interval on the lane line image is determined according to the filtered lane lines.

[0149] The above lane line detection method uses the change relationship between the left and right lane line points in pixel coordinates to screen the original lane line points, thereby filtering out the original lane line points that are offset in the lightweight neural network, and further determines the search interval on the lane line image based on the screened lane line points, avoiding repositioning in the wrong search interval, thereby improving the lane line detection results of the lightweight neural network.

[0150] In one embodiment, determining a search interval on the lane line image according to the filtered lane line points includes:

[0151] Acquire multiple historical cached lane line points; count the errors between the pixel horizontal coordinates of the filtered lane line points and the pixel horizontal coordinates of each of the historical cached lane line points to obtain a lane point coordinate error; cache the filtered lane line points when the lane point coordinate error is less than a preset error threshold; determine a search interval on the lane line image based on the multiple historical cached lane line points and the cached filtered lane line points.

[0152] The historical cached lane line points may be filtered lane line points in the cache area of ​​the vehicle-mounted terminal 102 .

[0153] Specifically, the vehicle-mounted terminal 102 can continuously capture multiple frames of lane line images. For each frame of lane line images, the lightweight neural network will generate corresponding original lane line points. After obtaining the filtered lane line points for the original lane line points of each frame of lane line images, the filtered lane line points corresponding to each frame of lane line images can be cached, thereby obtaining historical cached lane line points of multiple frames of lane line images.

[0154] For the currently obtained filtered lane line point, its pixel horizontal coordinate can be calculated with the multiple historical cache lane line points to obtain the line point coordinate error. For example, the mean square error between the pixel horizontal coordinate of the filtered lane line point and the pixel horizontal coordinate of the multiple historical cache lane line points can be calculated by the mean square error method as the above line point coordinate error.

[0155] In practical applications, the line point coordinate error can be calculated using the following formula:

[0156]

[0157] Among them, err represents the line point coordinate error; Represents the average of the pixel horizontal coordinates of the historical cache lane line points corresponding to multiple lane line images; x i It is the pixel horizontal coordinate of the i-th filtered lane line point in the lane line image of the current frame.

[0158] If the line point coordinate error is less than a preset error threshold, it indicates that the currently obtained filtered lane line point is accurate. Therefore, the filtered lane line point is cached, and then the search interval is determined using the cached filtered lane line point and multiple historical cached lane line points.

[0159] For example, assuming that the preset error threshold is 500 pixels, when the line point coordinate error err is less than 500 pixels, it means that the difference between multiple historical cached lane line points and the filtered lane line points is small, and the possibility of erroneous detection results is low. Therefore, the filtered lane line points can be cached, and the currently cached historical cached lane line points and the filtered lane line points can be used to determine the search interval on the lane line image.

[0160] It should be noted that after the filtered lane line points are cached, the cached filtered lane line points become new historical cached lane line points. In subsequent processing, after a new lane line image is captured, the filtered lane line points obtained based on the new lane line image will be repeated through the above steps to obtain the latest lane line.

[0161] The above lane line detection method utilizes the error between the historical cache lane line points and the filtered lane line points in the horizontal coordinate of the pixel, caches the filtered lane line points when the error is small, and further utilizes the cached filtered lane line points and multiple historical cache lane line points to determine the search interval, thereby avoiding determining the search interval when there is a large deviation between the historical cache lane line points and the filtered lane line points, thereby improving the accuracy of lane line detection.

[0162] In one embodiment, determining the search interval on the lane line image according to the plurality of historical cached lane line points and the filtered lane line points includes:

[0163] Perform curve fitting on the multiple historical cache lane line points and the filtered lane line points to obtain fitted lane line points; generate the search interval; the search interval is an area on the lane line image centered on the fitted lane line points and meeting preset pixel coordinate conditions.

[0164] First of all, it should be noted that the number of original lane line points output by the lightweight neural network is limited. The historical cache lane line points and the filtered lane line points obtained based on the original lane line points are discrete, resulting in the final lane line being incoherent, affecting the user's judgment of the lane line. To address this problem, the present application can improve it by fitting the discrete historical cache lane line points and the filtered lane line points.

[0165] Specifically, before determining the search interval, the least square method can be used to perform quadratic curve fitting on multiple historical cache lane line points and filtered lane line points to obtain multiple relatively coherent lane line points as the above-mentioned fitted lane line points. Then, the area on the lane line image centered on each fitted lane line point and meeting the preset pixel coordinate conditions is used as the above-mentioned search interval, so that coherent relocated lane line points can be further obtained.

[0166] The lane line detection method described above obtains a plurality of coherent fitted lane line points by fitting multiple historical cached lane line points and filtered lane line points, generates a search interval based on the multiple coherent fitted lane line points, obtains relocated lane line points based on the search interval, and obtains a final relatively coherent lane line, thereby avoiding the problem of incoherent lane lines due to the limited number of original lane line points output by the lightweight neural network.

[0167] In one embodiment, the lane line detection method may further include:

[0168] When the line point coordinate error is greater than a preset error threshold, a search interval on the lane line image is determined according to a plurality of the historical cached lane line points.

[0169] Specifically, when the line point coordinate error is greater than the preset error threshold, it indicates that there is a large deviation between the filtered lane line points obtained based on the current lane line image and the historical cached lane line points obtained based on the historical lane line image. If the subsequent lane line detection process continues according to the filtered lane line points, it may cause large fluctuations in the lane line detection results. Therefore, when the line point coordinate error is greater than the preset error threshold, the search interval on the lane line image can be determined by directly using multiple historical cached lane line points for further subsequent lane line detection processing, avoiding the use of filtered lane line points with large deviations for lane line detection, and avoiding large fluctuations in lane line detection results.

[0170] In practical applications, the number of times the line point coordinate error is greater than a preset error threshold can be counted as the error number. If the error number exceeds the preset number threshold, the lane line output can be stopped to avoid providing the user with erroneous lane line detection results.

[0171] In one embodiment, the plurality of historical cached lane line points respectively have corresponding historical lane line images, and the historical lane line images have a cache time. After the lane line point coordinate error is less than a preset error threshold and the filtered lane line point is cached, the method further includes:

[0172] Count the number of current lane line images; the number of current lane line images is the sum of the number of multiple historical lane line images and the number of lane line images; when the number of current lane line images exceeds a preset number threshold, determine the historical lane line image with the earliest cache time; among the multiple historical cache lane line points, clear the historical cache lane line point corresponding to the historical lane line image with the earliest cache time.

[0173] The historical lane line image may be a lane line image corresponding to a historical cached lane line point.

[0174] The cache time may be the time for caching the filtered lane line points according to the historical lane line images. When caching the filtered lane line points, the corresponding historical lane line images are also cached. Therefore, the cache time of the historical lane line images is the cache time of the filtered lane line points.

[0175] Specifically, when the line point coordinate error is less than a preset error threshold, after the lane line points are filtered and cached, the sum of the number of multiple historical lane line images and the number of lane line images can be further counted as the number of current lane line images mentioned above. When the number of current lane line images exceeds the preset threshold, the historical cached lane line points corresponding to the historical lane line image with the earliest cache time can be cleared from the cache. For example, when the number of current lane line images exceeds 5 frames, the lane line image of the first frame is cleared.

[0176] The above lane line detection method counts the number of lane line images corresponding to the currently cached lane line points, and clears the historical cached lane line points corresponding to the lane line image with the earliest cache time when the number exceeds a threshold, thereby controlling the historical cached lane line points used for lane line detection within a certain number, reducing the amount of calculation and improving the speed of lane line detection.

[0177] In one embodiment, when the number of the current lane line images is less than a preset number threshold, the method further includes:

[0178] Capture an initialization image; input the initialization image into the lightweight neural network to obtain the initialization lane line points output by the lightweight neural network for the initialization image; perform straight line fitting on the initialization lane line points to obtain an initialization left lane line and an initialization right lane line; determine the lane line intersection of the initialization left lane line and the initialization right lane line; when the lane line intersection does not match the vanishing point of the initialization image, return to the step of capturing the initialization image; when the lane line intersection matches the vanishing point of the initialization image, cache the initialization lane line points as the historical cache lane line points, and return to the step of capturing the lane line image.

[0179] Specifically, if the number of current lane line images is less than the preset number threshold, it indicates that the number of lane line points used for lane line detection is insufficient, and more historical cache lane line points need to be obtained. Therefore, the on-board camera can be controlled to capture a lane line image as the above-mentioned initialization image, and the initialization image is input into the lightweight neural network to obtain the lane line point output by the lightweight neural network for the initialization image. In order to distinguish the explanation, the lane line point is named the initialization lane line point.

[0180] Then, a straight line fitting is performed on the initialized lane line points to obtain the initialized left lane line and the initialized right lane line. Further, the intersection of the two lane lines is determined as the above lane line intersection point. If the lane line intersection point does not match the vanishing point of the initialized image, return to the step of shooting the initialized image and reshoot the initialized image. If the lane line intersection point matches the vanishing point of the initialized image, it can be determined that the lane line detection algorithm is initialized, the initialized lane line points are cached as historical cached lane line points, and the step of shooting the lane line image is returned.

[0181] In practical applications, the following formula can be used to determine the lane intersection point p(x p ,y p ) and the vanishing point P(x) of the initialization image P ,y P ) matches:

[0182]

[0183] According to the above formula, when the difference between the horizontal coordinates of the intersection point p and the vanishing point P is less than 20 pixels and the difference between the vertical coordinates is less than 30 pixels, it can be determined that the two match.

[0184] The above lane line detection method, when the number of current lane line images is less than a preset threshold, uses the initialized lane line points of the initialized image to perform straight line fitting, and based on the matching result of the intersection of the lane lines obtained by straight line fitting and the vanishing point of the initialized image, caches the initialized lane line points as historical cached lane line points. Thus, when the number of current lane line images is insufficient, relatively accurate historical cached lane line points are obtained for subsequent lane line detection, thereby avoiding erroneous detection results caused by lane line detection in the absence of historical cached lane line points, thereby improving the accuracy of lane line detection.

[0185] In order to facilitate those skilled in the art to have a deeper understanding of the embodiments of the present application, specific examples will be described below with reference to the accompanying drawings.

[0186] Figure 3 The present invention is a flowchart of a lane line detection process based on a cropped UFLD network according to an embodiment. As shown in the figure, the captured lane line image is input into the UFLD network, the UFLD network recognizes the lane line in the lane line image, and outputs a set of original lane line points matching the recognized lane line.

[0187] After obtaining the set of original lane line points, each original lane line point in the set of original lane line points can be first divided into left lane line points and right lane line points. For any two left lane line points, determine whether the condition that the pixel horizontal coordinate increases while the pixel vertical coordinate also increases is met. If so, output them as the filtered left lane line points; for any two right lane line points, determine whether the condition that the pixel horizontal coordinate increases while the pixel vertical coordinate decreases is met. If so, output them as the filtered right lane line points. Finally, the filtered left lane point and the filtered right lane point are used as the filtered lane line point set for the next lane line detection process.

[0188] If the algorithm initialization has been completed, the line point coordinate error between the filtered lane line points and the historical cache lane line points can be further calculated. If the line point coordinate error is greater than the preset error threshold, it is further determined whether the number of times the line point coordinate error is greater than the preset error threshold reaches the preset number threshold. If so, the detection result is temporarily not output to avoid false alarms; if not, the lane line points obtained based on the current lane line image are abandoned, and the historical cache lane line points are directly used for quadratic curve fitting to obtain fitted lane line points, and the fitted lane line points are used for subsequent lane line detection processing.

[0189] If the line point coordinate error is less than the preset error threshold, it is further determined whether the currently cached lane line image reaches the preset number threshold. If not, it returns to the step of shooting the lane line image. If it is, it further caches the filtered lane line point, and performs quadratic curve fitting based on the historically cached lane line points and the latest cached filtered lane line points to obtain the fitted lane line point. In addition, it is further determined whether the cached lane line image exceeds the number threshold. If so, the lane line image with the earliest cache time can be cleared.

[0190] After obtaining the fitted lane line points, the lane line point relocation processing can be performed based on the fitted lane line points. Specifically, the pixel points whose pixel values ​​meet the preset conditions can be traversed in the search interval of the fitted lane line points, and the relocated lane line points can be obtained according to the pixel coordinates of the pixel points that meet the conditions.

[0191] Next, the relocated lane line points are merged with the fitted lane line points to obtain a new lane line point set, which is clustered by K-means to form several lane line point classes. The target lane line point class that meets the conditions is determined based on the number of lane line points in each lane line point class and the aspect ratio of the shape they form. Finally, the lane line is fitted based on the lane line points in the target lane line point class to form the final lane line, and the lane line is output as the detection result display.

[0192] Figure 4It is a schematic diagram of a final lane line of an embodiment, and 401 shown in the figure is the final output lane line. The lane line output by the lane line detection method mentioned above in the present application is more consistent with the real lane line.

[0193] It should be understood that although Figure 2-3 The steps in the figure are shown in sequence as indicated by the arrows, but these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified in this document, there is no strict order restriction for the execution of these steps, and these steps can be executed in other orders. Moreover, Figure 2-3 At least part of the steps may include multiple steps or multiple stages. These steps or stages are not necessarily performed at the same time, but can be performed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed in turn or alternately with other steps or at least part of the steps or stages in other steps.

[0194] In one embodiment, Figure 5 As shown, a lane line detection device is provided, comprising:

[0195] An image capturing module 502 is used to capture lane line images;

[0196] A network input module 504, configured to input the lane line image into a preset lightweight neural network to obtain original lane line points output by the lightweight neural network for the lane line image;

[0197] An interval determination module 506 is used to determine a search interval on the lane line image according to the original lane line point; the search interval includes pixel points;

[0198] A search module 508, configured to search for a relocated lane line point of the original lane line point in the search interval according to a pixel value of each pixel point in the search interval;

[0199] The lane line output module 510 is used to obtain the lane line in the lane line image according to the relocated lane line point.

[0200] In one embodiment, the original lane line point has a corresponding fitted lane line point, and the lane line output module 510 is specifically used to:

[0201] The fitted lane line points and the relocated lane line points are clustered to obtain a plurality of lane line point classes; a line point quantity feature of each lane line point class is determined, and a shape feature of each lane line point class is determined; a target lane line point class is determined from the plurality of lane line point classes based on the line point quantity feature and the shape feature; and a lane line in the lane line image is obtained based on the target lane line point class.

[0202] In one embodiment, the search module 508 is specifically configured to:

[0203] In the search interval, pixel points whose pixel values ​​are greater than a preset pixel value threshold are determined to obtain positioning reference pixels; the sum of the pixel coordinates of the positioning reference pixels is counted to obtain the total pixel coordinates, and the number of the positioning reference pixels is counted to obtain the total pixel points; the ratio of the total pixel coordinates to the total pixel points is calculated to obtain the repositioning pixel coordinates; and the pixel point at the repositioning pixel coordinates is determined as the repositioning lane line point.

[0204] In one embodiment, the lane line output module 510 is specifically used to:

[0205] Counting the total number of lane line points in the lane line point class to obtain the lane point quantity feature;

[0206] Determine a lane shape formed by lane line points in the lane line point class;

[0207] Calculating the aspect ratio of the lane shape to obtain the shape feature;

[0208] Determine a lane line point class whose aspect ratio of the lane shape is less than a preset aspect ratio threshold, and obtain a plurality of candidate lane line point classes;

[0209] Among the plurality of candidate lane line point classes, a candidate lane line point class having the largest total number of lane line points in the lane line point class is determined as the target lane line point class.

[0210] In one embodiment, the original lane line points include left lane line points and right lane line points, and the interval determination module 506 is specifically used to:

[0211] Among the left lane line points, the filtered left lane line points are screened out; the filtered left lane line points include a first left lane line point and a second left lane line point; the pixel ordinate of the first left lane line point is greater than the pixel ordinate of the second left lane line point; the pixel abscissa of the first left lane line point is greater than the pixel abscissa of the second left lane line point; among the right lane line points, the filtered right lane line points are screened out; the filtered right lane line points include a first right lane line point and a second right lane line point; the pixel ordinate of the first right lane line point is greater than the pixel ordinate of the second right lane line point; the pixel abscissa of the first right lane line point is less than the pixel abscissa of the second right lane line point; the filtered left lane line points and the filtered right lane line points are used as filtered lane line points; and the search interval on the lane line image is determined according to the filtered lane line points.

[0212] In one embodiment, the interval determination module 506 is specifically configured to:

[0213] Acquire multiple historical cached lane line points; count the errors between the pixel horizontal coordinates of the filtered lane line points and the pixel horizontal coordinates of each of the historical cached lane line points to obtain a lane point coordinate error; cache the filtered lane line points when the lane point coordinate error is less than a preset error threshold; determine a search interval on the lane line image based on the multiple historical cached lane line points and the cached filtered lane line points.

[0214] In one embodiment, the interval determination module 506 is specifically configured to:

[0215] Perform curve fitting on the multiple historical cache lane line points and the filtered lane line points to obtain fitted lane line points; generate the search interval; the search interval is an area on the lane line image centered on the fitted lane line points and meeting preset pixel coordinate conditions.

[0216] In one embodiment, the interval determination module 506 is further specifically configured to:

[0217] When the line point coordinate error is greater than a preset error threshold, a search interval on the lane line image is determined according to a plurality of the historical cached lane line points.

[0218] In one embodiment, the plurality of historical cached lane line points respectively have corresponding historical lane line images, the historical lane line images have a cache time, and the device further includes:

[0219] A quantity statistics module, used to count the number of current lane line images; the number of current lane line images is the sum of the number of multiple historical lane line images and the number of lane line images;

[0220] A clearing module is used to determine the historical lane line image with the earliest cache time when the number of the current lane line images exceeds a preset number threshold; among the multiple historical cache lane line points, clear the historical cache lane line point corresponding to the historical lane line image with the earliest cache time.

[0221] In one embodiment, it further includes:

[0222] An algorithm initialization module is used to capture an initialization image; input the initialization image into the lightweight neural network to obtain the initialization lane line points output by the lightweight neural network for the initialization image; perform straight line fitting on the initialization lane line points to obtain an initialization left lane line and an initialization right lane line; determine the lane line intersection of the initialization left lane line and the initialization right lane line; when the lane line intersection does not match the vanishing point of the initialization image, jump to the image capture module 502; when the lane line intersection matches the vanishing point of the initialization image, cache the initialization lane line points as the historical cache lane line points, and jump to the image capture module 502.

[0223] The specific definition of the lane line detection device can be found in the definition of the lane line detection method above, which will not be repeated here. Each module in the above lane line detection device can be implemented in whole or in part by software, hardware and a combination thereof. The above modules can be embedded in or independent of the processor in the computer device in hardware form, or can be stored in the memory of the computer device in software form, so that the processor can call and execute the operations corresponding to the above modules.

[0224] In one embodiment, a computer device is provided. The computer device may be a vehicle-mounted terminal, and its internal structure diagram may be as follows: Figure 6 As shown. The computer device includes a processor, a memory, a communication interface, a display screen and an input device connected through a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The communication interface of the computer device is used to communicate with an external terminal in a wired or wireless manner, and the wireless manner can be achieved through WIFI, an operator network, NFC (near field communication) or other technologies. When the computer program is executed by the processor, a lane line detection method is implemented. The display screen of the computer device can be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer device can be a touch layer covering the display screen, or a button, trackball or touchpad set on the computer device housing, or an external keyboard, touchpad or mouse, etc.

[0225] Those skilled in the art will understand that Figure 6 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0226] In one embodiment, a computer device is provided, including a memory and a processor, wherein a computer program is stored in the memory, and the processor implements the steps in the above-mentioned method embodiments when executing the computer program.

[0227] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps in the above-mentioned method embodiments are implemented.

[0228] In one embodiment, a computer program product is provided, including a computer program, and when the computer program is executed by a processor, the steps in the above method embodiments are implemented.

[0229] Those of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory or optical memory, etc. Volatile memory can include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM).

[0230] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0231] The above-mentioned embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the invention patent. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the patent of the present application shall be subject to the attached claims.

Claims

1. A lane line detection method, characterized in that: The method comprises: Capture lane line images; Inputting the lane line image into a preset lightweight neural network to obtain original lane line points output by the lightweight neural network for the lane line image; Determine a search interval on the lane line image according to the original lane line point; the search interval includes pixel points; In the search interval, pixel points whose pixel values ​​are greater than a preset pixel value threshold are determined to obtain positioning reference pixels; the sum of pixel coordinates of the positioning reference pixels is counted to obtain a pixel coordinate sum, and the number of the positioning reference pixels is counted to obtain a pixel point sum; the ratio of the pixel coordinate sum to the pixel point sum is calculated to obtain a repositioning pixel coordinate; and a pixel point at the repositioning pixel coordinate is determined as a repositioning lane line point; According to the relocated lane line points, the lane lines in the lane line image are obtained.

2. The method according to claim 1, characterized in that: The original lane line point has a corresponding fitted lane line point, and obtaining a lane line in the lane line image according to the relocated lane line point includes: Clustering the fitted lane line points and the relocated lane line points to obtain a plurality of lane line point classes; Determine the number of line points of each lane line point class, and determine the shape characteristics of each lane line point class; Determine a target lane line point class from the plurality of lane line point classes according to the line point quantity feature and the shape feature; According to the target lane line point class, a lane line in the lane line image is obtained.

3. The method according to claim 2, characterized in that The determining of the line point quantity feature of each lane line point class includes: Counting the total number of lane line points in the lane line point class to obtain the lane point quantity feature; The determining of the shape features of each lane line point class includes: Determine a lane shape formed by lane line points in the lane line point class; Calculating the aspect ratio of the lane shape to obtain the shape feature; The step of determining a target lane line point class from among the plurality of lane line point classes according to the line point quantity feature and the shape feature comprises: Determine a lane line point class whose aspect ratio of the lane shape is less than a preset aspect ratio threshold, and obtain a plurality of candidate lane line point classes; Among the plurality of candidate lane line point classes, a candidate lane line point class having the largest total number of lane line points in the lane line point class is determined as the target lane line point class.

4. The method according to claim 1, characterized in that The original lane line points include left lane line points and right lane line points, and determining the search interval on the lane line image according to the original lane line points includes: Among the left lane line points, screened left lane line points are selected; the screened left lane line points include a first left lane line point and a second left lane line point; the pixel ordinate of the first left lane line point is greater than the pixel ordinate of the second left lane line point; the pixel abscissa of the first left lane line point is greater than the pixel abscissa of the second left lane line point; Filter out filtered right lane line points from the right lane line points; the filtered right lane line points include a first right lane line point and a second right lane line point; the pixel ordinate of the first right lane line point is greater than the pixel ordinate of the second right lane line point; the pixel abscissa of the first right lane line point is less than the pixel abscissa of the second right lane line point; The filtered left lane line point and the filtered right lane line point are used as filtered lane line points; A search interval on the lane line image is determined according to the filtered lane line points.

5. The method according to claim 4, characterized in that The step of determining a search interval on the lane line image according to the filtered lane line points includes: Get multiple historical cache lane line points; Counting the errors between the pixel horizontal coordinates of the filtered lane line points and the pixel horizontal coordinates of each of the historical cached lane line points to obtain line point coordinate errors; When the line point coordinate error is less than a preset error threshold, caching the filtered lane line point; A search interval on the lane line image is determined based on the plurality of historical cached lane line points and the cached filtered lane line points.

6. A lane line detection device, characterized in that: The device comprises: An image shooting module, used for shooting lane line images; A network input module, used to input the lane line image into a preset lightweight neural network to obtain original lane line points output by the lightweight neural network for the lane line image; An interval determination module, used to determine a search interval on the lane line image according to the original lane line point; the search interval includes pixel points; A search module is used to determine, in the search interval, pixel points whose pixel values ​​are greater than a preset pixel value threshold value to obtain positioning reference pixels; count the sum of pixel coordinates of the positioning reference pixels to obtain a pixel coordinate sum, and count the number of the positioning reference pixels to obtain a pixel point sum; calculate the ratio of the pixel coordinate sum to the pixel point sum to obtain a repositioning pixel coordinate; determine a pixel point at the repositioning pixel coordinate as a repositioning lane line point; A lane line output module is used to obtain the lane line in the lane line image according to the relocated lane line point.

7. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that: When the processor executes the computer program, the steps of the lane line detection method described in any one of claims 1 to 5 are implemented.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the lane line detection method described in any one of claims 1 to 5 are implemented.

9. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the lane line detection method according to any one of claims 1 to 5 is implemented.