Lane line detection and classification method, device, equipment and medium

Abstract

The present application discloses a lane line detection and classification method, device, equipment and medium, and relates to the field of lane line detection. The method comprises: performing dedistortion processing on an original image, using a trained deep learning detection model to detect the dedistorted image to obtain a lane line detection result, performing a first preprocessing on the lane line detection result to obtain a first preprocessed image; performing expansion processing on the lane line points in the first preprocessed image to obtain a plurality of lane line areas; generating a mask image based on the lane line area; performing a second preprocessing on the mask image to obtain the target color pixel ratio in the lane line area of ​​the mask image, and determining the color classification result of each lane line; performing a third preprocessing on the mask image to obtain a matching template; using a template matching algorithm, performing template matching on each lane line mask in the mask image according to the matching template, and obtaining a virtual-real classification result of each lane line. The present application improves the detection and classification accuracy.

Description

Technical Field

[0001] The present application relates to the field of lane line detection, and in particular to a lane line detection and classification method, device, equipment and medium. Background Art

[0002] Lane line detection is very important in autonomous driving algorithms. First, capturing lane line information on the road can enable the vehicle to accurately identify the location and width of the lane, and ensure that the vehicle is driving in the correct lane. Secondly, the lane line detection algorithm can provide real-time visual assistance to the driver in complex road environments. For example, at night and in bad weather, the lane line information provided by the algorithm can help the driver judge the driving status of the vehicle in the lane. In addition, the lane line detection results can also be applied to the lane keeping system. When the vehicle deviates from the lane, it can provide deviation warnings or actively correct the vehicle's driving trajectory to improve driving safety.

[0003] Currently, template matching algorithms can be used for lane line detection, but traditional template matching algorithms are not sensitive to rotation and deformation. When the target image is rotated and deformed relative to the template, the template matching algorithm may not be able to accurately match the target template. In road scenes, lane lines may be straight lines or curves, so during the matching process, the curve matching degree may be insufficient, resulting in the failure of the matching algorithm. The matching algorithm of multiple templates will consume more computing resources and even cause the system's real-time performance to decrease. Summary of the invention

[0004] The purpose of this application is to provide a lane line detection and classification method, device, equipment and medium, which can improve the accuracy of lane line detection and classification.

[0005] To achieve the above objectives, this application provides the following solutions:

[0006] In a first aspect, the present application provides a lane line detection and classification method, comprising:

[0007] Acquire an original image; the original image is obtained by photographing the lane where the target vehicle is traveling using a monocular camera;

[0008] Performing dedistortion processing on the original image to obtain a dedistorted image;

[0009] Inputting the dedistorted image into a trained deep learning detection model to obtain a lane line detection result in the dedistorted image; the lane line detection result includes a plurality of lane line position information; each lane line position information includes pixel coordinates of a plurality of lane line points on the lane line in the dedistorted image;

[0010] Performing a first preprocessing on the lane line detection result to obtain a first preprocessed image; performing an expansion process on the lane line points in the first preprocessed image to obtain a plurality of lane line regions; and generating a mask image based on the lane line regions;

[0011] Performing a second preprocessing on the mask image to obtain a target color pixel ratio in the lane line area of ​​the mask image; determining a color classification result of each lane line according to the target color pixel ratio in each lane line area of ​​the mask image; the target color is white or yellow;

[0012] The mask image is subjected to a third preprocessing to obtain a matching template; and a template matching algorithm is used to perform template matching on each lane line mask in the mask image according to the matching template to obtain a virtual-real classification result of each lane line.

[0013] In a second aspect, the present application provides a lane line detection and classification device, comprising:

[0014] The original image acquisition module is used to: acquire an original image; the original image is obtained by photographing the lane where the target vehicle is traveling using a monocular camera;

[0015] The dedistortion processing module is used to: perform dedistortion processing on the original image to obtain a dedistorted image;

[0016] A lane line detection module, used to: input the dedistorted image into a trained deep learning detection model to obtain a lane line detection result in the dedistorted image; the lane line detection result includes a plurality of lane line position information; each lane line position information includes pixel coordinates of a plurality of lane line points on the lane line in the dedistorted image;

[0017] The mask image generation module is used to: perform a first preprocessing on the lane line detection result to obtain a first preprocessed image; perform an expansion process on the lane line points in the first preprocessed image to obtain a plurality of lane line regions; and generate a mask image based on the lane line regions;

[0018] A color classification module is used to: perform a second preprocessing on the mask image to obtain a target color pixel ratio in the lane line area of ​​the mask image; determine a color classification result for each lane line according to the target color pixel ratio in each lane line area of ​​the mask image; the target color is white or yellow;

[0019] The virtual-real classification module is used to: perform a third preprocessing on the mask image to obtain a matching template; use a template matching algorithm to perform template matching on each lane line mask in the mask image according to the matching template to obtain a virtual-real classification result for each lane line.

[0020] In a third aspect, the present application provides a computer device, comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the above-mentioned lane line detection and classification method.

[0021] In a fourth aspect, the present application provides a computer-readable storage medium having a computer program stored thereon, which implements the above-mentioned lane line detection and classification method when executed by a processor.

[0022] According to the specific embodiments provided in this application, this application discloses the following technical effects:

[0023] The present application provides a lane line detection and classification method, apparatus, device and medium, which uses a monocular camera to obtain an original image, performs dedistortion processing on the original image, uses a trained deep learning detection model to detect the dedistorted image to obtain a lane line detection result, performs a first preprocessing on the lane line detection result to obtain a first preprocessed image; performs expansion processing on the lane line points in the first preprocessed image to obtain a number of lane line areas; generates a mask image based on the lane line area; performs a second preprocessing on the mask image to obtain the target color pixel ratio in the lane line area of ​​the mask image, and determines the color classification result of each lane line; performs a third preprocessing on the mask image to obtain a matching template; uses a template matching algorithm to perform template matching on each lane line mask in the mask image according to the matching template to obtain a virtual and real classification result of each lane line. The present application improves the detection and classification accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0025] Figure 1 This is an application environment diagram of a lane line detection and classification method in one embodiment of the present application;

[0026] Figure 2 A schematic diagram of a process flow of a lane line detection and classification method provided in one embodiment of the present application;

[0027] Figure 3 A schematic diagram of a de-distortion processing flow provided in an embodiment of the present application;

[0028] Figure 4 A schematic diagram of a specific process of lane line detection and classification provided in an embodiment of the present application;

[0029] Figure 5 A schematic diagram of the detection results of lane lines provided in an embodiment of the present application;

[0030] Figure 6 A schematic diagram of a smoothed image provided by an embodiment of the present application;

[0031] Figure 7 A detailed schematic diagram before point smoothing and outlier removal provided in an embodiment of the present application;

[0032] Figure 8 A detailed schematic diagram of point smoothing and outlier point removal provided in an embodiment of the present application;

[0033] Fig. 9 A schematic diagram of a mask image provided by an embodiment of the present application;

[0034] Fig.10 A schematic diagram of the structure of a computer device provided in one embodiment of the present application. DETAILED DESCRIPTION

[0035] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0036] Template matching algorithm is a technique used to find specific targets in an image. The algorithm moves the template across the entire image and calculates the similarity between the template and the covered window on the image. It is mainly implemented through two-dimensional convolution. During the convolution process, the output pixel value is calculated by multiplying and summing the elements of two matrices. One matrix is ​​the image itself, and the other matrix is ​​the template, that is, the convolution kernel.

[0037] In order to make the above-mentioned objects, features and advantages of the present application more obvious and easy to understand, the present application is further described in detail below with reference to the accompanying drawings and specific implementation methods.

[0038] The lane line detection and classification method provided in the embodiment of the present application can be applied to Figure 1In the application environment shown. Among them, the terminal 102 communicates with the server 104 through the network. The data storage system can store the data that the server 104 needs to process. The data storage system can be set up separately, integrated on the server 104, or placed on the cloud or other servers. The terminal 102 can send the original image to the server 104. After the server 104 receives the original image, for the original image, the server 104 performs dedistortion processing on the original image, uses the trained deep learning detection model to detect the dedistorted image to obtain the lane line detection result, performs a first preprocessing on the lane line detection result to obtain a first preprocessed image; performs dilation processing on the lane line points in the first preprocessed image to obtain several lane line areas; generates a mask image based on the lane line area; performs a second preprocessing on the mask image to obtain the target color pixel ratio in the lane line area of ​​the mask image, and determines the color classification result of each lane line; performs a third preprocessing on the mask image to obtain a matching template; uses a template matching algorithm to perform template matching on each lane line mask in the mask image according to the matching template to obtain the virtual and real classification result of each lane line. The server 104 may feed back the obtained lane line classification result for the original image to the terminal 102. In addition, in some embodiments, the lane line detection and classification method may also be implemented by the server 104 or the terminal 102 alone, such as the terminal 102 may directly perform lane line detection and classification processing on the original image, or the server 104 may obtain the original image from the data storage system and perform lane line detection and classification processing on the original image.

[0039] The terminal 102 may be, but is not limited to, various desktop computers and laptop computers. The server 104 may be implemented as an independent server or a server cluster consisting of multiple servers, or may be a cloud server.

[0040] In an exemplary embodiment, Figure 2 As shown, a lane line detection and classification method is provided. The method is executed by a computer device, and can be executed by a computer device such as a terminal or a server alone, or by a terminal and a server together. In the embodiment of the present application, the method is applied to Figure 1 The server 104 in the example is used as an example to illustrate the method, which includes the following steps 201 to 206. Among them:

[0041] Step 201: Acquire an original image; the original image is obtained by photographing the lane where the target vehicle is traveling using a monocular camera.

[0042] Step 202: Dedistort the original image to obtain a dedistorted image.

[0043] Step 203: Input the dedistorted image into a trained deep learning detection model to obtain a lane line detection result in the dedistorted image; the lane line detection result includes a plurality of lane line position information; each lane line position information includes pixel coordinates of a plurality of lane line points on the lane line in the dedistorted image.

[0044] Step 204: performing a first preprocessing on the lane line detection result to obtain a first preprocessed image; performing dilation processing on the lane line points in the first preprocessed image to obtain a plurality of lane line regions; and generating a mask image based on the lane line regions.

[0045] Step 205: Perform a second preprocessing on the mask image to obtain the target color pixel ratio in the lane line area of ​​the mask image; determine the color classification result of each lane line according to the target color pixel ratio in each lane line area of ​​the mask image; the target color is white or yellow.

[0046] Step 206: performing a third preprocessing on the mask image to obtain a matching template; using a template matching algorithm, performing template matching on each lane line mask in the mask image according to the matching template to obtain a virtual-real classification result for each lane line.

[0047] Implementing the above steps 201 to 206, overall, this model uses a monocular camera for lane line detection, which can reduce hardware costs, and remove distortion through calibration to improve lane line detection accuracy. While using a deep learning detection model to detect lane lines, considering that most models lack the ability to classify lane lines, the color information of the mask area is statistically analyzed through color space conversion, and the classification results of yellow / white lane lines are obtained. At the same time, the situation of lighting differences is also taken into account, and a brightness balancing function is added. At the same time, an adaptive generative template method is used to classify the virtual and real lane lines. Compared with the traditional template matching algorithm that only uses one or more templates, it is more flexible.

[0048] This application uses an ordinary monocular camera instead of a fisheye camera. Ordinary cameras can completely cover the four lanes on the road. Monocular cameras perceive the surrounding environment through the principle of optical imaging. The images they collect not only contain a large amount of RGB color information, but also retain a large amount of edge, corner, texture and other information. This information can be used as input for machine vision algorithms and deep learning models for tasks such as target detection, recognition and tracking. These technologies are currently widely used in the field of autonomous driving, providing key visual perception capabilities for the realization of intelligent systems. In addition, monocular cameras have the advantages of small size and low cost, making them more advantageous for use in mass-produced vehicles.

[0049] In another exemplary embodiment of the present application, Figure 3 As shown, the above step 202 specifically includes the following steps 301~302.

[0050] Step 301: using a checkerboard calibration plate to calibrate a monocular camera to obtain camera parameters; the camera parameters include camera intrinsic parameters and distortion coefficients;

[0051] Step 302: using a dedistortion model, dedistorting the original image by projecting the normalized coordinates back to the original coordinate space to obtain a dedistorted image.

[0052] Use the checkerboard calibration plate to calibrate the monocular camera and obtain the camera parameters including its intrinsic parameters and distortion coefficients. The specific method is to hold the calibration plate and take more than 20 original images in front of the camera. The calibration plate in each original image must occupy more than 20% of the image space, and the calibration plate in these 20 original images should cover the entire camera field of view as much as possible. Then use the Zhang Zhengyou calibration method for calibration.

[0053] After obtaining the distortion parameters, different dedistortion methods are used according to different camera types. Ordinary cameras are pinhole models, and a simplified dedistortion model can be used to achieve dedistortion by projecting the normalized coordinates back to the original coordinate space. Assume that the original coordinates are , the coordinates after dedistortion are , the dedistortion process includes radial distortion and tangential distortion, and the specific formula is as follows:

[0054] Radial distortion:

[0055] (1);

[0056] (2);

[0057] Tangential distortion:

[0058] (3);

[0059] (4);

[0060] in, and are the radial distortion parameters and tangential distortion parameters of the monocular camera respectively. After this process, the dedistorted image corresponding to the original image can be obtained.

[0061] Before inputting the dedistorted image into the trained deep learning detection model in step 203, the deep learning detection model needs to be trained. The training process is as follows:

[0062] Collect pictures on the road, name them sample images, and use points to mark the lane lines in the sample images to obtain marked pictures. Use the marked pictures as labels for the sample images. We only focus on the two lane lines on the left and right sides of the current vehicle and the lane lines adjacent to these two lane lines.

[0063] Using the labeled pictures, sample images as input, and sample image labels as output, the network model is trained to obtain a trained deep learning detection model. The backbone network (backbone) of the deep learning detection model training in this application is the ResNet18 model, which is smaller in scale and runs faster on the vehicle-mounted edge device. The total loss function of the deep learning detection model training process includes classification loss, distance loss, and regression loss.

[0064] 1) The calculation formula of classification loss is as follows:

[0065] (5);

[0066] in, is the classification loss The weight of Represents the width and height of the sample image and The lane line marking result of the grid is 1 if it contains lane line points, otherwise it is 0; Represents the width and height of the sample image and The model prediction results of the grid; and Represents the number of grids that the sample image is divided into as a whole, Represents one of the four lane lines, that is, the total number of lane lines In this embodiment, the resolution is 1920 The 1080 sample image is divided into 192 108 grids.

[0067] 2) The calculation formula of distance loss is as follows:

[0068] (6);

[0069] in, Distance loss The weight of Represents the actual lane line points marked (located in the sample image in the width and height directions and The coordinates of the grid) in the pixel coordinate system, Represents the coordinates of the lane line points predicted by the deep learning detection model in the pixel coordinate system.

[0070] 3) The calculation formula of variance loss is as follows:

[0071] (7);

[0072] in, Variance loss The weight of Representative The mean of the coordinates of all lane line points on the lane line in the pixel coordinate system, It is the coordinate in the pixel coordinate system detected by the deep learning detection model.

[0073] The above three losses are weighted proportionally to obtain the final total loss function :

[0074] (8);

[0075] Setting the weight of the classification loss and the weight of the distance loss is 1, the weight of variance loss is 0.001. Then the model parameters are updated by stochastic gradient descent SGD, and a deep learning detection model that can detect lane lines can be obtained after training.

[0076] Using the trained deep learning detection model to infer the original image captured by the monocular camera, four lane line detection results will be obtained. See the results. Figure 5 The result of each lane line is expressed in pixel coordinates. Each point belongs to a lane line. The detection results are shown in Figure 5 .

[0077] In the above step 204, the lane line detection result is subjected to a first preprocessing to obtain a first preprocessed image, which specifically includes: point smoothing and outlier removal are performed on the lane line detection result to obtain a smoothed image; the smoothed image is the first preprocessed image.

[0078] Perform point smoothing and outlier removal. Suppose a lane line detected by the trained deep learning detection model has Lane line points, the detected lane line points are used as reference points for the points to be optimized , a lane line point after smoothing The coordinates are , the cost function includes the smoothness cost Length cost and offset cost The expression of smoothness cost is as follows:

[0079] (9);

[0080] in, and To detect the The previous adjacent point and the next adjacent point of a lane line point. It can be understood as and The modulus of the sum of the vectors. The smaller the value, the more likely it is that there are three points (the Lane line points and The value is 0, and the three points are collinear. The meaning is the distance between two adjacent lane line points. The expression of length cost is as follows:

[0081] (10);

[0082] The expression of the offset cost is as follows:

[0083] (11);

[0084] Offset cost That is, the distance between the optimized point and the original reference point should not be too far. The smoothed image result is shown in Figure 6 . For detailed comparison before and after fitting, see Figure 7 and Figure 8 .

[0085] The smoothed lane line pixels are expanded to generate the area to be detected, i.e. the lane line area, and then the subsequent yellow / white classification and virtual / real classification are performed. Assume that a point at a certain position is , since most lane lines are vertical, we only need to expand to the left / right, and the expansion equation is:

[0086] (12);

[0087] (13);

[0088] In the formula, is the expansion value in the left direction, The right expansion value

[0089] This calculation method is due to the characteristics of the camera, from near to far, that is, In the decreasing direction, the lane line becomes thinner and thinner, so in this direction, the area that needs to be expanded also decreases to avoid the expansion area including too many non-lane line areas on the road. Taking the rightmost lane line as an example, the mask effect is shown in Fig. 9 .

[0090] A mask image is generated based on the dilated area (lane line area) generated above. The number of all pixels in the mask image and the number of pixels belonging to yellow / white are calculated, and finally the pixel ratio of each color is calculated to classify the lane line yellow / white.

[0091] The mask image is subjected to a second preprocessing to obtain the target color pixel ratio in the lane line area of ​​the mask image, which specifically includes the following steps 401 to 406:

[0092] Step 401: Convert the mask image from the RGB color space to the HSV color space image. After obtaining the mask image, convert the mask image from the RGB color space to the HSV color space. The HSV color space includes hue , Saturation , Hue V Three servings.

[0093] The image captured by the camera is a three-channel image of R / G / B. The pixel at each position contains RGB values. In order to calculate the HSV color space, first calculate the maximum and minimum values ​​of RGB:

[0094] (14);

[0095] (15);

[0096] (16);

[0097] In the formula, is the maximum value of RGB, is the minimum value of RGB, It is the difference between the maximum and minimum values ​​of RGB.

[0098] Then normalize the RGB values:

[0099] (17);

[0100] In the formula, , , After normalization, R value, G value, B value.

[0101] Then according to the normalized R value, G value, B Value calculation hue :

[0102] (18);

[0103] Saturation The calculation formula is as follows:

[0104] (19);

[0105] Hue The calculation formula is as follows:

[0106] (20).

[0107] Step 402: Convert the HSV color space image from the HSV color space to the HLS color space image. Since the color of the lane line may change due to lighting and other issues, the brightness equalization algorithm based on this needs to first convert the image from HSV to HLS color space. The three components of the HLS color space are hue, , brightness L, saturation , the calculation formulas are as follows:

[0108] (twenty one);

[0109] (twenty two);

[0110] (twenty three);

[0111] In the formula, and is the hue and saturation of HSV obtained previously.

[0112] Step 403: Calculate the mean of the HLS color space brightness components of all pixels in the mask image to obtain the brightness arithmetic mean.

[0113] Step 404: Perform brightness equalization on the brightness components of the lane line points in the HLS color space image according to the brightness arithmetic mean value to obtain equalized brightness.

[0114] Step 405: Calculate a new hue value according to the equalized brightness and the saturation in the HLS color space image.

[0115] Step 406: Determine the target color pixel ratio in the lane line area of ​​the mask image according to the new hue value and the HSV threshold.

[0116] After obtaining the brightness value, calculate the arithmetic mean of the brightness of all pixels in the mask image. , the brightness after equalization , using the equalized brightness and Saturation Find the new hue value. The calculation formula of the new hue value is as follows:

[0117] (twenty four);

[0118] Then, the HSV threshold is used to obtain the proportion of white pixels in the mask image. When the target color pixel ratio in the lane line area of ​​the mask image is greater than the set white pixel threshold, the lane line is a white line. Conversely, when the target color pixel ratio in the lane line area of ​​the mask image is not greater than the set white pixel threshold, the lane line is a yellow line.

[0119] Because lane lines have only two colors, yellow and white, lane line color classification is a binary classification problem. In addition, since the length of the dotted line on domestic roads is 4 meters and the interval is also 4 meters, considering that the pixels of the road may be included in the expansion process, the target color pixel ratio is The white pixel threshold is set to 0.4. According to the white pixel HSV range shown in Table 1, the image is traversed to obtain the number of white pixels in the mask. Considering the reasons of lighting and fading, the value of the white pixel HSV range is slightly larger than the range in the HSV color reference table.

[0120] Table 1 Pixel HSV range

[0121]

[0122] In Table 1, Hmin and Hmax are the minimum and maximum values ​​of hue, Smin and Smax are the minimum and maximum values ​​of saturation, Vmin and Vmax are the minimum and maximum values ​​of hue,

[0123] After obtaining the yellow / white classification, an adaptive template matching algorithm is used to classify the lane line virtual / real. In the above step 206, the mask image is subjected to a third preprocessing to obtain a matching template, which specifically includes the following steps 601 to 603:

[0124] Step 601: grayscale processing is performed on the mask image to obtain a grayscale image.

[0125] Step 602: binarize the grayscale image to obtain a filling mask.

[0126] Step 603: performing fuzzy processing on the filling mask to obtain a matching template specifically includes: performing fuzzy processing on the filling mask according to the speed of the target vehicle to obtain a matching template.

[0127] The mask image obtained in step 204 is first converted into a grayscale image and then binarized to obtain a filled mask, in which the background is black and the lane line is filled with white, i.e., the template required for subsequent matching. The image features of this template are similar to the solid lane line. Moreover, the template will be automatically generated as the vehicle moves and the lane line changes.

[0128] In addition, the lane lines may be blurred when the vehicle speed is high, so it is also necessary to blur the mask edge according to the current vehicle speed. Since the lane lines are solid, there is no need to identify the lane line contours first and then blur the contours. The template can be blurred as a whole. The blurring is achieved by adding a motion blur kernel in the vertical direction:

[0129] (25);

[0130] in, is the speed of the car, in km / h, is a coefficient, usually 0.05. H is the motion blur kernel in the vertical direction, After that, you can use OpenCV The function generates a motion blurred image as a new template, namely the matching template.

[0131] After generating new dotted and solid line templates (matching templates), use the template matching algorithm to The masks of the lane lines are matched with templates respectively, and the correlation coefficients between each lane line in the mask image and the dotted line matching template and the solid line matching template are calculated respectively. :

[0132] (26);

[0133] in, and represents the mean of the pixel values ​​of the mask image and the matching template, For the The pixel value of a pixel in the mask image, For the The pixel value of a pixel in the matching template, is the number of pixels. The larger the calculated correlation coefficient is, the higher the matching degree is. You can use OpenCV () function implements the matching operation. For each lane line in the mask image, when the correlation coefficient between the lane line and the dotted line matching template is greater than the correlation coefficient between the lane line and the solid line matching template, the lane line is a dotted line; when the correlation coefficient between the lane line and the dotted line matching template is less than the correlation coefficient between the lane line and the solid line matching template, the lane line is a solid line, and finally the virtual-real classification result of each lane line in the mask image is obtained. The overall workflow is as follows: Figure 4 shown.

[0134] The present application also provides an application scenario, which applies the above-mentioned lane line detection and classification method. Specifically: The lane line detection and classification method provided in this embodiment can be applied in content distribution scenarios. The lane line classification scenario includes an image acquisition link, a lane line detection and classification link; the original image enters the lane line detection and classification link from the image acquisition link, and the corresponding lane line classification result is obtained through human-computer collaboration. The lane line detection and classification method provided in this embodiment belongs to the lane line detection and classification link. Specifically, in the lane line detection and classification link process for the original image, the original image can be dedistorted, and the dedistorted image can be detected using a trained deep learning detection model to obtain a lane line detection result. The lane line detection result is subjected to a first preprocessing to obtain a first preprocessed image; the lane line points in the first preprocessed image are expanded to obtain a number of lane line areas; a mask image is generated based on the lane line area; the mask image is subjected to a second preprocessing to obtain the target color pixel ratio in the lane line area of ​​the mask image, and the color classification result of each lane line is determined; the mask image is subjected to a third preprocessing to obtain a matching template; and a template matching algorithm is used to perform template matching on each lane line mask in the mask image according to the matching template to obtain a virtual-real classification result of each lane line.

[0135] Based on the same inventive concept, the embodiment of the present application also provides a lane line detection and classification device for implementing the lane line detection and classification method involved above. The implementation scheme for solving the problem provided by the device is similar to the implementation scheme recorded in the above method, so the specific limitations in one or more lane line detection and classification device embodiments provided below can refer to the limitations of the lane line detection and classification method above, and will not be repeated here.

[0136] In an exemplary embodiment, the present application further provides a lane line detection and classification device including:

[0137] The original image acquisition module is used to: acquire an original image; the original image is obtained by photographing the lane where the target vehicle is traveling using a monocular camera;

[0138] The dedistortion processing module is used to: perform dedistortion processing on the original image to obtain a dedistorted image;

[0139] A lane line detection module, used to: input the dedistorted image into a trained deep learning detection model to obtain a lane line detection result in the dedistorted image; the lane line detection result includes a plurality of lane line position information; each lane line position information includes pixel coordinates of a plurality of lane line points on the lane line in the dedistorted image;

[0140] The mask image generation module is used to: perform a first preprocessing on the lane line detection result to obtain a first preprocessed image; perform an expansion process on the lane line points in the first preprocessed image to obtain a plurality of lane line regions; and generate a mask image based on the lane line regions;

[0141] A color classification module is used to: perform a second preprocessing on the mask image to obtain a target color pixel ratio in the lane line area of ​​the mask image; determine a color classification result for each lane line according to the target color pixel ratio in each lane line area of ​​the mask image; the target color is white or yellow;

[0142] The virtual-real classification module is used to: perform a third preprocessing on the mask image to obtain a matching template; use a template matching algorithm to perform template matching on each lane line mask in the mask image according to the matching template to obtain a virtual-real classification result for each lane line.

[0143] In an exemplary embodiment, a computer device is provided. The computer device may be a server or a terminal. The internal structure diagram thereof may be as follows: Fig.10 As shown. The computer device includes a processor, a memory, an input / output interface (Input / Output, referred to as I / O) and a communication interface. The processor, the memory and the input / output interface are connected through a system bus, and the communication interface is connected to the system bus through the input / output interface. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store lane line detection and classification processing data. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a lane line detection and classification method is implemented.

[0144] Those skilled in the art will understand that Fig.10The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0145] In an exemplary embodiment, a computer device is further provided, including a memory and a processor, wherein a computer program is stored in the memory, and the processor implements the steps in the above-mentioned method embodiments when executing the computer program.

[0146] In an exemplary embodiment, a computer-readable storage medium is provided, storing a computer program, and when the computer program is executed by a processor, the steps in the above method embodiments are implemented.

[0147] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant regulations.

[0148] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetoresistive random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM may be in various forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM).

[0149] The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. The non-relational database may include a distributed database based on blockchain, etc., but is not limited thereto. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, etc., but is not limited thereto.

[0150] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0151] This article uses specific examples to illustrate the principles and implementation methods of this application. The description of the above embodiments is only used to help understand the method and core ideas of this application. At the same time, for those skilled in the art, according to the ideas of this application, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as limiting this application.

Claims

1. A lane line detection and classification method, characterized in that: The lane line detection and classification method comprises: Acquire an original image; the original image is obtained by photographing the lane where the target vehicle is traveling using a monocular camera; Performing dedistortion processing on the original image to obtain a dedistorted image; Inputting the dedistorted image into a trained deep learning detection model to obtain a lane line detection result in the dedistorted image; the lane line detection result includes a plurality of lane line position information; each lane line position information includes pixel coordinates of a plurality of lane line points on the lane line in the dedistorted image; Performing a first preprocessing on the lane line detection result to obtain a first preprocessed image; performing an expansion process on the lane line points in the first preprocessed image to obtain a plurality of lane line regions; and generating a mask image based on the lane line regions; Performing a second preprocessing on the mask image to obtain the target color pixel ratio in the lane line area of ​​the mask image, specifically including: converting the mask image from the RGB color space to the HSV color space image; converting the HSV color space image from the HSV color space to the HLS color space image; calculating the mean of the HLS color space brightness components of all pixels in the mask image to obtain a brightness arithmetic mean; performing brightness equalization on the brightness components of the lane line points in the HLS color space image according to the brightness arithmetic mean to obtain a balanced brightness; calculating a new hue value according to the balanced brightness and the saturation in the HLS color space image; determining the target color pixel ratio in the lane line area of ​​the mask image according to the new hue value and the HSV threshold; Determine the color classification result of each lane line according to the target color pixel ratio in each lane line area of ​​the mask image; the target color is white or yellow; Performing a third preprocessing on the mask image to obtain a matching template, specifically comprising: performing grayscale processing on the mask image to obtain a grayscale image; performing binarization processing on the grayscale image to obtain a filled mask; performing blur processing on the filled mask to obtain a matching template; Using the template matching algorithm, template matching is performed on each lane line mask in the mask image according to the matching template to obtain a virtual and real classification result for each lane line.

2. The lane line detection and classification method according to claim 1, characterized in that: Performing a dedistortion process on the original image to obtain a dedistorted image specifically includes: The monocular camera is corrected using a checkerboard calibration plate to obtain camera parameters including camera intrinsic parameters and distortion coefficients; The dedistortion model is adopted to project the normalized coordinates back to the original coordinate space, and the original image is dedistorted to obtain a dedistorted image.

3. The lane line detection and classification method according to claim 1, characterized in that: The backbone network of the deep learning detection model is the ResNet18 model.

4. The lane line detection and classification method according to claim 1, characterized in that: Performing a first preprocessing on the lane line detection result to obtain a first preprocessed image specifically includes: Point smoothing and outlier removal are performed on the lane line detection result to obtain a smoothed image; the smoothed image is the first preprocessed image.

5. The lane line detection and classification method according to claim 1, characterized in that: The filling mask is subjected to blurring processing, specifically comprising: The filling mask is blurred according to the speed of the target vehicle.

6. A lane line detection and classification device based on the lane line detection and classification method according to claim 1, characterized in that: The lane line detection and classification device comprises: The original image acquisition module is used to: acquire an original image; the original image is obtained by photographing the lane where the target vehicle is traveling using a monocular camera; The dedistortion processing module is used to: perform dedistortion processing on the original image to obtain a dedistorted image; A lane line detection module, used to: input the dedistorted image into a trained deep learning detection model to obtain a lane line detection result in the dedistorted image; the lane line detection result includes a plurality of lane line position information; each lane line position information includes pixel coordinates of a plurality of lane line points on the lane line in the dedistorted image; The mask image generation module is used to: perform a first preprocessing on the lane line detection result to obtain a first preprocessed image; perform an expansion process on the lane line points in the first preprocessed image to obtain a plurality of lane line regions; and generate a mask image based on the lane line regions; A color classification module is used to: perform a second preprocessing on the mask image to obtain a target color pixel ratio in the lane line area of ​​the mask image; determine a color classification result for each lane line according to the target color pixel ratio in each lane line area of ​​the mask image; the target color is white or yellow; The virtual-real classification module is used to: perform a third preprocessing on the mask image to obtain a matching template; use a template matching algorithm to perform template matching on each lane line mask in the mask image according to the matching template to obtain a virtual-real classification result for each lane line.

7. A computer device comprising: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the lane line detection and classification method according to any one of claims 1 to 5.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the lane line detection and classification method described in any one of claims 1 to 5 is implemented.

Images