Lane line recognition method, device, electronic device and storage medium
The lane line recognition method segments and enhances resolution to improve accuracy, addressing the issue of inaccurate distant lane line recognition in autonomous driving, ensuring safer operations.
Patent Information
- Application Number
- CN202210613135.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2022-05-31
- Publication Date
- 2024-12-03
- Estimated Expiration
- 2042-05-31
AI Technical Summary
Existing lane line recognition methods, particularly using deep learning models, suffer from inaccurate recognition of distant lane lines due to blurring, compromising the safety of autonomous driving vehicles.
A lane line recognition method that segments the lane line image into two parts based on size, processes one part to enhance resolution using a super-resolution model, and combines the recognition results to improve accuracy.
Enhances the accuracy of lane line recognition by ensuring clearer resolution and improved recognition of distant lane lines, thereby enhancing the safety and reliability of autonomous driving systems.
Smart Images

Figure CN114926801B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of image recognition, and in particular to a lane line recognition method, device, electronic equipment and storage medium. Background Art
[0002] Lane detection plays an important role in the field of autonomous driving and is an important part of the autonomous driving environment perception module. At present, there are two mainstream methods for lane detection, namely traditional image processing methods and deep learning image segmentation methods. The main process of the traditional image processing method is: for image color information, first filter the image noise through a Gaussian filter; then detect the edge information of the lane line in a simple scene; finally, obtain the driving lane line through the Hough transform, or perform curve fitting after perspective transformation to calculate the curvature of the lane line. The deep learning image segmentation method uses a convolutional neural network instead of a manual method for feature extraction. The main process is: input the lane line image to be identified collected by the image sensor into the trained neural network model, and the model outputs the lane line recognition result.
[0003] However, in the prior art, when the lane line image to be identified is recognized based on the deep learning model, since the distant image is more blurred than the nearby image, it often leads to inaccurate recognition of the "distant" lane line in the lane line image to be identified, thereby failing to ensure the safety of the autonomous driving vehicle. Summary of the invention
[0004] In view of this, an embodiment of the present invention provides a lane line recognition method, aiming to solve the problem of inaccurate lane line recognition in the prior art.
[0005] According to a first aspect, an embodiment of the present invention provides a lane line recognition method, comprising:
[0006] Obtaining a lane line image to be identified;
[0007] According to the size of the lane line image to be identified, the lane line image to be identified is segmented along a preset direction to generate a first image and a second image; the first image is located above the second image;
[0008] Processing the resolution of the first image to generate a third image, wherein the resolution of the third image is higher than that of the first image;
[0009] The lane lines in the third image and the second image are recognized respectively, and the recognition results are spliced to generate a lane line recognition result corresponding to the lane line image to be recognized.
[0010] The lane line recognition method provided by the embodiment of the present invention obtains a lane line image to be recognized, and then divides the lane line image to be recognized along a preset direction according to the size of the lane line image to be recognized, and generates a first image and a second image, wherein the first image is located above the second image, so that the lane line image included in the generated first image can be guaranteed to be smaller and less clear, and the lane line image included in the generated second image can be guaranteed to be larger and clearer. Then, the resolution of the first image is processed to generate a third image, and the resolution of the third image is higher than that of the first image, which ensures that the lane line image included in the generated third image is clearer. Then, the lane lines in the third image and the second image are respectively recognized, and the recognition results are spliced to generate a lane line recognition result corresponding to the lane line image to be recognized. Thereby, the accuracy of the lane line recognition results of the second image and the third image is guaranteed, and then the accuracy of the lane line recognition result corresponding to the generated lane line image to be recognized is guaranteed. In the above method, the lane line image to be recognized is divided, and then the resolution of the first image located above the lane line image to be recognized is processed, so as to improve the resolution of the generated third image. This ensures the accuracy of the lane line recognition result corresponding to the generated lane line image to be recognized, rather than directly performing lane line recognition on the lane line image to be recognized.
[0011] In combination with the first aspect, in a first implementation of the first aspect, processing the resolution of the first image to generate a third image includes:
[0012] The first image is input into the super-resolution model, and the super-resolution model extracts features from the first image and outputs a third image.
[0013] The lane line recognition method provided by the embodiment of the present invention inputs the first image into the super-resolution model, and the super-resolution model extracts features from the first image and outputs a third image. The resolution of the output third image is ensured to be high, thereby ensuring the accuracy of the lane line recognition result obtained by performing lane line recognition on the third image.
[0014] In combination with the first implementation of the first aspect, in the second implementation of the first aspect, the training process of the super-resolution model includes:
[0015] Obtain a super-resolution image training set; the super-resolution image training set is generated according to a preset image of each lane line image in the lane line image set;
[0016] Based on the super-resolution image training set, the initial super-resolution network is trained to generate a super-resolution model.
[0017] The lane line recognition method provided by the embodiment of the present invention obtains a super-resolution image training set; wherein the super-resolution image training set is generated based on preset images in each lane line image in the lane line image set. Based on the super-resolution image training set, an initial super-resolution network is trained to generate a super-resolution model. This ensures that the generated super-resolution model can be more suitable for the first image in the lane line images to be recognized, thereby ensuring that the resolution of the lane line image in the third image output by inputting the first image into the super-resolution model is higher, thereby ensuring that the lane line recognition result obtained by performing lane line recognition on the third image is accurate.
[0018] In combination with the second implementation of the first aspect, in the third implementation of the first aspect, obtaining a super-resolution image training set includes:
[0019] Get a lane line image set;
[0020] For each lane line image in the lane line image set, segment the lane line image along a preset direction to generate a first training image and a second training image; the first training image is located above the second training image;
[0021] compressing each second training image to generate a compressed image; the compressed image has a lower resolution than the second training image;
[0022] Each compressed image is used as a training image, and a second training image corresponding to each compressed image is used as annotation information corresponding to the training image to generate a super-resolution image training set.
[0023] The lane line recognition method provided by the embodiment of the present invention obtains a lane line image set, and for each lane line image in the lane line image set, segments the lane line image along a preset direction to generate a first training image and a second training image; the first training image is located above the second training image; it is ensured that the lane line image included in the generated first training image is smaller and less clear, and the lane line image included in the generated second training image is larger and clearer. Then, each second training image is compressed to generate a compressed image, which ensures that the resolution of the generated compressed image is low. Then, each compressed image is used as a training image, and the second training image corresponding to each compressed image is used as the annotation information corresponding to the training image to generate a super-resolution image training set, thereby ensuring the accuracy of the generated super-resolution image training set, and further ensuring the accuracy of the super-resolution model trained based on the super-resolution image training set.
[0024] In combination with the first aspect, in a fourth implementation of the first aspect, respectively recognizing lane lines in the third image and the second image includes:
[0025] The third image and the second image are input into the lane line recognition model, and the lane line recognition results corresponding to the third image and the second image are output respectively.
[0026] The lane line recognition method provided by the embodiment of the present invention inputs the third image and the second image into the lane line recognition model, and outputs the lane line recognition results corresponding to the third image and the second image respectively, thereby ensuring the accuracy of the outputted lane line recognition results corresponding to the third image and the second image.
[0027] In combination with the fourth implementation manner of the first aspect, in the fifth implementation manner of the first aspect, the training process of the lane line recognition model includes:
[0028] Obtain a lane line image training set; each lane line training image in the lane line image training set has lane line annotation information;
[0029] Based on the lane line image training set, the initial lane line recognition network is trained to generate a lane line recognition model.
[0030] The lane line recognition method provided by the embodiment of the present invention obtains a lane line image training set; each lane line training image in the lane line image training set has lane line annotation information; based on the lane line image training set, an initial lane line recognition network is trained to generate a lane line recognition model, thereby ensuring the accuracy of the trained lane line recognition model.
[0031] In combination with the fifth implementation of the first aspect, in the sixth implementation of the first aspect, the lane line image to be identified is segmented along a preset direction according to the size of the lane line image to be identified to generate the first image and the second image, including:
[0032] According to the size of the lane line image to be identified, a horizontal median line corresponding to the lane line image to be identified is obtained;
[0033] The lane line image to be identified is segmented along the horizontal median line to generate a first image and a second image.
[0034] The lane line recognition method provided by the embodiment of the present invention obtains the horizontal median line corresponding to the lane line image to be recognized according to the size of the lane line image to be recognized, thereby ensuring the accuracy of the horizontal median line corresponding to the lane line image to be recognized. Then, the lane line image to be recognized is segmented along the horizontal median line to generate a first image and a second image. The lane line image included in the generated first image is smaller and less clear, and the lane line image included in the generated second image is larger and clearer, thereby achieving the division of the lane line image to be recognized according to the clarity of the lane line image in the lane line image to be recognized, thereby ensuring the accuracy of the generated first image and the second image.
[0035] According to the second aspect, an embodiment of the present invention further provides a lane line recognition device, comprising:
[0036] An acquisition module, used to acquire a lane line image to be identified;
[0037] A segmentation module, used to segment the lane line image to be identified along a preset direction according to the size of the lane line image to be identified, to generate a first image and a second image; the first image is located above the second image;
[0038] A processing module, used for processing the resolution of the first image to generate a third image, wherein the resolution of the third image is higher than that of the first image;
[0039] The recognition module is used to recognize the lane lines in the first image and the second image respectively, and to splice the recognition results to generate a lane line recognition result corresponding to the lane line image to be recognized.
[0040] The lane line recognition device provided by the embodiment of the present invention obtains a lane line image to be recognized, and then divides the lane line image to be recognized along a preset direction according to the size of the lane line image to be recognized, and generates a first image and a second image, wherein the first image is located above the second image, so that the lane line image included in the generated first image can be guaranteed to be smaller and less clear, and the lane line image included in the generated second image can be guaranteed to be larger and clearer. Then, the resolution of the first image is processed to generate a third image, and the resolution of the third image is higher than that of the first image, so that the lane line image included in the generated third image is guaranteed to be clearer. Then, the lane lines in the third image and the second image are respectively recognized, and the recognition results are spliced to generate a lane line recognition result corresponding to the lane line image to be recognized. Thereby, the accuracy of the lane line recognition results of the second image and the third image is guaranteed, and then the accuracy of the lane line recognition result corresponding to the generated lane line image to be recognized is guaranteed. In the above method, the lane line image to be recognized is divided, and then the resolution of the first image located above the lane line image to be recognized is processed, so as to improve the resolution of the generated third image. This ensures the accuracy of the lane line recognition result corresponding to the generated lane line image to be recognized, rather than directly performing lane line recognition on the lane line image to be recognized.
[0041] According to the third aspect, an embodiment of the present invention provides an electronic device, including a memory and a processor, the memory and the processor are communicatively connected to each other, computer instructions are stored in the memory, and the processor executes the lane line recognition method in the first aspect or any one of the embodiments of the first aspect by executing the computer instructions.
[0042] According to a fourth aspect, an embodiment of the present invention provides a computer-readable storage medium, which stores computer instructions, and the computer instructions are used to enable a computer to execute the lane line recognition method in the first aspect or any one of the embodiments of the first aspect. BRIEF DESCRIPTION OF THE DRAWINGS
[0043] In order to more clearly illustrate the specific implementation methods of the present invention or the technical solutions in the prior art, the drawings required for use in the specific implementation methods or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some implementation methods of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.
[0044] Figure 1 is a flow chart of a lane line recognition method provided by an embodiment of the present invention;
[0045] Figure 2 is a schematic diagram of a super-resolution model structure provided by another embodiment of the present invention;
[0046] Figure 3 is a schematic diagram of performing resolution processing on a first image provided by another embodiment of the present invention;
[0047] Figure 4 is a flow chart of a lane line recognition method provided by another embodiment of the present invention;
[0048] Figure 5 is a schematic diagram of generating a compressed image provided by another embodiment of the present invention;
[0049] Figure 6 is a schematic diagram of an image before and after degradation provided by another embodiment of the present invention;
[0050] Figure 7 is a flow chart of a lane line recognition method provided by another embodiment of the present invention;
[0051] Figure 8 is a flow chart of a lane line recognition method provided by another embodiment of the present invention;
[0052] Fig. 9 It is a functional module diagram of a lane line recognition device provided by an embodiment of the present invention;
[0053] Fig.10 It is a schematic diagram of the hardware structure of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION
[0054] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present invention.
[0055] It should be noted that the lane line recognition method provided in the embodiment of the present application may be executed by a lane line recognition device, which may be implemented by software, hardware, or a combination of software and hardware, and may be part or all of an electronic device, wherein the electronic device may be a control device in an intelligent driving vehicle, or may be a terminal device or server device connected to the intelligent driving vehicle but independent of the intelligent driving vehicle, wherein the server in the embodiment of the present application may be a single server or a server cluster composed of multiple servers, and the terminal in the embodiment of the present application may be a smart phone, a personal computer, a tablet computer, a wearable device, an intelligent robot, or other intelligent hardware devices. In the following method embodiments, the execution subject is an electronic device as an example for explanation.
[0056] In one embodiment of the present application, Figure 1 As shown, a lane line recognition method is provided, and the method is applied to an electronic device as an example for description, and includes the following steps:
[0057] S11, obtaining a lane line image to be identified.
[0058] Specifically, the electronic device can receive the lane line image to be identified collected by the visual sensor such as the camera device mounted on the intelligent driving vehicle based on the connection with the visual sensor such as the camera device. For example, the electronic device is part of the central computing platform of the intelligent driving system, and a communication connection is directly established between the camera and the computing platform.
[0059] S12. Segment the lane line image to be identified along a preset direction according to the size of the lane line image to be identified, to generate a first image and a second image.
[0060] When the preset direction is the horizontal direction, the first image is located above the second image.
[0061] In an optional implementation of the present application, a segmentation line corresponding to the lane line image to be identified is obtained according to the size of the lane line image to be identified, and then the lane line image to be identified is segmented along the direction of the segmentation line to generate a first image and a second image.
[0062] Exemplarily, the electronic device can obtain a dividing line that divides the lane line image to be identified into three equal parts in the horizontal direction according to the size of the lane line image to be identified, and divide the lane line image to be identified along the direction of the dividing line, with the upper two-thirds of the lane line image to be identified as the first image, and the lower one-third of the lane line image to be identified as the second image.
[0063] This step will be described in detail below.
[0064] S13: Process the resolution of the first image to generate a third image.
[0065] The resolution of the third image is higher than that of the first image.
[0066] In an optional implementation of the present application, the electronic device may process the resolution of the first image using image processing technology to generate a third image.
[0067] Among them, the image processing technology can be image enhancement processing, image sharpening processing, etc. The embodiment of the present application does not specifically limit the image processing technology.
[0068] This step will be described in detail below.
[0069] S14, respectively identify the lane lines in the third image and the second image, and concatenate the identification results to generate a lane line recognition result corresponding to the lane line image to be identified.
[0070] In an optional implementation of the present application, the electronic device may utilize image recognition technology to respectively identify the lane lines in the third image and the second image, and splice the recognition results to generate a lane line recognition result corresponding to the lane line image to be recognized.
[0071] This step will be described in detail below.
[0072] The lane line recognition method provided by the embodiment of the present invention obtains a lane line image to be recognized, and then divides the lane line image to be recognized along a preset direction according to the size of the lane line image to be recognized, and generates a first image and a second image, wherein the first image is located above the second image, so that the lane line image included in the generated first image can be guaranteed to be smaller and less clear, and the lane line image included in the generated second image can be guaranteed to be larger and clearer. Then, the resolution of the first image is processed to generate a third image, and the resolution of the third image is higher than that of the first image, so that the lane line image included in the generated third image is guaranteed to be clearer. Then, the lane lines in the third image and the second image are respectively recognized, and the recognition results are spliced to generate a lane line recognition result corresponding to the lane line image to be recognized. Thereby, the accuracy of the lane line recognition results of the second image and the third image is guaranteed, and then the accuracy of the lane line recognition result corresponding to the lane line image to be recognized is guaranteed. In the above method, the lane line image to be recognized is divided, and then the resolution of the first image located above the lane line image to be recognized is processed to generate a third image with improved resolution. This ensures the accuracy of the lane line recognition result corresponding to the generated lane line image to be recognized, rather than directly performing lane line recognition on the lane line image to be recognized.
[0073] In an optional embodiment of the present application, in the above step S13, “processing the resolution of the first image to generate a third image” may include the following contents:
[0074] The first image is input into the super-resolution model, and the super-resolution model extracts features from the first image and outputs a third image.
[0075] Specifically, the electronic device may input the first image into a super-resolution model, the super-resolution model extracts features from the first image, and then outputs a third image based on the extracted features.
[0076] Among them, the super-resolution model can be a machine learning network model, and of course it can also be other network models. The machine learning network model can be DNN (Deep Neural Networks), CNN (Convolutional Neural Networks), RNN (Recurrent Neural Network), etc. When the super-resolution model is CNN, it can be a V-Net model, U-Net model, Generative Adversarial Nets model, etc. The embodiment of the present application does not specifically limit the type of super-resolution model. For example, the structural diagram of the super-resolution model can be as follows Figure 2 The structures of the models are shown in Figure 2.
[0077] For example, Figure 3 As shown, it is a schematic diagram of an electronic device inputting a first image into a super-resolution model and outputting a third image.
[0078] The lane line recognition method provided by the embodiment of the present invention inputs the first image into the super-resolution model, and the super-resolution model extracts features from the first image and outputs a third image. The resolution of the output third image is ensured to be high, thereby ensuring the accuracy of the lane line recognition result obtained by performing lane line recognition on the third image.
[0079] In an optional embodiment of the present application, Figure 4 As shown, the training process of the super-resolution model can include the following steps:
[0080] S21. Obtain a super-resolution image training set.
[0081] The super-resolution image training set is generated based on preset images in each lane line image in the lane line image set.
[0082] The preset image is an image located below each lane line image that is generated after each lane line image is segmented along a preset direction.
[0083] In an optional implementation of the present application, the electronic device may receive a super-resolution image training set sent by other devices, or may receive a super-resolution image training set input by a user.
[0084] In another optional implementation manner of the present application, the above S21 “obtaining a super-resolution image training set” may include the following steps:
[0085] S211. Obtain a lane line image set.
[0086] Specifically, the electronic device can receive lane line images captured by visual sensors such as cameras in smart driving vehicles and process and store them into lane line image sets based on the connection with visual sensors such as cameras in smart driving vehicles; the electronic device can also receive lane line image sets sent by other devices, such as lane line image sets uploaded by cloud servers; the electronic device can also receive lane line image sets input by users. The embodiments of the present application do not specifically limit the manner in which the electronic device obtains the lane line image sets.
[0087] S212: For each lane line image in the lane line image set, segment the lane line image along a preset direction to generate a first training image and a second training image.
[0088] The first training image is located above the second training image.
[0089] Specifically, for each lane line image in the lane line image set, the electronic device can rotate the lane line image so that the lane line is perpendicular to the horizontal plane, obtain the median line in the direction perpendicular to the lane line corresponding to each lane line image, and then segment each lane line image along the median line in the direction perpendicular to the lane line to generate a first training image and a second training image.
[0090] S213: compress each second training image to generate a compressed image.
[0091] wherein the resolution of the compressed image is lower than that of the second training image;
[0092] In an optional implementation direction of the present application, the electronic device may perform compression processing on each second training image to generate a compressed image.
[0093] Exemplarily, the electronic device may compress each second training image to 1 / 4 of the original size, thereby generating a compressed image.
[0094] For example, Figure 5 As shown, the electronic device will obtain the median line in the direction perpendicular to the lane line corresponding to each lane line image, and then segment each lane line image along the median line in the direction perpendicular to the lane line to generate a first training image and a second training image. The second training image is a "close" image, and the second training image is compressed to generate a compressed image, so that the resolution of the generated compressed image is lower.
[0095] S214 , using each compressed image as a training image and a second training image corresponding to each compressed image as annotation information corresponding to the training image to generate a super-resolution image training set.
[0096] Specifically, the electronic device uses each compressed image as a training image and the second training image corresponding to each compressed image as the annotation information corresponding to the training image to generate a super-resolution image training set. In other words, each compressed image is used as a low-resolution image and the second training image corresponding to each compressed image is used as a high-resolution image to generate a super-resolution image training set.
[0097] In another optional implementation of the present application, the electronic device may also utilize a degradation technology to perform degradation processing on the second training image to generate a compressed image.
[0098] For example, Figure 6 The following is a schematic diagram of the image before and after degradation processing.
[0099] In another optional implementation of the present application, the electronic device may also perform resolution improvement processing on the first training images in each lane line image to generate a high-resolution image corresponding to each first training image, and then generate a super-resolution image training set based on each first training image and the high-resolution image corresponding to each first training image.
[0100] S22. Based on the super-resolution image training set, train an initial super-resolution network to generate a super-resolution model.
[0101] Specifically, the electronic device can train an initial super-resolution network based on a super-resolution image training set to generate a super-resolution model.
[0102] Among them, the super-resolution model can be a machine learning network model, and of course it can also be other network models. The machine learning network model can be DNN (Deep Neural Networks), CNN (Convolutional Neural Networks), RNN (Recurrent Neural Network), etc. When the super-resolution model is CNN, it can be a V-Net model, U-Net model, Generative Adversarial Nets model, etc. The embodiment of the present application does not specifically limit the type of super-resolution model.
[0103] Furthermore, when training the super-resolution model, the Adam optimizer can be selected to optimize the super-resolution model, so that the super-resolution model can converge quickly and have good generalization ability.
[0104] When the Adam optimizer is used to optimize the super-resolution model, a learning rate can also be set for the optimizer. Here, the learning rate range test (LR Range Test) technique can be used to select the best learning rate and set it to the optimizer. The learning rate selection process of this test technique is as follows: first, the learning rate is set to a very small value, and then the super-resolution model and the super-resolution image training set are simply iterated several times. After each iteration, the learning rate is increased, and the training loss (loss) is recorded each time. Then the LR Range Test graph is drawn. Generally, the ideal LR Range Test graph contains three areas: the first area has a learning rate that is too small and the loss remains basically unchanged, the second area has a loss that decreases and converges quickly, and the last area has a learning rate that is too large so that the loss begins to diverge. Then the learning rate corresponding to the lowest point in the LR Range Test graph can be used as the best learning rate, and the best learning rate can be used as the initial learning rate of the Adam optimizer and set to the optimizer.
[0105] The lane line recognition method provided by the embodiment of the present invention obtains a lane line image set, and for each lane line image in the lane line image set, the lane line image is segmented along a preset direction to generate a first training image and a second training image; the first training image is located above the second training image; it is ensured that the lane line image included in the generated first training image is smaller and less clear, and the lane line image included in the generated second training image is larger and clearer. Then, each second training image is compressed to generate a compressed image, and the resolution of the generated compressed image is ensured to be low. Then, each compressed image is used as a training image, and the second training image corresponding to each compressed image is used as the annotation information corresponding to the training image to generate a super-resolution image training set, thereby ensuring the accuracy of the generated super-resolution image training set, and further ensuring the accuracy of the super-resolution model obtained by training based on the super-resolution image training set. The above-mentioned resolution image training set does not require manual annotation, and the acquired lane line image data can be directly compressed, so that the labor cost is not increased, etc., and a large amount of data is helpful to generate a better model. A super-resolution model with better accuracy can generate clearer distant data. Then, based on the super-resolution image training set, an initial super-resolution network is trained to generate a super-resolution model. This ensures that the generated super-resolution model is more suitable for the first image in the lane line image to be identified, thereby ensuring that when the first image is input into the super-resolution model, the resolution of the lane line image in the output third image is higher, thereby ensuring the accuracy of the lane line recognition result obtained by performing lane line recognition on the third image.
[0106] In an optional embodiment of the present application, the “recognizing lane lines in the third image and the second image respectively” in the above S14 may include the following steps:
[0107] The third image and the second image are input into the lane line recognition model, and the lane line recognition results corresponding to the third image and the second image are output respectively.
[0108] Specifically, the electronic device can input the third image and the second image into the lane line recognition model, the lane line recognition model extracts features from the third image and the second image, and then outputs the lane line recognition results corresponding to the third image and the second image respectively based on the extracted features.
[0109] The lane line recognition model can be a machine learning network model, or other network models. The machine learning network model can be DNN (Deep Neural Networks), CNN (Convolutional Neural Networks), RNN (Recurrent Neural Network), etc. When the lane line recognition model is CNN, it can be a V-Net model, U-Net model, Generative Adversarial Nets model, etc. The embodiment of the present application does not specifically limit the type of lane line recognition model.
[0110] The lane line recognition method provided by the embodiment of the present invention inputs the third image and the second image into the lane line recognition model, and outputs the lane line recognition results corresponding to the third image and the second image respectively, thereby ensuring the accuracy of the outputted lane line recognition results corresponding to the third image and the second image.
[0111] In an optional embodiment of the present application, Figure 7 As shown, the training process of the above lane line recognition model includes:
[0112] S31. Obtain a lane line image training set.
[0113] Among them, each lane line training image in the lane line image training set carries lane line annotation information.
[0114] Specifically, the electronic device can receive a lane line image training set collected by a visual sensor such as a camera device in a smart driving vehicle based on a connection with the visual sensor. The electronic device can also receive a lane line image training set sent by other devices, such as a lane line image set uploaded by a cloud server. The electronic device can also receive a lane line image training set input by a user. The embodiment of the present application does not specifically limit the manner in which the electronic device obtains the lane line image training set.
[0115] S32. Based on the lane line image training set, train an initial lane line recognition network to generate a lane line recognition model.
[0116] Specifically, the electronic device can train an initial lane line recognition network based on the acquired lane line image training set to generate a lane line recognition model.
[0117] Furthermore, when training the lane line recognition model, the Adam optimizer can be selected to optimize the lane line recognition model, so that the lane line recognition model can converge quickly and have good generalization ability.
[0118] When the lane recognition model is optimized using the Adam optimizer, a learning rate can also be set for the optimizer. Here, the learning rate range test (LR Range Test) technique can be used to select the best learning rate and set it to the optimizer. The learning rate selection process of this test technique is as follows: first, the learning rate is set to a very small value, then the lane recognition model and the lane image training set are simply iterated several times, the learning rate is increased after each iteration, and the training loss (loss) is recorded each time, and then the LR Range Test graph is drawn. Generally, the ideal LR Range Test graph contains three regions: the first region has a learning rate that is too small and the loss remains basically unchanged, the second region has a loss that decreases and converges quickly, and the last region has a learning rate that is too large so that the loss begins to diverge. In this case, the learning rate corresponding to the lowest point in the LR Range Test graph can be used as the best learning rate, and the best learning rate can be used as the initial learning rate of the Adam optimizer and set to the optimizer.
[0119] The lane line recognition method provided by the embodiment of the present invention obtains a lane line image training set; each lane line training image in the lane line image training set has lane line annotation information; based on the lane line image training set, an initial lane line recognition network is trained to generate a lane line recognition model, thereby ensuring the accuracy of the trained lane line recognition model.
[0120] In one embodiment of the present application, Figure 8 As shown, a lane line recognition method is provided, and the method is applied to an electronic device as an example for description, and includes the following steps:
[0121] S41, obtaining a lane line image to be identified.
[0122] For more information about this step, see Figure 1 The introduction of S11 will not be elaborated here.
[0123] S42, segmenting the lane line image to be identified along a preset direction according to the size of the lane line image to be identified, to generate a first image and a second image.
[0124] When the preset direction is the horizontal direction, the first image is located above the second image.
[0125] In an optional implementation of the present application, the above-mentioned S21 of “segmenting the lane line image to be identified along a preset direction according to the size of the lane line image to be identified to generate the first image and the second image” may include the following steps:
[0126] S421. Obtain a horizontal median line corresponding to the lane line image to be identified according to the size of the lane line image to be identified.
[0127] Specifically, after obtaining the size of the lane line image to be identified, the electronic device can calculate the horizontal median line corresponding to the lane line image to be identified based on the size of the lane line image to be identified, thereby obtaining the horizontal median line corresponding to the lane line image to be identified.
[0128] S422, segmenting the lane line image to be identified along the horizontal median line to generate a first image and a second image.
[0129] Specifically, the electronic device divides the lane line image to be identified along the horizontal median line to generate a first image and a second image.
[0130] S43: Process the resolution of the first image to generate a third image.
[0131] The resolution of the third image is higher than that of the first image.
[0132] For more information about this step, see Figure 1 The introduction of S13 will not be elaborated here.
[0133] S44, respectively identify the lane lines in the third image and the second image, and concatenate the identification results to generate a lane line recognition result corresponding to the lane line image to be identified.
[0134] For more information about this step, see Figure 1 The introduction of S14 will not be elaborated here.
[0135] The lane line recognition method provided by the embodiment of the present invention obtains the horizontal median line corresponding to the lane line image to be recognized according to the size of the lane line image to be recognized, thereby ensuring the accuracy of the horizontal median line corresponding to the lane line image to be recognized. Then, the lane line image to be recognized is segmented along the horizontal median line to generate a first image and a second image. The lane line image included in the generated first image is smaller and less clear, and the lane line image included in the generated second image is larger and clearer, thereby achieving the division of the lane line image to be recognized according to the clarity of the lane line image in the lane line image to be recognized, thereby ensuring the accuracy of the generated first image and the second image.
[0136] It should be understood that although Figure 1 , Figure 4 , Figure 7 as well as Figure 8 The steps in the flowchart are shown in sequence as indicated by the arrows, but these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified in this document, there is no strict order restriction for the execution of these steps, and these steps can be executed in other orders. Moreover, Figure 1 , Figure 4 , Figure 7 as well as Figure 8 At least part of the steps may include multiple steps or multiple stages. These steps or stages are not necessarily performed at the same time, but can be performed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed in turn or alternately with other steps or at least part of the steps or stages in other steps.
[0137] like Fig. 9 As shown, this embodiment provides a lane line recognition device, including:
[0138] An acquisition module 51 is used to acquire a lane line image to be identified;
[0139] The segmentation module 52 is used to segment the lane line image to be identified along a preset direction according to the size of the lane line image to be identified, and generate a first image and a second image; the first image is located above the second image;
[0140] A processing module 53, configured to process the resolution of the first image to generate a third image, wherein the resolution of the third image is higher than that of the first image;
[0141] The recognition module 54 is used to recognize the lane lines in the first image and the second image respectively, and to combine the recognition results to generate a lane line recognition result corresponding to the lane line image to be recognized.
[0142] In one embodiment of the present application, the processing module 53 is specifically used to input the first image into a super-resolution model, and the super-resolution model extracts features from the first image and outputs a third image.
[0143] In one embodiment of the present application, the above-mentioned processing module 53 is specifically used to obtain a super-resolution image training set; the super-resolution image training set is generated based on preset images in each lane line image in the lane line image set; based on the super-resolution image training set, an initial super-resolution network is trained to generate a super-resolution model.
[0144] In one embodiment of the present application, the above-mentioned processing module 53 is specifically used to obtain a lane line image set; for each lane line image in the lane line image set, the lane line image is segmented along a preset direction to generate a first training image and a second training image; the first training image is located above the second training image; each second training image is compressed to generate a compressed image; the resolution of the compressed image is lower than that of the second training image; each compressed image is used as a training image, and the second training image corresponding to each compressed image is used as the annotation information corresponding to the training image to generate a super-resolution image training set.
[0145] In one embodiment of the present application, the above-mentioned recognition module 54 is specifically used to input the third image and the second image into the lane line recognition model, and output the lane line recognition results corresponding to the third image and the second image respectively.
[0146] In one embodiment of the present application, the above-mentioned recognition module 54 is specifically used to obtain a lane line image training set; each lane line training image in the lane line image training set carries lane line annotation information; based on the lane line image training set, an initial lane line recognition network is trained to generate a lane line recognition model.
[0147] In one embodiment of the present application, the segmentation module 52 is specifically used to obtain the horizontal median line corresponding to the lane line image to be identified according to the size of the lane line image to be identified; and segment the lane line image to be identified along the horizontal median line to generate a first image and a second image.
[0148] For the specific limitations and beneficial effects of the lane line recognition device, please refer to the limitations of the lane line recognition method above, which will not be repeated here. Each module in the above lane line recognition device can be implemented in whole or in part by software, hardware and a combination thereof. The above modules can be embedded in or independent of the processor in the electronic device in the form of hardware, or can be stored in the memory of the electronic device in the form of software, so that the processor can call and execute the operations corresponding to the above modules.
[0149] The embodiment of the present invention also provides an electronic device having the above Fig. 9 The lane line recognition device shown.
[0150] like Fig.10 As shown, Fig.10 is a schematic diagram of the structure of an electronic device provided by an optional embodiment of the present invention, such as Fig.10 As shown, the electronic device may include: at least one processor 61, such as a CPU (Central Processing Unit), at least one communication interface 63, a memory 64, and at least one communication bus 62. The communication bus 62 is used to realize the connection and communication between these components. The communication interface 63 may include a display screen (Display), a keyboard (Keyboard), and the optional communication interface 63 may also include a standard wired interface and a wireless interface. The memory 64 may be a high-speed RAM memory (Random Access Memory) or a non-volatile memory (non-volatile memory), such as at least one disk storage. The memory 64 may optionally be at least one storage device located away from the aforementioned processor 61. The processor 61 may be combined with Fig. 9In the described device, the memory 64 stores an application program, and the processor 61 calls the program code stored in the memory 64 to execute any of the above method steps.
[0151] The communication bus 62 may be a peripheral component interconnect (PCI) bus or an extended industry standard architecture (EISA) bus. The communication bus 62 may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Fig.10 Only one thick line is used in the diagram, but this does not mean that there is only one bus or only one type of bus.
[0152] Among them, the memory 64 may include a volatile memory (English: volatile memory), such as a random access memory (English: random-access memory, abbreviated: RAM); the memory may also include a non-volatile memory (English: non-volatile memory), such as a flash memory (English: flash memory), a hard disk drive (English: hard disk drive, abbreviated: HDD) or a solid-state drive (English: solid-state drive, abbreviated: SSD); the memory 64 may also include a combination of the above types of memory.
[0153] The processor 61 may be a central processing unit (CPU), a network processor (NP), or a combination of a CPU and a NP.
[0154] The processor 61 may further include a hardware chip. The hardware chip may be an application-specific integrated circuit (ASIC), a programmable logic device (PLD), or a combination thereof. The PLD may be a complex programmable logic device (CPLD), a field-programmable gate array (FPGA), a generic array logic (GAL), or any combination thereof.
[0155] Optionally, the memory 64 is also used to store program instructions. The processor 61 can call the program instructions to implement the present application Figure 1 , Figure 4 , Figure 7 as well as Figure 8 The lane line recognition method shown in the embodiment.
[0156] The embodiment of the present invention further provides a non-transitory computer storage medium, which stores computer executable instructions, and the computer executable instructions can execute the lane line recognition method in any of the above method embodiments. The storage medium can be a disk, an optical disk, a read-only memory (ROM), a random access memory (RAM), a flash memory, a hard disk drive (HDD) or a solid-state drive (SSD), etc.; the storage medium can also include a combination of the above types of memory.
[0157] Although the embodiments of the present invention have been described in conjunction with the accompanying drawings, those skilled in the art may make various modifications and variations without departing from the spirit and scope of the present invention, and such modifications and variations are all within the scope defined by the appended claims.
Claims
1. A lane line recognition method, characterized in that: include: Obtaining a lane line image to be identified; According to the size of the lane line image to be identified, segment the lane line image to be identified along a preset direction to generate a first image and a second image; The first image is located above the second image; Processing the resolution of the first image to generate a third image, wherein the resolution of the third image is higher than that of the first image; Respectively identifying the lane lines in the third image and the second image, and splicing the identification results to generate a lane line recognition result corresponding to the lane line image to be identified; The step of processing the resolution of the first image to generate a third image includes: Inputting the first image into a super-resolution model, wherein the super-resolution model extracts features from the first image and outputs the third image; The training process of the super-resolution model includes: Obtaining a super-resolution image training set; the super-resolution image training set is generated based on a preset image of each lane line image in a lane line image set; Based on the super-resolution image training set, an initial super-resolution network is trained to generate the super-resolution model.
2. The method according to claim 1, characterized in that The step of obtaining a super-resolution image training set comprises: Get a lane line image set; For each lane line image in the lane line image set, segment the lane line image along a preset direction to generate a first training image and a second training image; the first training image is located above the second training image; compressing each of the second training images to generate a compressed image; the compressed image has a lower resolution than the second training image; The super-resolution image training set is generated by using each of the compressed images as a training image and a second training image corresponding to each of the compressed images as annotation information corresponding to the training image.
3. The method according to claim 1, characterized in that: The identifying the lane lines in the third image and the second image respectively includes: The third image and the second image are input into a lane line recognition model, and lane line recognition results corresponding to the third image and the second image are output respectively.
4. The method according to claim 3, characterized in that The training process of the lane line recognition model includes: Acquire a lane line image training set; each lane line training image in the lane line image training set has lane line annotation information; Based on the lane line image training set, an initial lane line recognition network is trained to generate the lane line recognition model.
5. The method according to claim 1, characterized in that The step of segmenting the lane line image to be identified along a preset direction according to the size of the lane line image to be identified to generate a first image and a second image includes: According to the size of the lane line image to be identified, obtaining a horizontal median line corresponding to the lane line image to be identified; The lane line image to be identified is segmented along the horizontal median line to generate the first image and the second image.
6. A lane line recognition device, characterized in that: include: An acquisition module, used to acquire a lane line image to be identified; A segmentation module, configured to segment the lane line image to be identified along a preset direction according to the size of the lane line image to be identified, to generate a first image and a second image; the first image is located above the second image; A processing module, configured to process the resolution of the first image to generate a third image, wherein the resolution of the third image is higher than that of the first image; wherein the processing of the resolution of the first image to generate the third image comprises: inputting the first image into a super-resolution model, wherein the super-resolution model extracts features from the first image, and outputs the third image; wherein the training process of the super-resolution model comprises: obtaining a super-resolution image training set; wherein the super-resolution image training set is generated based on preset images in each lane line image in a lane line image set; and based on the super-resolution image training set, training an initial super-resolution network to generate the super-resolution model; The recognition module is used to recognize the lane lines in the first image and the second image respectively, and to splice the recognition results to generate a lane line recognition result corresponding to the lane line image to be recognized.
7. An electronic device, characterized in that: The invention comprises a memory and a processor, wherein the memory stores computer instructions, and the processor executes the lane line recognition method according to any one of claims 1 to 5 by executing the computer instructions.
8. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a computer to execute the lane line recognition method according to any one of claims 1 to 5.
Citation Information
Patent Citations
Image recognition method and device and computer-readable storage medium
CN107944351A
Text recognition method and device, electronic equipment and storage medium
CN111461070A
Image super-resolution processing method and device, electronic equipment and storage medium
CN111598779A
Lane line recognition method and device, computer equipment and storage medium
CN112241646A
Vehicle parking detection method and device, terminal equipment and readable storage medium
CN114445788A
