A vehicle lane recognition system, method and storage medium based on vehicle-mounted camera

By installing a camera on the vehicle to collect and process the front and rear road images, forming a road model and identifying lane lines, the problem of inaccurate lane line identification in the prior art is solved, and high-reliability roadway recognition is achieved.

CN116977968BActive Publication Date: 2024-11-08HEBEI HAIKANG BOSHI TECH CO LTD

Patent Information

Application Number
CN202311003935.0
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2023-08-10
Publication Date
2024-11-08
Estimated Expiration
2043-08-10

AI Technical Summary

Technical Problem

In the prior art, the data structure of the lane identification system is simple and cannot ensure the accuracy of lane line identification, especially when the road conditions ahead and behind the vehicle are complex.

Method used

The roadway recognition system based on the vehicle camera is adopted. The roadway ahead and rear of the vehicle is collected through the image acquisition module, the image processing module performs image processing and edge extraction, the road condition fitting module forms a road model, and marks it in the roadway recognition module. Lane line interval.

Benefits of technology

It improves the accuracy and reliability of lane line identification, reduces the system operation pressure, and provides users with fast and accurate roadway information guarantees.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN116977968B_ABST
    Figure CN116977968B_ABST
Patent Text Reader

Abstract

The present application discloses a lane recognition system, method and storage medium based on a vehicle-mounted camera, and relates to the field of image processing technology. The system includes an image acquisition module configured to acquire images of at least the road in front of the vehicle and the road behind the vehicle through a vehicle-mounted camera, an image processing module configured to perform image processing on the acquired road images, a road condition fitting module configured to form a road model by fitting the processed images corresponding to the front of the vehicle and the processed images corresponding to the rear of the vehicle based on the image processing module, and a lane recognition module configured to mark and display the lane line interval where the vehicle is located based on the processed images corresponding to the front of the vehicle and the road model corresponding to the image processing module. The method is applicable to the above-mentioned system, and the storage medium stores a computer program that implements the method when executed by a processor. The present application can provide lane conditions to users intuitively and quickly, and has high reliability.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and specifically to a vehicle lane recognition system, method and storage medium based on a vehicle-mounted camera. Background Art

[0002] Lane recognition is a research hotspot in the field of safe driving. Auxiliary equipment that can identify lanes and issue warnings in the event of lane deviation can effectively reduce the occurrence of traffic accidents and protect the safety of drivers and pedestrians. In the existing technology, front cameras are often used to detect the lane lines in front of the vehicle, and the vehicle is corrected and controlled based on the detection results. However, this method has a relatively simple data structure and cannot ensure the accuracy of lane line recognition. Summary of the invention

[0003] The purpose of the present application is to provide a lane recognition system, method and storage medium based on a vehicle-mounted camera to solve the problems existing in the prior art mentioned in the above background technology.

[0004] To achieve the above objectives, this application discloses the following technical solutions:

[0005] In a first aspect, the present application provides a lane recognition system based on a vehicle-mounted camera, including an image acquisition module, an image processing module, a road condition fitting module, and a lane recognition module;

[0006] The image acquisition module is configured to: acquire images of at least the road in front of the vehicle and the road behind the vehicle through the vehicle-mounted camera;

[0007] The image processing module is configured to: perform image processing on the collected road image to enhance the display effect of the lane lines in the image;

[0008] The road condition fitting module is configured to: form a road model based on the image processed by the image processing module for the image in front of the vehicle and the image processed for the image behind the vehicle;

[0009] The lane recognition module is configured to mark and display the lane line section where the vehicle is located based on the image processed by the image processing module corresponding to the front of the vehicle and the road model.

[0010] In one implementation, the capturing of images of at least the road in front of the vehicle and the road behind the vehicle specifically includes:

[0011] The road in front of the vehicle is captured by a vehicle-mounted camera 1 located in front of the vehicle, and the captured image 1 includes the distribution of lane lines;

[0012] The road behind the vehicle is imaged by a second on-vehicle camera located at the rear of the vehicle, and the image collected includes the distribution of lane lines.

[0013] In one implementation, the image processing of the collected road image includes:

[0014] Performing image segmentation on the elements in the first image and the second image by image normalization processing to obtain segmented elements;

[0015] Performing edge extraction on the segmented elements to obtain element edges;

[0016] Comparing the element edge with a preset lane line edge feature to identify the lane line in the image, wherein the lane line edge feature includes at least a straight lane line edge feature and a curved lane line edge feature;

[0017] The identified lane lines are deepened in the image 1 and the image 2 to obtain the auxiliary image 1 and the auxiliary image 2 in which the lane lines are highlighted.

[0018] In one embodiment, the deepening process specifically includes:

[0019] Identify the main color of the identified lane line in the corresponding image, where the main color is the color of the lane line with the largest proportion in the image;

[0020] The outline of the identified lane line in the corresponding image is processed with the main color as a pure color; the content other than the identified lane line outline in the corresponding image is defined as a background element, and the background element is blurred.

[0021] In one implementation, the collected image also includes the road extension layout, and the image processing of the collected road image also includes:

[0022] Performing image segmentation on the elements in the image 1 by image normalization processing to obtain the segmented elements;

[0023] Performing edge extraction on the segmented elements to obtain element edges;

[0024] Comparing the edge of the element with a preset road feature to identify a road extension element in the image, wherein the road extension element includes at least a road fence and a curb;

[0025] The identified road extension elements are deepened in the image one to obtain a second auxiliary image one with a highlighted path, and the same road extension elements are connected as a road layout path.

[0026] In one embodiment, the connecting of the same road extension elements along the road extension direction as a road layout path specifically includes:

[0027] Extracting edges of all the road extension elements along the vehicle travel direction;

[0028] Project all extracted edges vertically onto the same horizontal plane;

[0029] A continuous curve is generated on the horizontal plane as a path line.

[0030] In one embodiment, generating a continuous curve on the horizontal plane as a path line specifically includes:

[0031] defining a plurality of edge vertical projections on the horizontal plane as edge projection line 1, edge projection line 2, ..., edge projection line N in sequence;

[0032] When all edge projection lines do not overlap, all edge projection lines are fitted into a continuous curve as a path line;

[0033] When the edge projection line i and the edge projection line i+1 have an overlapping area, calculate the continuity ρ1 of the edge projection line i and the edge projection line i-1, calculate the continuity ρ2 of the edge projection line i and the edge projection line i+2, calculate the continuity ρ3 of the edge projection line i+1 and the edge projection line i-1, calculate the continuity ρ4 of the edge projection line i+1 and the edge projection line i+2, and obtain the fitness W of the insulation projection line i and the fitness M of the edge projection line i+1, W = ρ1 + ρ2 + K1, M = ρ3 + ρ4 + K2, where K1 and K2 are both curvature compensation values. Compare the sizes of W and M. When W>M, select edge projection line i as the connecting line between edge projection line i-1 and edge projection line i+2. When W=M, select edge projection line i or edge projection line i+1 as the connecting line between edge projection line i-1 and edge projection line i+2. When W<M, select edge projection line i+1 as the connecting line between edge projection line i-1 and edge projection line i+2. Fit all edge projection lines and corresponding connecting lines into continuous curves as path lines.

[0034] In one embodiment, the forming of the road model by fitting the processed image corresponding to the front of the vehicle and the processed image corresponding to the rear of the vehicle based on the image processing module specifically includes:

[0035] The acquired path lines and lane lines are fused to generate a road model, and the lane lines are augmented in the road model.

[0036] In a third aspect, the present application provides a method for lane recognition based on a vehicle-mounted camera, the method comprising the following steps:

[0037] Configure the above-mentioned vehicle-mounted camera-based lane recognition system;

[0038] At least collecting images of the road in front of the vehicle and the road behind the vehicle through the vehicle-mounted camera;

[0039] The image processing module performs image processing on the collected road images to enhance the display effect of the lane lines in the images;

[0040] The road condition fitting module forms a road model based on the image processed by the image processing module for the image in front of the vehicle and the image processed for the image behind the vehicle;

[0041] The lane recognition module identifies and displays the lane line section where the vehicle is located based on the image processed by the image processing module corresponding to the front of the vehicle and the road model.

[0042] In a third aspect, the present application provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, it implements the lane recognition method based on a vehicle-mounted camera as described in claim 9.

[0043] Beneficial effects: The lane recognition system, method and storage medium based on the vehicle-mounted camera of the present application obtain a road model of the vehicle's current situation by analyzing the road conditions in front of and behind the vehicle, and identify and obtain the lane line interval in which the vehicle is located based on the image collected in front of the vehicle. The obtained results are highly reliable, the calculation and image processing steps are simple, the operating pressure of the system / vehicle computer is reduced, and a guarantee is provided for users to quickly and accurately obtain the lane. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For those skilled in the art, other drawings can be obtained based on these drawings without paying any creative work.

[0045] Figure 1 This is a structural block diagram of a lane recognition system based on a vehicle-mounted camera in an embodiment of the present application;

[0046] Figure 2 This is a flowchart of a lane recognition method based on a vehicle-mounted camera in an embodiment of the present application. DETAILED DESCRIPTION

[0047] The technical solutions in the embodiments of the present application are described clearly and completely below. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present application.

[0048] In this article, the term "comprising" is intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also includes other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of more restrictions, the elements defined by the sentence "comprising..." do not exclude the presence of other identical elements in the process, method, article or device including the elements.

[0049] In a first aspect, this embodiment discloses Figure 1 A lane recognition system based on a vehicle-mounted camera is shown, comprising an image acquisition module, an image processing module, a road condition fitting module and a lane recognition module.

[0050] Specifically, the image acquisition module is configured to acquire images of at least the road in front of the vehicle and the road behind the vehicle through a vehicle-mounted camera.

[0051] Specifically, the image processing module is configured to perform image processing on the collected road image to enhance the display effect of the lane lines in the image.

[0052] Specifically, the road condition fitting module is configured to form a road model based on the image processed by the image processing module to the image in front of the vehicle and the image processed to the rear of the vehicle.

[0053] Specifically, the lane recognition module is configured to mark and display the lane line section where the vehicle is located based on the image processed by the image processing module corresponding to the front of the vehicle and the road model.

[0054] The step of collecting images of at least the road in front of the vehicle and the road behind the vehicle specifically includes:

[0055] The road in front of the vehicle is captured by a vehicle-mounted camera 1 located in front of the vehicle, and the captured image 1 includes the distribution of lane lines;

[0056] The road behind the vehicle is imaged by a second on-vehicle camera located at the rear of the vehicle, and the image collected includes the distribution of lane lines.

[0057] Based on the acquisition of the above image, the image processing of the collected road image includes:

[0058] Performing image segmentation on the elements in the first image and the second image by image normalization processing to obtain segmented elements;

[0059] Performing edge extraction on the segmented elements to obtain element edges;

[0060] Comparing the element edge with a preset lane line edge feature to identify the lane line in the image, wherein the lane line edge feature includes at least a straight lane line edge feature and a curved lane line edge feature;

[0061] The identified lane lines are deepened in the image 1 and the image 2 to obtain the auxiliary image 1 and the auxiliary image 2 with the lane lines highlighted. The deepening process specifically includes:

[0062] Identify the main color of the identified lane line in the corresponding image, where the main color is the color of the lane line with the largest proportion in the image;

[0063] The outline of the identified lane line in the corresponding image is processed with the main color as a pure color; the content other than the identified lane line outline in the corresponding image is defined as a background element, and the background element is blurred.

[0064] By highlighting the lane lines as described above, the lane lines can be distinguished from all elements in the background image, thereby accurately identifying the lane lines and providing the user with intuitive lane line recognition results.

[0065] Based on the acquisition of the above image, when the front of the vehicle is not blocked by other vehicles or obstacles, the acquired image 1 also includes the road extension layout situation, and the road extension layout situation refers to the direction of the road. The image processing of the acquired road image also includes:

[0066] Performing image segmentation on the elements in the image 1 by image normalization processing to obtain the segmented elements;

[0067] Performing edge extraction on the segmented elements to obtain element edges;

[0068] Comparing the edge of the element with a preset road feature to identify a road extension element in the image, wherein the road extension element includes at least a road fence and a curb;

[0069] The identified road extension elements are deepened in the image 1 to obtain a second auxiliary image 1 with a highlighted path, and the same road extension elements are connected as a road layout path. The connecting of the same road extension elements along the road extension direction as a road layout path specifically includes:

[0070] Extracting edges of all the road extension elements along the vehicle travel direction;

[0071] Project all extracted edges vertically onto the same horizontal plane;

[0072] A continuous curve is generated on the horizontal plane as a path line.

[0073] Furthermore, the step of generating a continuous curve on the horizontal plane as a path line specifically includes:

[0074] defining a plurality of edge vertical projections on the horizontal plane as edge projection line 1, edge projection line 2, ..., edge projection line N in sequence;

[0075] When all edge projection lines do not overlap, all edge projection lines are fitted into a continuous curve as a path line;

[0076] When the edge projection line i and the edge projection line i+1 have an overlapping area, calculate the continuity ρ1 of the edge projection line i and the edge projection line i-1, calculate the continuity ρ2 of the edge projection line i and the edge projection line i+2, calculate the continuity ρ3 of the edge projection line i+1 and the edge projection line i-1, calculate the continuity ρ4 of the edge projection line i+1 and the edge projection line i+2, and obtain the fitness W of the insulation projection line i and the fitness M of the edge projection line i+1, W = ρ1 + ρ2 + K1, M = ρ3 + ρ4 + K2, where K1 and K2 are both curvature compensation values. Compare the sizes of W and M. When W>M, select edge projection line i as the connecting line between edge projection line i-1 and edge projection line i+2. When W=M, select edge projection line i or edge projection line i+1 as the connecting line between edge projection line i-1 and edge projection line i+2. When W<M, select edge projection line i+1 as the connecting line between edge projection line i-1 and edge projection line i+2. Fit all edge projection lines and corresponding connecting lines into continuous curves as path lines.

[0077] By acquiring the above-mentioned path lines, a driving road model can be provided for the vehicle, thereby providing an intuitive and clear data basis for subsequent lane line acquisition, thereby providing a guarantee for safe driving.

[0078] It is feasible that the forming of the road model by fitting the processed image corresponding to the front of the vehicle and the processed image corresponding to the rear of the vehicle based on the image processing module specifically includes:

[0079] The acquired path lines and lane lines are fused to generate a road model, and the lane lines are augmented in the road model.

[0080] Based on the above, this system obtains the road model of the vehicle's current situation by analyzing the road conditions in front of and behind the vehicle, and identifies the lane line interval in which the vehicle is located based on the image collected in front of the vehicle. The obtained results are highly reliable, and the calculation and image processing steps are simple, which reduces the operating pressure of the system / vehicle computer and provides a guarantee for users to quickly and accurately obtain the lane.

[0081] In a second aspect, this embodiment discloses Figure 2 A vehicle-mounted camera-based lane recognition method is shown, and this method is applicable to the above-mentioned vehicle-mounted camera-based lane recognition system. Specifically, the method includes the following steps:

[0082] S101-Configure the above-mentioned vehicle-mounted camera-based lane recognition system;

[0083] S102: collecting images of at least the road in front of the vehicle and the road behind the vehicle through the vehicle-mounted camera;

[0084] S103-The image processing module performs image processing on the collected road image to enhance the display effect of the lane lines in the image;

[0085] S104: the road condition fitting module forms a road model based on the image processed by the image processing module for the image in front of the vehicle and the image processed for the image behind the vehicle;

[0086] S105 - The lane recognition module identifies and displays the lane line section where the vehicle is located based on the image processed by the image processing module corresponding to the front of the vehicle and the road model.

[0087] During the execution of this method, the specific work content of each module can be referred to the relevant description in the previous text and will not be repeated here.

[0088] In a third aspect, this embodiment discloses a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, the above-mentioned vehicle-mounted camera-based lane recognition method is implemented.

[0089] It is understandable that the above system and method are based on hardware and software implementation. For hardware implementation, the processor can be implemented in one or more of the following units: application specific integrated circuit (ASIC), digital signal processor (DSP), digital signal processing device (DSPD), programmable logic device (PLD), field programmable gate array (FPGA), processor, controller, microcontroller, microprocessor, other electronic units designed to implement the functions described herein, or a combination thereof. For software implementation, part or all of the process of the embodiment can be completed by instructing the relevant hardware through a computer program. When implemented, the above program can be stored in a computer-readable medium or transmitted as one or more instructions or codes on a computer-readable medium. Computer-readable media include computer storage media and communication media, wherein the communication media include any medium that facilitates the transmission of computer programs from one place to another. The storage medium can be any available medium that can be accessed by a computer. Computer-readable media can include, but are not limited to, RAM, ROM, EEPROM, CD-ROM or other optical disk storage, disk storage media or other magnetic storage devices, or any other medium that can be used to carry or store the desired program code in the form of instructions or data structures and can be accessed by a computer.

[0090] Finally, it should be noted that the above is only a preferred embodiment of the present application and is not intended to limit the present application. Although the present application has been described in detail with reference to the aforementioned embodiments, it is still possible for those skilled in the art to modify the technical solutions described in the aforementioned embodiments or to make equivalent substitutions for some of the technical features therein. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present application shall be included in the protection scope of the present application.

Claims

1. A lane recognition system based on a vehicle-mounted camera, characterized in that: It includes an image acquisition module, an image processing module, a road condition fitting module and a lane recognition module; The image acquisition module is configured to: acquire images of at least the road in front of the vehicle and the road behind the vehicle through the vehicle-mounted camera; The image processing module is configured to: perform image processing on the collected road image to enhance the display effect of the lane lines in the image; The road condition fitting module is configured to: form a road model based on the image processed by the image processing module for the image in front of the vehicle and the image processed for the image behind the vehicle; The lane recognition module is configured to: mark and display the lane line section where the vehicle is located based on the image processed by the image processing module corresponding to the front of the vehicle and the road model; The step of at least collecting images of the road in front of the vehicle and the road behind the vehicle specifically includes: The road in front of the vehicle is captured by a vehicle-mounted camera 1 located in front of the vehicle, and the captured image 1 includes the distribution of lane lines; The road behind the vehicle is imaged by a second on-vehicle camera located at the rear of the vehicle, and the image collected includes the distribution of lane lines; The collected image 1 also includes the road extension layout, and the image processing of the collected road image also includes: Performing image segmentation on the elements in the image 1 by image normalization processing to obtain the segmented elements; Performing edge extraction on the segmented elements to obtain element edges; Comparing the edge of the element with a preset road feature to identify a road extension element in the image, wherein the road extension element includes at least a road fence and a curb; Deepening the identified road extension elements in the first image to obtain a second auxiliary image with a highlighted path, and connecting the same road extension elements as a road layout path; Connect the same road extension elements along the road extension direction as the road layout path, including: Extracting edges of all the road extension elements along the vehicle travel direction; Project all extracted edges vertically onto the same horizontal plane; A continuous curve is generated on the horizontal plane as a path line; The step of generating a continuous curve on the horizontal plane as a path line specifically includes: defining a plurality of edge vertical projections on the horizontal plane as edge projection line 1, edge projection line 2, ..., edge projection line N in sequence; When all edge projection lines do not overlap, all edge projection lines are fitted into a continuous curve as a path line; When the edge projection line i and the edge projection line i+1 have an overlapping area, calculate the continuity ρ1 of the edge projection line i and the edge projection line i-1, calculate the continuity ρ2 of the edge projection line i and the edge projection line i+2, calculate the continuity ρ3 of the edge projection line i+1 and the edge projection line i-1, calculate the continuity ρ4 of the edge projection line i+1 and the edge projection line i+2, and obtain the fitness W of the insulation projection line i and the fitness M of the edge projection line i+1, w=ρ1+ρ2+K1, M=ρ3+ρ4+K2, where K1 and K2 are both curvature compensation values. Compare the sizes of W and M. When W>M, select edge projection line i as the connecting line between edge projection line i-1 and edge projection line i+2. When W=M, select edge projection line i or edge projection line i+1 as the connecting line between edge projection line i-1 and edge projection line i+2. When W<M, select edge projection line i+1 as the connecting line between edge projection line i-1 and edge projection line i+2. Fit all edge projection lines and corresponding connecting lines into continuous curves as path lines.

2. The vehicle lane recognition system based on a vehicle-mounted camera according to claim 1, characterized in that: The image processing of the collected road image includes: Performing image segmentation on the elements in the first image and the second image by image normalization processing to obtain segmented elements; Performing edge extraction on the segmented elements to obtain element edges; Comparing the element edge with a preset lane line edge feature to identify the lane line in the image, wherein the lane line edge feature includes at least a straight lane line edge feature and a curved lane line edge feature; The identified lane lines are deepened in the image 1 and the image 2 to obtain the auxiliary image 1 and the auxiliary image 2 in which the lane lines are highlighted.

3. The vehicle lane recognition system based on a vehicle-mounted camera according to claim 2, characterized in that: The in-depth processing specifically includes: Identify the main color of the identified lane line in the corresponding image, where the main color is the color of the lane line with the largest proportion in the image; The outline of the identified lane line in the corresponding image is processed with the main color as a pure color; the content other than the identified lane line outline in the corresponding image is defined as a background element, and the background element is blurred.

4. The vehicle lane recognition system based on a vehicle-mounted camera according to claim 1, characterized in that: The forming of the road model by fitting the processed image corresponding to the front of the vehicle and the processed image corresponding to the rear of the vehicle based on the image processing module specifically includes: The acquired path lines and lane lines are fused to generate a road model, and the lane lines are augmented in the road model.

5. A method for lane recognition based on a vehicle-mounted camera, characterized in that: The method comprises the following steps: A vehicle lane recognition system based on a vehicle-mounted camera as described in any one of claims 1 to 4; At least collecting images of the road in front of the vehicle and the road behind the vehicle through the vehicle-mounted camera; The image processing module performs image processing on the collected road images to enhance the display effect of the lane lines in the images; The road condition fitting module forms a road model based on the image processed by the image processing module for the image in front of the vehicle and the image processed for the image behind the vehicle; The lane recognition module identifies and displays the lane line section where the vehicle is located based on the image processed by the image processing module corresponding to the front of the vehicle and the road model.

6. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method for lane recognition based on a vehicle-mounted camera as claimed in claim 5 is implemented.

Citation Information

Patent Citations

  • Apparatus and method for producing assistant information of driving vehicle for driver

    CN101101333A

  • Lane line recognition method, device and vehicle

    CN109325389A

  • Training method of dotted line lane line endpoint detection model

    CN111259704A

  • Lane line automatic identification method and device and readable storage medium

    CN112329694A

  • Method and device for generating rear road model

    CN114523978A

Cited By

  • Vehicle-mounted camera image processing method and device

    CN118351684A