Lane line recognition method, autonomous vehicle, and computer storage medium

Abstract

The application provides a lane line recognition method based on automatic driving, an automatic driving vehicle and a computer storage medium. The lane line recognition method comprises the following steps: starting a camera of the automatic driving vehicle; reading a first image pre-stored in the automatic driving vehicle, and collecting a real-time image of a current environment of the automatic driving vehicle by using the camera; comparing the real-time image with the first image; and when a difference value between the real-time image and the first image is less than a preset threshold, performing lane line recognition on the real-time image by using a lane line recognition result of the first image. By using the lane line recognition method, the lane line recognition efficiency of subsequent images can be improved by using the lane line recognition result of the first image, and the control speed of automatic driving is improved.

Description

Technical Field

[0001] This application relates to the field of autonomous driving technology, and in particular to a lane line recognition method based on autonomous driving, an autonomous vehicle, and a computer storage medium. Background Technology

[0002] Currently, lane detection is a fundamental function in autonomous driving. During autonomous driving, vehicles need to detect lane lines on the road.

[0003] However, for lane line recognition, traditional detection methods typically use edge detection combined with Hough transform for global recognition of each image frame. But in real-world applications, autonomous vehicles need to continuously acquire real-time images and perform lane line recognition on these images to ensure the accuracy and safety of autonomous driving. Traditional detection methods require global recognition of every image frame, placing extremely high demands on the vehicle's processing resources, thus limiting the potential for further improvements in lane line recognition efficiency. Summary of the Invention

[0004] To address the aforementioned technical problems, this application proposes a lane line recognition method based on autonomous driving, an autonomous vehicle, and a computer storage medium.

[0005] To address the aforementioned technical problems, this application proposes a lane line recognition method based on autonomous driving, the lane line recognition method comprising:

[0006] Activate the cameras of the autonomous vehicle;

[0007] Read the first image pre-stored in the autonomous vehicle and use the camera to capture real-time images of the current environment of the autonomous vehicle;

[0008] Compare the real-time image with the first image;

[0009] When the difference between the real-time image and the first image is less than a preset threshold, lane line recognition is performed on the real-time image using the lane line recognition result of the first image.

[0010] The first image is an environmental image taken by the autonomous vehicle when it is stationary.

[0011] Before reading the first image pre-stored by the autonomous vehicle, the lane line recognition method further includes:

[0012] The camera is used to capture still images of the autonomous vehicle in a stationary state, which are then pre-stored as the first image, and the vehicle pose of the first image is also stored.

[0013] The first image is subjected to binary thresholding and viewpoint transformation to obtain a standard image for lane line recognition;

[0014] Within the standard image for lane line recognition, lane line pixels are fitted to pixels with a pixel value of 1 to obtain a pre-stored standard lane line.

[0015] The step of performing lane line pixel fitting on pixels with a pixel value of 1 within the standard lane line recognition image to obtain a pre-stored standard lane line includes:

[0016] Obtain the standard histogram of the lane line recognition standard image;

[0017] The lane line recognition standard image is iteratively scanned and lane line pixels are detected at each peak position in the standard histogram;

[0018] Polynomial fitting is performed on the lane line pixels to obtain the standard lane line pre-stored.

[0019] The comparison of the real-time image with the first image includes:

[0020] The first lane line bounding box of the real-time image and the second lane line bounding box of the first image are detected using an object detection network;

[0021] Obtain the intersection-union ratio of the first lane line annotation box and the second lane line annotation box;

[0022] The difference between the real-time image and the first image is obtained using the intersection-union ratio.

[0023] The step of using the lane line recognition result of the first image to perform lane line recognition on the real-time image includes:

[0024] Obtain standard lane lines from the lane line recognition results of the first image;

[0025] Map the standard lane lines onto the real-time image;

[0026] Lane line recognition is performed on the area of ​​the real-time image other than the standard lane lines to obtain the real-time lane lines;

[0027] The standard lane lines and the real-time lane lines are fused together, and the lane line recognition result of the final real-time image is obtained by using straight line fitting.

[0028] The step of mapping the standard lane lines onto the real-time image includes:

[0029] The standard pose of the autonomous vehicle is obtained from the lane line recognition results of the first image;

[0030] The real-time pose of the autonomous vehicle is acquired when the real-time image is captured;

[0031] A mapping matrix is ​​generated using the standard pose and the real-time pose;

[0032] The standard lane lines are mapped onto the real-time image according to the mapping matrix.

[0033] The step of reading the first image pre-stored in the autonomous vehicle and using the camera to capture real-time images of the current environment of the autonomous vehicle includes:

[0034] Read the first image, and the first environmental information when the first image was acquired;

[0035] Obtain secondary environmental information of the current environment;

[0036] The difference between the first environmental information and the second environmental information is used to obtain the acquisition parameters;

[0037] Adjust the camera parameters according to the acquisition parameters described;

[0038] The real-time images are acquired according to the adjusted camera parameters;

[0039] The acquisition parameters include at least one of the following: whether the flash is turned on, light intensity, and exposure time.

[0040] To address the aforementioned technical problems, this application also proposes an autonomous driving vehicle, which includes a memory and a processor coupled to the memory; wherein the memory is used to store program data, and the processor is used to execute the program data to implement the lane line recognition method described above.

[0041] To address the aforementioned technical problems, this application also proposes a computer storage medium for storing program data, which, when executed by a computer, is used to implement the lane line recognition method described above.

[0042] Compared with existing technologies, the beneficial effects of this application are as follows: the autonomous vehicle activates its camera; it reads the pre-stored first image and uses the camera to capture a real-time image of the current environment of the autonomous vehicle; it compares the real-time image with the first image; when the difference between the real-time image and the first image is less than a preset threshold, it uses the lane line recognition result of the first image to perform lane line recognition on the real-time image. Through the above lane line recognition method, the lane line recognition efficiency of subsequent images can be improved by utilizing the lane line recognition result of the first image, thereby increasing the control speed of autonomous driving. Attached Figure Description

[0043] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0044] in:

[0045] Figure 1 This is a flowchart illustrating an embodiment of the lane line recognition method provided in this application;

[0046] Figure 2 This is a flowchart illustrating another embodiment of the lane line recognition method provided in this application;

[0047] Figure 3 yes Figure 1 The diagram shows the specific process flow of step S14 in the lane line recognition method.

[0048] Figure 4 This is a schematic diagram of the structure of an embodiment of the autonomous vehicle provided in this application;

[0049] Figure 5 This is a schematic diagram of another embodiment of the autonomous vehicle provided in this application;

[0050] Figure 6 This is a schematic diagram of the structure of an embodiment of the computer storage medium provided in this application. Detailed Implementation

[0051] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of the embodiments. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of this application.

[0052] The terms “first,” “second,” “third,” “fourth,” etc. (if present) in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a particular order or sequence. It should be understood that such data can be interchanged where appropriate so that embodiments of the application described herein can be implemented, for example, in orders other than those illustrated or described herein. Furthermore, the terms “comprising” and “having,” and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0053] Please refer to details. Figure 1 , Figure 1 This is a flowchart illustrating an embodiment of the lane line recognition method provided in this application.

[0054] The lane recognition method of this application is applied to autonomous vehicles. The autonomous vehicle can be a server, a terminal device, or a system in which the server and terminal device cooperate. Accordingly, all components of the autonomous vehicle, such as units, subunits, modules, and submodules, can be entirely located in the server, entirely located in the terminal device, or separately located in both the server and the terminal device.

[0055] Furthermore, the aforementioned server can be either hardware or software. When the server is hardware, it can be implemented as a distributed server cluster consisting of multiple servers, or as a single server. When the server is software, it can be implemented as multiple software programs or software modules, such as software or software modules used to provide distributed server functionality, or as a single software program or software module; no specific limitations are made here.

[0056] It should be noted that the lane line recognition method of this application can also be applied to autonomous driving platforms that remotely control autonomous vehicles, and no specific limitations are imposed here.

[0057] like Figure 1 As shown, the specific steps are as follows:

[0058] Step S11: Activate the camera of the autonomous vehicle.

[0059] In this embodiment of the application, when the autonomous vehicle starts, the cameras and lidar on the vehicle are activated simultaneously.

[0060] Furthermore, after the camera is activated, the autonomous vehicle can also calibrate the camera, including but not limited to: removing lens distortion effects and changing the radial deviation of the chessboard model.

[0061] Specifically, autonomous vehicles can convert images into a chessboard model, calibrating cameras to ensure that white and black squares in the image are of the same size, thereby correcting radial deviations. Autonomous vehicles can also identify the center of the chessboard and use the desired chessboard size to calculate distortion coefficients, thus correcting distortion.

[0062] Step S12: Read the first image pre-stored by the autonomous vehicle and use the camera to capture real-time images of the current environment of the autonomous vehicle.

[0063] In this embodiment of the application, the autonomous vehicle reads the pre-stored first image and uses a camera to capture real-time images of the current environment.

[0064] First, the autonomous vehicle needs to determine whether the real-time image of the current environment is the first image captured during this driving session. The first image is the environmental image taken by the autonomous vehicle while stationary. If the real-time image is the first image captured during this driving session, then global lane line recognition is performed on the real-time image according to the normal lane line recognition process. If the real-time image is not the first image captured during this driving session, then the lane line recognition result of the first image is used as a reference to reduce the lane line recognition area and workload of the real-time image, thereby speeding up the lane line recognition process.

[0065] Furthermore, autonomous vehicles can adjust camera parameters based on the difference between environmental information when acquiring the first image and environmental information when acquiring real-time images, thereby making the real-time images closer to the first image, which makes it easier to use the first image as a reference in the lane line recognition process of real-time images.

[0066] For example, if the first image is captured during the daytime and the real-time image is captured during the nighttime, the autonomous vehicle can improve the overall brightness of the real-time image by turning on the flash, increasing the brightness of the lights, or extending the exposure time, so as to approximate the overall brightness of the first image.

[0067] For details on the process of autonomous vehicles recognizing and storing the first image, please refer to [link / reference needed]. Figure 2 , Figure 2 This is a flowchart illustrating another embodiment of the lane line recognition method provided in this application.

[0068] like Figure 2 As shown, the specific steps are as follows:

[0069] Step S21: Use a camera to capture a still image of the autonomous vehicle in a stationary state, pre-store it as the first image, and store the vehicle pose of the first image.

[0070] Step S22: Perform binary thresholding and viewpoint transformation on the first image to obtain a standard image for lane line recognition.

[0071] Step S23: Fit lane line pixels to pixels with a pixel value of 1 in the standard lane line recognition image to obtain the standard lane line pre-stored.

[0072] In this embodiment, the autonomous vehicle binarizes the first image to obtain a black and white image, and then analyzes the pixel value distribution of the black and white image to obtain a standard histogram. Each peak in the standard histogram represents a lane line. Therefore, the autonomous vehicle can use a sliding window technique to iteratively scan the image at the positions corresponding to the peaks to detect the pixels of the lane lines, and perform polynomial fitting to obtain all the pixels of each lane line.

[0073] Autonomous vehicles can obtain standard lane lines by remapping all the pixels of each lane line onto the first image.

[0074] Step S13: Compare the real-time image with the first image.

[0075] In this embodiment, the autonomous vehicle inputs the real-time image and the first image into a pre-trained target detection network to obtain the first lane line bounding box of the real-time image and the second lane line bounding box of the first image.

[0076] Then, the autonomous vehicle uses the degree of overlap between the first lane line bounding box and the second lane line bounding box to evaluate the difference between the real-time image and the first image. Specifically, the autonomous vehicle calculates the intersection-union ratio (IUR) of the first and second lane line bounding boxes; the higher the IUR, the smaller the difference between the real-time image and the first image.

[0077] Step S14: When the difference between the real-time image and the first image is less than a preset threshold, lane line recognition is performed on the real-time image using the lane line recognition result of the first image.

[0078] In this embodiment, when the difference between the real-time image and the first image is higher than a preset threshold, the autonomous vehicle can consider that the real-time image and the first image are significantly different. If the lane line recognition result of the first image is still referenced, the detection accuracy of the real-time image may be reduced. Therefore, the autonomous vehicle needs to perform global recognition on the real-time image at this time.

[0079] Specifically, in addition to using sliding window search and iterative scanning techniques to identify lane lines in real-time images, autonomous vehicles can also use dual filters to smooth and preserve edges, and introduce an optimized intensity threshold range (OITR) to improve the performance of the canny operator, which detects the edges of low-intensity (colored, eroded, or blurred) lane markings.

[0080] When the difference between the real-time image and the first image exceeds a preset threshold, the autonomous vehicle can consider the difference between the real-time image and the first image to be very small. It can then refer to the lane line recognition results of the first image to accelerate the lane line recognition efficiency of the real-time image. The specific process is as follows:

[0081] Please continue reading. Figure 3 , Figure 3 yes Figure 1 The diagram shows the specific process flow of step S14 in the lane line recognition method.

[0082] like Figure 3 As shown, the specific steps are as follows:

[0083] Step S141: Obtain standard lane lines from the lane line recognition results of the first image.

[0084] Step S142: Map the standard lane lines onto the real-time image.

[0085] In this embodiment, the autonomous vehicle obtains its standard pose from the lane line recognition result of the first image; obtains the real-time pose of the autonomous vehicle when the real-time image is acquired; generates a mapping matrix using the standard pose and the real-time pose; and maps the standard lane line onto the real-time image according to the mapping matrix.

[0086] Step S143: Perform lane line recognition on the area other than the standard lane lines in the real-time image to obtain the real-time lane lines.

[0087] In this embodiment, the autonomous vehicle directly maps the standard lane lines onto the area of ​​the real-time image as the lane lines, while the area outside the mapped area continues to operate according to the conventional lane line recognition process.

[0088] Step S144: Fuse the standard lane lines and the real-time lane lines, and use straight line fitting to obtain the final lane line recognition result of the real-time image.

[0089] In this embodiment of the application, the autonomous vehicle fuses the mapped lane lines and the real-time lane lines identified in step S143 to obtain the final lane line recognition result of the real-time image, and performs autonomous driving according to the real-time lane line recognition result.

[0090] Furthermore, autonomous vehicles can also correct the lane lines obtained after fusion, for example, by interpolating and supplementing broken lane lines, or by trimming excessively long lane lines.

[0091] In this embodiment, the camera of the autonomous vehicle is activated; the pre-stored first image of the autonomous vehicle is read, and the camera is used to acquire a real-time image of the current environment of the autonomous vehicle; the real-time image is compared with the first image; when the difference between the real-time image and the first image is less than a preset threshold, lane line recognition is performed on the real-time image using the lane line recognition result of the first image. Through the above lane line recognition method, the lane line recognition efficiency of subsequent images can be improved by utilizing the lane line recognition result of the first image, thereby increasing the control speed of autonomous driving.

[0092] Those skilled in the art will understand that, in the above-described method of the specific implementation, the order in which each step is written does not imply a strict execution order and does not constitute any limitation on the implementation process. The specific execution order of each step should be determined by its function and possible internal logic.

[0093] To implement the lane line recognition method described above, this application also proposes an autonomous driving vehicle, for details please refer to [link / reference needed]. Figure 4 , Figure 4 This is a structural schematic diagram of an embodiment of the autonomous vehicle provided in this application.

[0094] The autonomous vehicle 300 in this embodiment includes a startup module 31, a data acquisition module 32, a comparison module 33, and an identification module 34.

[0095] The startup module 31 is used to start the camera of the autonomous vehicle.

[0096] The acquisition module 32 is used to read the first image pre-stored by the autonomous vehicle and to acquire real-time images of the current environment of the autonomous vehicle using the camera.

[0097] The comparison module 33 is used to compare the real-time image with the first image.

[0098] The recognition module 34 is used to recognize lane lines in the real-time image using the lane line recognition result of the first image when the difference between the real-time image and the first image is less than a preset threshold.

[0099] To implement the lane line recognition method described above, this application also proposes another autonomous driving vehicle, for details please refer to [link / reference needed]. Figure 5 , Figure 5 This is a structural schematic diagram of another embodiment of the autonomous vehicle provided in this application.

[0100] The autonomous vehicle 400 in this embodiment includes a processor 41, a memory 42, an input / output device 43, and a bus 44.

[0101] The processor 41, memory 42, and input / output device 43 are respectively connected to the bus 44. The memory 42 stores program data, and the processor 41 is used to execute the program data to implement the lane line recognition method described in the above embodiment.

[0102] In this embodiment, processor 41 can also be referred to as a CPU (Central Processing Unit). Processor 41 may be an integrated circuit chip with signal processing capabilities. Processor 41 can also be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. The general-purpose processor can be a microprocessor, or processor 41 can be any conventional processor.

[0103] This application also provides a computer storage medium; please refer to the following: Figure 6 , Figure 6 This is a schematic diagram of a computer storage medium according to an embodiment of the present application. The computer storage medium 600 stores a computer program 61, which, when executed by a processor, is used to implement the lane line recognition method of the above embodiment.

[0104] When the embodiments of this application are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) or processor to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0105] The above description is merely an embodiment of this application and does not limit the patent scope of this application. Any equivalent structural or procedural transformations made using the content of this application's specification and drawings, or direct or indirect applications in other related technical fields, are similarly included within the patent protection scope of this application.

Claims

1. A lane line recognition method based on autonomous driving, characterized in that, The lane line recognition method includes: Activate the cameras of the autonomous vehicle; Read the first image pre-stored in the autonomous vehicle and use the camera to capture real-time images of the current environment of the autonomous vehicle; Compare the real-time image with the first image; When the difference between the real-time image and the first image is less than a preset threshold, lane line recognition is performed on the real-time image using the lane line recognition result of the first image; The step of using the lane line recognition result of the first image to perform lane line recognition on the real-time image includes: Obtain standard lane lines from the lane line recognition results of the first image; Map the standard lane lines onto the real-time image; Lane line recognition is performed on the area of ​​the real-time image other than the standard lane lines to obtain the real-time lane lines; The standard lane lines and the real-time lane lines are fused together, and the lane line recognition result of the final real-time image is obtained by using straight line fitting. The step of mapping the standard lane lines onto the real-time image includes: The standard pose of the autonomous vehicle is obtained from the lane line recognition results of the first image; The real-time pose of the autonomous vehicle is acquired when the real-time image is captured; A mapping matrix is ​​generated using the standard pose and the real-time pose; The standard lane lines are mapped onto the real-time image according to the mapping matrix.

2. The lane line recognition method according to claim 1, characterized in that, The first image is an environmental image taken by the autonomous vehicle while it is stationary.

3. The lane line recognition method according to claim 2, characterized in that, Before reading the first image pre-stored by the autonomous vehicle, the lane line recognition method further includes: The camera is used to capture still images of the autonomous vehicle in a stationary state, which are then pre-stored as the first image, and the vehicle pose of the first image is also stored. The first image is subjected to binary thresholding and viewpoint transformation to obtain a standard image for lane line recognition; Within the standard image for lane line recognition, lane line pixels are fitted to pixels with a pixel value of 1 to obtain a pre-stored standard lane line.

4. The lane line recognition method according to claim 3, characterized in that, The step of fitting lane line pixels to pixels with a pixel value of 1 within the standard lane line recognition image to obtain a pre-stored standard lane line includes: Obtain the standard histogram of the lane line recognition standard image; The lane line recognition standard image is iteratively scanned and lane line pixels are detected at each peak position in the standard histogram; Polynomial fitting is performed on the lane line pixels to obtain the standard lane line pre-stored.

5. The lane line recognition method according to claim 1, characterized in that, The comparison of the real-time image with the first image includes: The first lane line bounding box of the real-time image and the second lane line bounding box of the first image are detected using an object detection network; Obtain the intersection-union ratio of the first lane line annotation box and the second lane line annotation box; The difference between the real-time image and the first image is obtained using the intersection-union ratio.

6. The lane line recognition method according to claim 1, characterized in that, The step of reading the first image pre-stored in the autonomous vehicle and using the camera to capture real-time images of the current environment of the autonomous vehicle includes: Read the first image, and the first environmental information when the first image was acquired; Obtain secondary environmental information of the current environment; The difference between the first environmental information and the second environmental information is used to obtain the acquisition parameters; Adjust the camera parameters according to the acquisition parameters described; The real-time images are acquired according to the adjusted camera parameters; The acquisition parameters include at least one of the following: whether the flash is turned on, light intensity, and exposure time.

7. An autonomous vehicle, characterized in that, The autonomous vehicle includes a memory and a processor coupled to the memory; The memory is used to store program data, and the processor is used to execute the program data to implement the lane line recognition method as described in any one of claims 1-6.

8. A computer storage medium, characterized in that, The computer storage medium is used to store program data, which, when executed by the computer, is used to implement the lane line recognition method as described in any one of claims 1-6.

Images