A lane line recognition method and system based on automatic driving

Abstract

The embodiment of the present invention discloses a lane line recognition method and system based on automatic driving, the method comprising: acquiring image information; acquiring real-time positioning information and driving parameters of a vehicle, and associating the positioning information with the acquired image information; sending the associated image to a pre-connected map library for calculation based on the average grayscale value of the image to obtain a clear image to be recognized at the positioning point; sending the image to be recognized to a pre-constructed lane line recognition model for recognition to obtain a lane line recognition result; the lane line recognition model is obtained by training based on lane and road marking benchmarks, the benchmarks including multiple lanes, road marking categories and a variety of different harsh environment scenarios; the beneficial effect is: it has good robustness in actual application scenarios, thereby improving the accuracy of lane line detection.

Description

Technical Field

[0001] The present invention relates to the field of autonomous driving technology, and in particular to a lane line recognition method and system based on autonomous driving. Background Art

[0002] At present, lane detection is a basic function in autonomous driving. During the autonomous driving process, the vehicle needs to detect the lane lines on the road. However, for lane recognition, the traditional detection method generally uses edge detection + Hough transform, which is applicable when the image is clear and the lane lines are not blocked; but in actual application scenarios, the scenes faced by lane detection have diverse characteristics. Under the influence of harsh environmental interference and surrounding obstacles, the recognition reliability will be reduced, resulting in the defect of low detection accuracy. Summary of the invention

[0003] In view of the technical defects in the prior art, the purpose of the embodiments of the present invention is to provide a lane line recognition method and system based on automatic driving to improve the accuracy of lane line detection.

[0004] To achieve the above objectives, in a first aspect, an embodiment of the present invention provides a lane line recognition method based on automatic driving, the method comprising:

[0005] Acquire image information; wherein the image information includes information detected by a perception layer deployed on the vehicle;

[0006] Acquire real-time positioning information and driving parameters of the vehicle, and associate the positioning information with the acquired image information;

[0007] The associated image is sent to the pre-connected map library for calculation based on the average gray value of the image to obtain a clear image to be identified at the positioning point;

[0008] The image to be identified is sent to a pre-constructed lane line recognition model for identification to obtain a lane line recognition result; wherein the lane line recognition model is trained based on lane and road marking benchmarks, and the benchmarks include multiple lanes, road marking categories and a variety of different harsh environment scenarios.

[0009] Preferably, the method further comprises:

[0010] Acquire the vehicle's operating signal, and construct the vehicle's actual driving trajectory in combination with the acquired driving parameters;

[0011] Analyzing a predicted driving trajectory of the vehicle based on the driving parameters and the lane line recognition result;

[0012] Then the predicted driving trajectory is matched with the actual driving trajectory to determine whether the trajectories of the two are within a preset deviation range;

[0013] If the deviation range is exceeded, the corresponding lane line recognition result is used as training data to optimize the lane line recognition model.

[0014] Preferably, when training the lane recognition model, each lane is trained as its own instance, and image-based learning perspective transformation is performed to achieve recognition of lane changes.

[0015] Preferably, the method further comprises:

[0016] Build a lane shape model based on the road structure and the angle from which the image information is acquired;

[0017] During recognition, the lane parameters output by the lane shape model are then passed into the lane line recognition model to achieve accurate positioning of the lane line.

[0018] In a second aspect, an embodiment of the present invention further provides a lane line recognition system based on automatic driving, comprising:

[0019] An image acquisition module, used to acquire image information; wherein the image information includes information detected by a perception layer deployed on the vehicle;

[0020] An information association module, used to obtain real-time positioning information and driving parameters of the vehicle, and associate the positioning information with the acquired image information;

[0021] The preprocessing module is used to send the associated image to the pre-connected map library for calculation based on the average gray value of the image, so as to obtain a clear image to be identified at the positioning point;

[0022] A recognition module is used to send the image to be recognized into a pre-constructed lane line recognition model for recognition to obtain a lane line recognition result; wherein the lane line recognition model is trained based on lane and road marking benchmarks, and the benchmarks include multiple lanes, road marking categories and a variety of different harsh environment scenarios.

[0023] Preferably, the lane line recognition system based on autonomous driving further includes an optimization module, wherein the optimization module is used to:

[0024] Acquire the vehicle's operating signal and construct the vehicle's actual driving trajectory in combination with the acquired driving parameters;

[0025] Analyzing a predicted driving trajectory of the vehicle based on the driving parameters and the lane line recognition result;

[0026] Then the predicted driving trajectory is matched with the actual driving trajectory to determine whether the trajectories of the two are within a preset deviation range;

[0027] If the deviation range is exceeded, the corresponding lane line recognition result is used as training data to optimize the lane line recognition model.

[0028] Preferably, when training the lane recognition model, each lane is trained as its own instance, and image-based learning perspective transformation is performed to achieve recognition of lane changes.

[0029] Preferably, the identification module is also used for:

[0030] Build a lane shape model based on the road structure and the angle from which the image information is acquired;

[0031] During recognition, the lane parameters output by the lane shape model are then passed into the lane line recognition model to achieve accurate positioning of the lane line.

[0032] An embodiment of the present invention also provides a lane line recognition system based on autonomous driving, including a perception layer and electronic devices deployed on a vehicle, wherein the electronic devices include one or more processors, one or more input devices, one or more output devices and a memory, wherein the processor, input device, output device and memory are interconnected via a bus, wherein the memory is used to store a computer program, wherein the computer program includes program instructions, and wherein the processor is configured to call the program instructions to execute the method steps described in the first aspect.

[0033] In implementing the embodiment of the present invention, by associating image information with the positioning information of the vehicle, a clear image of the location is found in the gallery, and then sent to the constructed lane line recognition model for recognition. The model is based on lane and road marking benchmarks, and the benchmarks include multiple lanes, road marking categories and a variety of different harsh environment scenarios. This makes the model have good robustness in actual application scenarios, thereby improving the accuracy of lane line detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] In order to more clearly illustrate the specific implementation of the present invention or the technical solution in the prior art, the drawings required for use in the specific implementation or the description of the prior art are briefly introduced below.

[0035] Figure 1 is a flow chart of a lane line recognition method based on automatic driving provided by an embodiment of the present invention;

[0036] Figure 2 is a structural block diagram of a lane line recognition system based on automatic driving provided by an embodiment of the present invention;

[0037] Figure 3 It is a schematic diagram of the structure of another electronic device in a lane line recognition system based on autonomous driving provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0038] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0039] Please refer to Figure 1 , an embodiment of the present invention provides a lane line recognition method based on automatic driving, the method comprising:

[0040] S101, obtaining image information; wherein, the image information includes information detected by a perception layer deployed on the vehicle.

[0041] Specifically, the perception layer includes a variety of sensors, including lidar, ultrasound, millimeter wave, vehicle-mounted camera, GPS, IMU, accelerometer and gyroscope, etc.

[0042] S102, obtaining real-time positioning information and driving parameters of the vehicle, and associating the positioning information with the obtained image information.

[0043] Specifically, the positioning information may include longitude and latitude information; the driving parameters include ECU information of the vehicle, such as current angle, linear speed, direction of movement, gear shifting, clutch, braking, and GPS, IMU, accelerometer and gyroscope data.

[0044] By associating the collected image with the current position, the location of the collected image can be more accurate, thereby avoiding interference from related similar images.

[0045] S103, sending the associated image to a pre-connected map library to perform calculation based on the average gray value of the image, so as to obtain a clear image to be identified at the positioning point.

[0046] Specifically, in this embodiment, the map library includes high-precision maps (for example, NavNet) and historical image information acquired in the past, and a clear image corresponding to the current positioning point is matched by calculating the average grayscale value of the image; since the positioning information and historical images are associated, even if the road changes, the high-precision map is not updated in time, or the current picture clarity is insufficient due to environmental influences, a clear image of the positioning point can be found from the historical image as a recognition reference, thereby improving the accuracy of subsequent recognition.

[0047] S104, sending the image to be recognized into a pre-constructed lane line recognition model for recognition to obtain a lane line recognition result; wherein the lane line recognition model is trained based on lane and road marking benchmarks, and the benchmarks include multiple lanes, road marking categories and a variety of different harsh environment scenarios.

[0048] Specifically, the image to be identified is also pre-processed by noise reduction and other methods; the lane line recognition model uses a deep learning algorithm, takes the lane detection problem as a segmentation task, and performs pixel segmentation training; wherein, given an input image, the output is a segmentation map with predictions for each pixel. During training, the training data set includes image samples under various severe weather conditions, multiple lanes and road marking categories, and multiple task networks are obtained through training to adapt to the existing scenes. Road markings include various markings and arrows on the road; severe weather includes common heavy rain, night and fog, etc.

[0049] Furthermore, in order to further cope with adverse weather conditions, for example, when heavy fog occurs, the image to be identified is also denoised; for example, a defogging algorithm based on a dark channel is combined with an atmospheric scattering model to obtain atmospheric light and transmittance, thereby obtaining the defogged image to be identified.

[0050] In another embodiment, to further improve the recognition effect, the method further includes:

[0051] Acquire the vehicle's operating signal, and construct the vehicle's actual driving trajectory in combination with the acquired driving parameters;

[0052] Analyzing a predicted driving trajectory of the vehicle based on the driving parameters and the lane line recognition result;

[0053] Then the predicted driving trajectory is matched with the actual driving trajectory to determine whether the trajectories of the two are within a preset deviation range;

[0054] If the deviation range is exceeded, the corresponding lane line recognition result is used as training data to optimize the lane line recognition model.

[0055] By comparing the two driving trajectories, the accuracy of lane line recognition can be verified; for example, if the recognized lane line is a straight line, the predicted trajectory should be a straight line, and the speed should be maintained or increased; but if the vehicle's operating signal shows deceleration or steering, the vehicle's gyroscope senses the angle change, and the trajectories formed before and after exceed the deviation, it is considered that the current result is an abnormal processing result, and then the abnormal result is re-trained to optimize the model to improve the recognition accuracy.

[0056] The above scheme associates image information with the vehicle's positioning information to find a clear image of the location in the gallery, and then sends it to the constructed lane line recognition model for recognition. The model is based on lane and road marking benchmarks, and the benchmarks include multiple lanes, road marking categories, and a variety of different harsh environment scenarios. This makes it have good robustness in actual application scenarios, thereby improving the accuracy of lane line detection.

[0057] Furthermore, considering the lane changes of the car during movement, the lane line recognition model also trains each lane as its own instance during training, and performs image-based learning perspective transformation to achieve lane change recognition.

[0058] Specifically, by performing instance training, the lane detection problem is converted into an instance segmentation problem. Each lane forms its own instance, achieving end-to-end training and parameterizing the segmented lane instances before fitting the lanes.

[0059] The learning perspective transformation of the image can adopt affine transformation, which refers to the process in geometry where a vector space is transformed into another vector space by a linear transformation followed by a translation. It maintains the "straightness" (a straight line remains a straight line after transformation) and "parallelism" (the relative position relationship between two-dimensional graphics remains unchanged, parallel lines remain parallel lines, and the position order of points on the straight line remains unchanged) of the two-dimensional graphics, so that lane recognition is not limited to detecting a fixed number of lanes, and can handle the situation of variable number of lanes and lane changes.

[0060] Furthermore, in order to improve the efficiency of processing lane parameters, in another embodiment, the method further includes:

[0061] Build a lane shape model based on the road structure and the angle from which the image information is acquired;

[0062] During recognition, the lane parameters output by the lane shape model are then passed into the lane line recognition model to achieve accurate positioning of the lane line.

[0063] Specifically, the angle for acquiring image information comes from the angle of the acquisition device; the lane shape model adopts a multi-attention model to learn richer structures and contexts through its own established network, thereby improving processing efficiency; at the same time, the output physical parameters are used to achieve accurate positioning of the lane line; it should be noted that the application field of the above technical solution is not limited to autonomous driving, but is also used in scenarios where lane line recognition is required. As a lane line recognition method, its solution is the same as the above and will not be repeated here.

[0064] Based on the same inventive concept, the embodiment of the present invention also provides a lane line recognition system based on automatic driving, such as Figure 2 As shown, including:

[0065] An image acquisition module, used to acquire image information; wherein the image information includes information detected by a perception layer deployed on the vehicle;

[0066] An information association module, used to obtain real-time positioning information and driving parameters of the vehicle, and associate the positioning information with the acquired image information;

[0067] The preprocessing module is used to send the associated image to the pre-connected map library for calculation based on the average gray value of the image, so as to obtain a clear image to be identified at the positioning point;

[0068] A recognition module is used to send the image to be recognized into a pre-constructed lane line recognition model for recognition to obtain a lane line recognition result; wherein the lane line recognition model is trained based on lane and road marking benchmarks, and the benchmarks include multiple lanes, road marking categories and a variety of different harsh environment scenarios.

[0069] Furthermore, in another embodiment, the lane line recognition system based on autonomous driving further includes an optimization module, wherein the optimization module is used to:

[0070] Acquire the vehicle's operating signal, and construct the vehicle's actual driving trajectory in combination with the acquired driving parameters;

[0071] Analyzing a predicted driving trajectory of the vehicle based on the driving parameters and the lane line recognition result;

[0072] Then the predicted driving trajectory is matched with the actual driving trajectory to determine whether the trajectories of the two are within a preset deviation range;

[0073] If the deviation range is exceeded, the corresponding lane line recognition result is used as training data to optimize the lane line recognition model.

[0074] When applied, the lane recognition model trains each lane as its own instance during training, and performs image-based learning perspective transformation to achieve recognition of lane changes.

[0075] At the same time, the identification module is also used for:

[0076] Build a lane shape model based on the road structure and the angle from which the image information is acquired;

[0077] During recognition, the lane parameters output by the lane shape model are then passed into the lane line recognition model to achieve accurate positioning of the lane line.

[0078] In another embodiment, the lane line recognition system based on autonomous driving further includes a fusion processing module, wherein the fusion processing module is used to:

[0079] Obtaining navigation information generated by a vehicle-mounted navigation unit; wherein the navigation information includes a destination, a departure point, and a navigation track;

[0080] The navigation information is combined with the current positioning information and the lane line recognition result obtained based on the current image to determine whether there is a deviation;

[0081] If it exists, the trajectory prediction is performed based on the currently obtained lane line recognition result and driving parameters to re-plan the navigation. Through this method, combined with the recognized lane line, it is more accurate to understand which lane the user is in, and then plan the navigation trajectory desired by the user based on the user's intention; similarly, the role of the fusion processing module can also be applied to the aforementioned method embodiment; the scheme of the system embodiment is not limited to autonomous driving, but also applied to a lane line recognition system that requires lane line recognition, and the schemes of the two are the same.

[0082] It should be noted that for more specific workflows of the identification system, please refer to the aforementioned method implementation examples, which will not be described in detail here.

[0083] The above scheme, by adopting a deep learning algorithm, treats the lane detection problem as a segmentation task and performs pixel segmentation training. During training, each lane is trained as its own instance and image-based learning perspective transformation is performed to achieve the recognition of lane changes. At the same time, a lane shape model is established based on the road structure and the angle of obtaining image information to achieve accurate positioning of the lane line, and the vehicle's own signal is introduced for verification, and the recognition model is optimized after the anomaly is identified.

[0084] In this embodiment, in a preferred embodiment of the present application, a lane line recognition system based on automatic driving includes a perception layer and electronic equipment deployed on a vehicle, such as Figure 3 As shown, the electronic device may include: one or more processors 101, one or more input devices 102, one or more output devices 103 and a memory 104, wherein the processors 101, the input devices 102, the output devices 103 and the memory 104 are interconnected via a bus 105. The memory 104 is used to store a computer program, wherein the computer program includes program instructions, and the processor 101 is configured to call the program instructions to execute the aforementioned method steps.

[0085] It should be understood that in the embodiment of the present invention, the processor 101 may be a central processing unit (CPU), and the processor may also be other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.

[0086] The input device 102 may include a keyboard, etc., and the output device 103 may include a display (LCD, etc.), a speaker, etc.

[0087] The memory 104 may include a read-only memory and a random access memory, and provides instructions and data to the processor 101. A portion of the memory 104 may also include a non-volatile random access memory. For example, the memory 104 may also store information on the device type.

[0088] In a specific implementation, the processor 101, input device 102, and output device 103 described in the embodiment of the present invention can execute the implementation method described in the embodiment of the method provided by the embodiment of the present invention, which will not be repeated here.

[0089] Those of ordinary skill in the art will appreciate that the modules and steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in terms of function in the above description. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention.

[0090] In the several embodiments provided in this application, it should be understood that the disclosed methods and systems can be implemented in other ways. For example, the system embodiments described above are only schematic, for example, the division of the modules is only a logical function division, and there may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed.

[0091] The above description is only a specific implementation mode of the present invention, but the protection scope of the present invention is not limited thereto. Any technician familiar with the technical field can easily think of various equivalent modifications or substitutions within the technical scope disclosed by the present invention, and these modifications or substitutions should be included in the protection scope of the present invention.

Claims

1. A lane line recognition method based on automatic driving, characterized in that: The method comprises: Acquire image information; wherein the image information includes information detected by a perception layer deployed on the vehicle; Acquire real-time positioning information and driving parameters of the vehicle, and associate the positioning information with the acquired image information; The associated image is sent to the pre-connected map library for calculation based on the average gray value of the image to obtain a clear image to be identified at the positioning point; The image to be identified is sent to a pre-constructed lane line recognition model for identification to obtain a lane line recognition result; wherein the lane line recognition model is trained based on lane and road marking benchmarks, and the benchmarks include multiple lanes, road marking categories, and multiple different harsh environment scenarios; The method further comprises: Acquire the vehicle's operating signal, and construct the vehicle's actual driving trajectory in combination with the acquired driving parameters; Analyzing a predicted driving trajectory of the vehicle based on the driving parameters and the lane line recognition result; Then the predicted driving trajectory is matched with the actual driving trajectory to determine whether the trajectories of the two are within a preset deviation range; If the deviation range is exceeded, the corresponding lane line recognition result is used as training data to optimize the lane line recognition model.

2. The lane line recognition method based on automatic driving according to claim 1, characterized in that: When training the lane recognition model, each lane is trained as its own instance, and image-based learning perspective transformation is performed to achieve recognition of lane changes.

3. The lane line recognition method based on automatic driving according to claim 2, characterized in that: The method further comprises: Build a lane shape model based on the road structure and the angle from which the image information is acquired; During recognition, the lane parameters output by the lane shape model are then passed into the lane line recognition model to achieve accurate positioning of the lane line.

4. A lane line recognition system based on automatic driving, characterized in that: include: An image acquisition module, used to acquire image information; wherein the image information includes information detected by a perception layer deployed on the vehicle; An information association module, used to obtain real-time positioning information and driving parameters of the vehicle, and associate the positioning information with the acquired image information; The preprocessing module is used to send the associated image to the pre-connected map library for calculation based on the average gray value of the image, so as to obtain a clear image to be identified at the positioning point; A recognition module, used for sending the image to be recognized into a pre-constructed lane recognition model for recognition, so as to obtain a lane recognition result; wherein the lane recognition model is trained based on lane and road marking benchmarks, and the benchmarks include multiple lanes, road marking categories and multiple different harsh environment scenarios; It also includes an optimization module, wherein the optimization module is used to: Acquire the vehicle's operating signal, and construct the vehicle's actual driving trajectory in combination with the acquired driving parameters; Analyzing a predicted driving trajectory of the vehicle based on the driving parameters and the lane line recognition result; Then the predicted driving trajectory is matched with the actual driving trajectory to determine whether the trajectories of the two are within a preset deviation range; If the deviation range is exceeded, the corresponding lane line recognition result is used as training data to optimize the lane line recognition model.

5. The lane line recognition system based on automatic driving according to claim 4, characterized in that: When training the lane recognition model, each lane is trained as its own instance, and image-based learning perspective transformation is performed to achieve recognition of lane changes.

6. A lane line recognition system based on automatic driving as claimed in claim 5, characterized in that: The identification module is also used for: Build a lane shape model based on the road structure and the angle from which the image information is acquired; During recognition, the lane parameters output by the lane shape model are then passed into the lane line recognition model to achieve accurate positioning of the lane line.

7. A lane line recognition system based on autonomous driving, comprising a perception layer and electronic equipment deployed on a vehicle, characterized in that: The electronic device includes one or more processors, one or more input devices, one or more output devices and a memory, wherein the processor, input device, output device and memory are interconnected via a bus, the memory is used to store a computer program, the computer program includes program instructions, and the processor is configured to call the program instructions to execute the method steps described in any one of claims 1-3.

Images