Positioning deviation correction method and device based on vehicle-mounted camera and map truth value

Abstract

The invention relates to a positioning correction method and device based on a vehicle-mounted camera and a map truth value. The method comprises the following steps: acquiring a road image, a high-precision map truth value and a positioning truth value which are acquired by the camera in the driving process of a vehicle; according to the high-precision map truth value and a preset lane distance, whether the positioning truth value is corrected or not is judged; if the positioning truth value is judged to be required to be corrected, identifying a lane and a lane line from the acquired road image, and correcting the positioning truth value according to the quality of the lane line and the distance of the lane line; and according to a difference value between the time stamp of the camera and the time stamp of the positioning truth value, whether the positioning truth value is corrected is judged. According to the method, the positioning truth value is rectified through the camera and the map truth value, the cost is low, the precision is high, and the flexibility is high.

Description

Technical Field

[0001] The present invention belongs to the field of intelligent driving technology, and in particular relates to a positioning and correction method and device based on a vehicle-mounted camera and map true values. Background Art

[0002] Traditional ground truth systems, when using positioning technologies like GPS and inertial navigation, are susceptible to various factors, leading to positioning errors and attitude drift. These errors and drifts can affect the vehicle's positioning accuracy, thereby reducing the accuracy and feasibility of the ground truth.

[0003] Visual positioning technology uses cameras and image processing algorithms to perceive and locate the surrounding environment in real time. By processing and analyzing road environment information captured by the camera, it can accurately perceive and locate parameters such as vehicle position, posture, and driving trajectory.

[0004] Map truth refers to pre-acquired, high-precision map information, including features like road signs and building outlines. Comparing the road environment information captured by cameras with the map truth enables high-precision correction of vehicle position and trajectory, improving positioning accuracy.

[0005] Image processing and computing algorithms play an important role in autonomous driving systems. By extracting, matching, and calculating features from images captured by the camera, accurate identification and correction of the vehicle's position and posture can be achieved.

[0006] Using a camera to correct the true value is an effective technical means. Through visual positioning technology and map true value comparison, it can achieve accurate correction of parameters such as position, posture, and driving trajectory in the true value system, thereby improving the positioning accuracy and stability of the true value system. Summary of the Invention

[0007] In order to solve the problems of positioning error and attitude drift of traditional truth value systems, a first aspect of the present invention provides a positioning correction method based on a vehicle-mounted camera and map truth value, including: obtaining a road image, a high-precision map truth value and a positioning truth value captured by a camera while the vehicle is driving; judging whether to correct the positioning truth value based on the high-precision map truth value and a preset lane distance; if the positioning truth value is judged to need correction, then: identifying lanes and lane lines from the captured road image, and correcting the positioning truth value based on the quality of the lane lines and the distance of the lane lines; judging whether to correct the positioning truth value based on the difference between the timestamp of the camera and the timestamp of the positioning truth value.

[0008] In some embodiments of the present invention, the determination of whether to correct the positioning true value through the high-precision map true value and the preset lane distance includes: determining whether the acquired high-precision map is the latest; if so, determining again whether to correct the positioning true value through the high-precision map true value and the preset lane distance; if not, determining again whether to correct the positioning true value through the high-precision map true value and the preset lane distance.

[0009] Furthermore, the determining whether to correct the positioning true value again based on the high-precision map true value and the preset lane distance includes: extracting the lane geometric information from the high-precision map, and calculating the shortest distance from the vehicle's current position to the lane based on the geometric information; and determining whether to correct the positioning true value based on the shortest distance from the vehicle's current position to the lane and the preset lane distance.

[0010] In some embodiments of the present invention, identifying lanes and lane lines from the captured road image and correcting the positioning true value based on the quality of the lane lines and the distance of the lane lines includes: capturing road images through a camera and identifying lanes and lane lines; filtering and identifying the road images in sequence based on the quality and position of the lane lines; and correcting the positioning true value based on the recognition results and the preset lane width.

[0011] Furthermore, the correcting of the positioning true value based on the recognition result and the preset lane width includes: calculating the current lane width based on the identified left and right lane lines; correcting the positioning true value based on the current lane width and the preset lane width: if the current lane width is between the first preset value and the first preset value, then the lane width is judged to be correct; otherwise, the positioning true value corresponding to the current frame image is discarded.

[0012] In the above embodiment, judging whether to correct the positioning true value based on the difference between the timestamp of the camera and the timestamp of the positioning true value includes: if the difference between the timestamp of the camera and the timestamp of the positioning true value is less than or equal to a preset value, correcting the positioning true value.

[0013] The second aspect of the present invention provides a positioning and correction device based on a vehicle-mounted camera and a map truth value, comprising: an acquisition module for acquiring a road image, a high-precision map truth value, and a positioning truth value captured by a camera while the vehicle is driving; a judgment module for judging whether to correct the positioning truth value based on the high-precision map truth value and a preset lane distance; a first correction module for, if the positioning truth value is judged to require correction, identifying lanes and lane lines from the captured road image, and correcting the positioning truth value based on the quality of the lane lines and the distance of the lane lines; a second correction module for judging whether to correct the positioning truth value based on the difference between the timestamp of the camera and the timestamp of the positioning truth value.

[0014] The third aspect of the present invention provides an electronic device comprising: one or more processors; a storage device for storing one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors implement the positioning and correction method based on the vehicle-mounted camera and the map true value provided in the first aspect of the present invention.

[0015] A fourth aspect of the present invention provides a computer-readable medium having a computer program stored thereon, wherein when the computer program is executed by a processor, the positioning and correction method based on a vehicle-mounted camera and a true map value provided in the first aspect of the present invention is implemented.

[0016] The beneficial effects of the present invention are:

[0017] In a first aspect of the present invention, a positioning and correction method based on a vehicle-mounted camera and a map truth value is provided, comprising: obtaining a road image, a high-precision map truth value, and a positioning truth value captured by a camera while the vehicle is driving; judging whether to correct the positioning truth value based on the high-precision map truth value and a preset lane distance; if the positioning truth value is judged to require correction, then: identifying lanes and lane lines from the captured road image, and correcting the positioning truth value based on the quality of the lane lines and the distance of the lane lines; judging whether to correct the positioning truth value based on the difference between the timestamp of the camera and the timestamp of the positioning truth value.

[0018] It can be seen that the present invention has the following advantages:

[0019] Low cost: Cameras are relatively low-cost sensors. Compared with some specific positioning sensors, such as lidar, cameras may be cheaper, which can reduce the cost of the entire high-precision positioning system. High precision: Cameras can provide high-resolution image data. Combined with computer vision technology, they can accurately extract features such as road markings, lane lines, and traffic signs, thereby achieving high-precision positioning and correction. Strong flexibility: If the vehicle is equipped with a camera device, there is no need for an additional camera device. The vehicle's own camera device can be directly used for correction, which is highly flexible. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] Figure 1 Schematic diagram of the basic flow of a positioning and correction method based on a vehicle-mounted camera and map truth in some embodiments of the present invention;

[0021] Figure 2 Schematic diagram of a specific process of a positioning and correction method based on a vehicle-mounted camera and a true map value in some embodiments of the present invention;

[0022] Figure 3Schematic diagram of the structure of a positioning and correction device based on a vehicle-mounted camera and a true map value in some embodiments of the present invention;

[0023] Figure 4 Schematic diagram of the specific structure of a positioning and correction device based on a vehicle-mounted camera and a true map value in some embodiments of the present invention;

[0024] Figure 5 Schematic diagram of the structure of an electronic device in some embodiments of the present invention. DETAILED DESCRIPTION

[0025] The principles and features of the present invention are described below with reference to the accompanying drawings. The examples given are only used to explain the present invention and are not used to limit the scope of the present invention.

[0026] refer to Figure 1 and Figure 2 In a first aspect of the present invention, a positioning and correction method based on a vehicle-mounted camera and a map truth value is provided, comprising: S100. Acquiring a road image, a high-precision map truth value, and a positioning truth value captured by a camera while the vehicle is driving; S200. Judging whether to correct the positioning truth value based on the high-precision map truth value and a preset lane distance; S300. If the positioning truth value is judged to require correction, then: identifying lanes and lane lines from the captured road image, and correcting the positioning truth value based on the quality of the lane lines and the distance of the lane lines; S400. Judging whether to correct the positioning truth value based on the difference between the timestamp of the camera and the timestamp of the positioning truth value.

[0027] In step S100 of some embodiments of the present invention, a road image, a high-precision map truth value, and a positioning truth value captured by a camera while the vehicle is driving are obtained. Specifically, vehicle road information, including lane information and lane line information, is collected; the positioning truth value is provided by a positioning truth value system, which is used to provide coarse vehicle position information, including position information obtained through RTK (real-time kinematic positioning) and IMU (inertial measurement unit) sensor fusion; the high-precision map truth value includes the actual position information of map markers and landmarks, as well as the road topology and other relevant geographic information, which is searched and matched with the road image.

[0028] In step S200 of some embodiments of the present invention, determining whether to correct the true positioning value based on the high-precision map true value and the preset lane distance includes:

[0029] S201. Determine whether the acquired high-precision map is the latest one: If so, determine whether to correct the positioning true value again based on the high-precision map true value and the preset lane distance;

[0030] Specifically, the latest map cache is obtained. If the latest high-precision map cache is not obtained, no correction processing is performed.

[0031] If not, determine again whether to correct the positioning true value based on the high-precision map true value and the preset lane distance.

[0032] Furthermore, the determining whether to correct the positioning true value again based on the high-precision map true value and the preset lane distance includes: extracting the lane geometric information from the high-precision map, and calculating the shortest distance from the vehicle's current position to the lane based on the geometric information; and determining whether to correct the positioning true value based on the shortest distance from the vehicle's current position to the lane and the preset lane distance.

[0033] Specifically, the map data is traversed to obtain lane information, including lane ID and geometry. For each lane, the geometry is converted to coordinates in the ENU coordinate system, and the shortest distance from the vehicle's current position to the lane is calculated. If the shortest distance is greater than 10, it indicates that no suitable lane can be found and no correction is performed. If the shortest distance is less than 10, the process continues. It should be noted that the shortest distance can be adjusted based on actual conditions.

[0034] In step S300 of some embodiments of the present invention, identifying lanes and lane lines from the acquired road image and correcting the true positioning value based on the quality of the lane lines and the distance of the lane lines includes:

[0035] S301. Capture a road image through a camera and identify lanes and lane lines;

[0036] Specifically, the current vehicle road and lane line information is obtained through the camera road information acquisition unit.

[0037] S302. Filter and identify the road image in sequence according to the quality and position of the lane lines;

[0038] Specifically, the quality of the currently recognized lane line is judged. If the lane line quality is too poor or unclear, the lane line data of this frame is discarded; if the lane line quality is good, continue.

[0039] S303. Correct the true positioning value based on the recognition result and the preset lane width.

[0040] Specifically, in step S303, the correction of the true positioning value according to the recognition result and the preset lane width includes:

[0041] S3031. Calculate the current lane width based on the identified left and right lane lines;

[0042] S3032. Correct the positioning true value based on the current lane width and the preset lane width: if the current lane width is between the first preset value and the first preset value, the lane width is judged to be correct; otherwise, the positioning true value corresponding to the current frame image is discarded.

[0043] Specifically, the left and right lane markings are identified. If both the upper and left lane markings are identified, the process continues. If both the upper and left lane markings cannot be identified, the frame data is discarded. The distances between the left and right lane markings, ldist and rdist, are calculated, and the sum of ldist and rdist is used as the current lane width. If the calculated lane width is between 2.4 and 6.0, the lane width is considered correct and the process continues. If the lane width is less than 2.4 or greater than 6.0, the lane width is considered incorrect and the frame data is discarded.

[0044] In step S400 of some embodiments of the present invention, judging whether to correct the positioning true value based on the difference between the timestamp of the camera and the timestamp of the positioning true value includes: if the difference between the timestamp of the camera and the timestamp of the positioning true value is less than or equal to a preset value, correcting the positioning true value.

[0045] Specifically, if the difference between the true value timestamp and the camera timestamp is large, for example, greater than 105 milliseconds, no correction processing is performed.

[0046] It can be understood that the camera can collect image data of the road environment in real time, and through real-time calculation and processing, it can achieve real-time correction of the vehicle's position and driving trajectory, meeting the real-time accuracy comparison requirements of autonomous driving and other applications requiring high-precision positioning. This invention improves the lateral accuracy of the high-precision positioning true value through camera recognition technology, making the true value data more comparable, providing more accurate true value data for the accuracy comparison of positioning systems in the autonomous driving field, and can be widely used in fields such as autonomous driving.

[0047] Example 2

[0048] refer to Figure 4 According to a second aspect of the present invention, a positioning and correction device 1 based on a vehicle-mounted camera and a map truth value is provided, comprising: an acquisition module 11 for acquiring a road image, a high-precision map truth value, and a positioning truth value captured by a camera while the vehicle is driving; a judgment module 12 for judging whether to correct the positioning truth value based on the high-precision map truth value and a preset lane distance; a first correction module 13 for, if the positioning truth value is judged to require correction, identifying lanes and lane lines from the captured road image, and correcting the positioning truth value based on the quality of the lane lines and the distance of the lane lines; and a second correction module 14 for judging whether to correct the positioning truth value based on the difference between the timestamp of the camera and the timestamp of the positioning truth value.

[0049] Furthermore, the second correction module 14 includes: an identification unit, which is used to capture road images through a camera and identify lanes and lane lines; a filtering unit, which is used to filter and identify road images in sequence according to the quality and position of lane lines; and a correction unit, which is used to correct the positioning true value according to the recognition result and the preset lane width.

[0050] refer to Figure 3 , in one embodiment of the present invention, a positioning and correction device 1 based on a vehicle-mounted camera and a true map value includes: a camera road information acquisition unit (10), for acquiring vehicle road information, including lane information and lane line information;

[0051] The central information processing unit (20) is used to extract features from the collected camera data and compare the extracted features with the true map values, perform feature matching, image registration and other processing, and then call a correction algorithm to correct the vehicle position, posture and driving trajectory based on the comparison results between the camera data and the true map values;

[0052] A general positioning truth value system (30) is used to provide rough position information of the vehicle, including position information obtained by fusion of RTK (real-time kinematic positioning) and IMU (inertial measurement unit) sensors;

[0053] The high-precision map information unit (40), including the real location information of map markers and landmarks, as well as the topological structure of the road and other relevant geographic information, is searched and matched with the camera perception system (20).

[0054] Example 3

[0055] refer to Figure 5 According to a third aspect of the present invention, an electronic device is provided, comprising: one or more processors; a storage device for storing one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors implement the positioning and correction method based on the vehicle-mounted camera and the true map value according to the first aspect of the present invention.

[0056] The electronic device 500 may include a processing device (e.g., a central processing unit, a graphics processing unit, etc.) 501, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 502 or a program loaded from a storage device 508 into a random access memory (RAM) 503. Various programs and data required for the operation of the electronic device 500 are also stored in the RAM 503. The processing device 501, the ROM 502, and the RAM 503 are connected to each other via a bus 504. An input / output (I / O) interface 505 is also connected to the bus 504.

[0057] Typically, the following devices may be connected to the I / O interface 505: an input device 506 including, for example, a touch screen, a touchpad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; an output device 507 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; a storage device 508 including, for example, a hard disk, etc.; and a communication device 509. The communication device 509 may allow the electronic device 500 to communicate with other devices wirelessly or by wire to exchange data. Figure 5 The electronic device 500 is shown with various devices, but it should be understood that it is not required to implement or possess all of the devices shown. More or fewer devices may be implemented or possessed instead. Figure 5 Each block shown in the figure may represent one device, or may represent multiple devices as needed.

[0058] In particular, according to an embodiment of the present disclosure, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present disclosure includes a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from the network through the communication device 509, or installed from the storage device 508, or installed from the ROM 502. When the computer program is executed by the processing device 501, the above-mentioned functions defined in the method of the embodiment of the present disclosure are executed. It should be noted that the computer-readable medium described in the embodiment of the present disclosure can be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium can be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to, an electrical connection having one or more conductors, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In embodiments of the present disclosure, a computer-readable storage medium may be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. In embodiments of the present disclosure, a computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, which carries computer-readable program code. Such a propagated data signal may take a variety of forms, including, but not limited to, an electromagnetic signal, an optical signal, or any suitable combination thereof. A computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium that can transmit, propagate, or transfer a program for use by or in conjunction with an instruction execution system, apparatus, or device. Program code embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to wire, optical cable, RF (radio frequency), etc., or any suitable combination thereof.

[0059] The computer-readable medium may be included in the electronic device, or may exist independently without being incorporated into the electronic device. The computer-readable medium carries one or more computer programs, which, when executed by the electronic device, cause the electronic device to:

[0060] Computer program code for performing the operations of embodiments of the present disclosure may be written in one or more programming languages, or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++, Python, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a stand-alone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., through the Internet using an Internet service provider).

[0061] The flowcharts and block diagrams in the accompanying drawings illustrate the possible implementation architecture, functions and operations of the systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each box in the flowchart or block diagram can represent a module, program segment, or a part of code, and the module, program segment, or a part of code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order than that marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should be noted that each box in the block diagram and / or flowchart, and the combination of the boxes in the block diagram and / or flowchart, can be implemented with a dedicated hardware-based system that performs the specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0062] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A positioning and correction method based on a vehicle-mounted camera and a true map value, characterized in that: include: Obtain road images, high-precision map truth values, and positioning truth values ​​collected by the camera while the vehicle is driving; Determine whether to correct the positioning true value based on the high-precision map true value and the preset lane distance; If the true positioning value is determined to need deviation correction, then: identifying lanes and lane lines from the collected road image, and correcting the true positioning value according to the quality of the lane lines and the distance of the lane lines; Based on the difference between the timestamp of the camera and the timestamp of the true positioning value, it is determined whether to correct the true positioning value.

2. The positioning and correction method based on vehicle-mounted camera and map true value according to claim 1 is characterized in that: The determining whether to correct the positioning true value by using the high-precision map true value and the preset lane distance includes: Determine whether the acquired high-precision map is the latest one: If so, determine whether to correct the positioning true value again based on the high-precision map true value and the preset lane distance; If not, the high-precision map true value and the preset lane distance are used to determine whether the positioning true value should be corrected.

3. The positioning and correction method based on vehicle-mounted camera and map true value according to claim 2 is characterized in that: The determining whether to correct the positioning true value again by using the high-precision map true value and the preset lane distance includes: Extracting lane geometry information from a high-precision map, and calculating the shortest distance from the vehicle's current position to the lane based on the geometry information; According to the shortest distance from the current position of the vehicle to the lane and the preset lane distance, it is determined whether to correct the positioning true value.

4. The positioning and correction method based on vehicle-mounted camera and map true value according to claim 1 is characterized in that: The identifying of lanes and lane lines from the collected road image and correcting the positioning true value according to the quality of the lane lines and the distance of the lane lines includes: The camera collects road images and identifies lanes and lane lines; The road image is filtered and recognized in turn according to the quality and position of the lane lines; According to the recognition results and the preset lane width, the positioning true value is corrected.

5. The positioning and correction method based on vehicle-mounted camera and map true value according to claim 4 is characterized in that: The correcting of the positioning true value according to the recognition result and the preset lane width includes: Calculate the current lane width based on the identified left and right lane lines; According to the current lane width and the preset lane width, the positioning true value is corrected: if the current lane width is between the first preset value and the first preset value, the lane width is judged to be correct; otherwise, the positioning true value corresponding to the current frame image is discarded.

6. The positioning and correction method based on vehicle-mounted camera and map true value according to claim 1 is characterized in that: The difference between the timestamp of the camera and the timestamp of the true positioning value is used to determine whether to correct the true positioning value, including: If the difference between the timestamp of the camera and the timestamp of the true positioning value is less than or equal to the preset value, the true positioning value is corrected.

7. A positioning and correction device based on a vehicle-mounted camera and a true map value, characterized in that: include: The acquisition module is used to obtain the road image, high-precision map truth value and positioning truth value collected by the camera while the vehicle is driving; A judgment module is used to judge whether to correct the positioning true value based on the high-precision map true value and the preset lane distance; A first deviation correction module is used for, if the positioning true value is judged to need deviation correction, identifying lanes and lane lines from the collected road image, and correcting the positioning true value according to the quality of the lane line and the distance of the lane line; The second correction module is used to determine whether to correct the positioning true value according to the difference between the timestamp of the camera and the timestamp of the positioning true value.

8. The positioning and deviation correction device based on a vehicle-mounted camera and a true map value according to claim 7, characterized in that: The second correction module comprises: A recognition unit, used to collect road images through a camera and recognize lanes and lane lines; A filtering unit, for filtering and identifying the road image in sequence according to the quality and position of the lane lines; The correction unit is used to correct the positioning true value according to the recognition result and the preset lane width.

9. An electronic device, comprising: one or more processors; A storage device for storing one or more programs, which, when executed by the one or more processors, enables the one or more processors to implement the positioning and correction method based on a vehicle-mounted camera and a true map value as described in any one of claims 1 to 6.

10. A computer readable medium having a computer program stored thereon, wherein: When the computer program is executed by a processor, the positioning and correction method based on a vehicle-mounted camera and a true map value is implemented as described in any one of claims 1 to 6.

Images