A multi-source matching and positioning method, system, electronic device and storage medium

Abstract

The present invention provides a multi-source matching and positioning method, system, electronic device and storage medium. The method includes: after system initialization, based on the current position information, obtaining a high-precision map of the corresponding local range, verifying whether the current single-frame real-time feature elements are valid, performing a first matching and fusion of the single-frame real-time feature elements with the high-precision map data, performing three-dimensional reconstruction on the local scene of the current position, performing a second matching and fusion of the local scene element information with the high-precision map data, correcting abnormal matching results according to the position correction information, matching confidence, and single-frame feature element quality inspection results of the two fusion matches, and obtaining a multi-source fusion positioning result based on the corrected matching results combined with IMU, GNSS, and VCU information. Through this solution, the problem of the failure of camera single-frame perception matching and positioning in specific scenarios can be solved, and the accuracy and reliability of positioning can be guaranteed through multi-source matching and positioning, meeting the high-precision positioning requirements in different scenarios.

Description

Technical Field

[0001] The present invention belongs to the technical field of positioning, and particularly relates to a multi-source matching positioning method, system, electronic device and storage medium. Background Art

[0002] With the wide application of intelligent driving services, the requirements of intelligent driving systems for the accuracy, stability and real-time performance of positioning are also getting higher and higher. Currently, a multi-sensor matching positioning system based on the single-frame perception result of a camera can perfectly solve the positioning error problem caused by abnormal GNSS-RTK signals in the area where the camera real-time perception is normal and the high-precision map coverage is available. However, in some scenarios where road features are not significant or the real-time perception result of the camera is invalid due to factors such as traffic jams and backlighting, the matching positioning system based on single-frame camera perception may also fail, resulting in low real-time positioning accuracy and reliability. Summary of the Invention

[0003] In view of this, embodiments of the present invention provide a multi-source matching positioning method, system, electronic device and storage medium, which are used to solve the problem of low positioning accuracy and reliability of the existing matching positioning based on single-frame camera perception in specific scenarios.

[0004] In the first aspect of the embodiments of the present invention, a multi-source matching positioning method is provided, including:

[0005] Receiving single-frame real-time feature element information, IMU information, GNSS information and VCU information, and initializing the positioning system based on the IMU information and GNSS information;

[0006] Obtaining high-precision map data of a corresponding local range based on the current position information;

[0007] Verifying whether the current single-frame real-time feature element is valid, and saving the corresponding quality inspection result;

[0008] Performing a first matching fusion according to the single-frame real-time feature element information, IMU information, GNSS information, VCU information and high-precision map data to obtain the horizontal and vertical correction information, attitude information and matching confidence of the current position;

[0009] Transmitting the corrected position information and attitude information after the first matching fusion to the real-time three-dimensional reconstruction module, and obtaining local scene element information of the corresponding position through three-dimensional reconstruction;

[0010] If the local scene elements meet the predetermined matching requirements, a second matching fusion is performed based on the local scene element information, the corrected position information after the first matching fusion and the high-precision map information;

[0011] Based on the horizontal and vertical correction information, matching confidence, and single-frame real-time feature quality inspection results output by the first matching fusion and the second matching fusion, correct the abnormal matching results;

[0012] Fuse the corrected matching results with the IMU information, GNSS information, and VCU information to obtain the final speed, position, and attitude information.

[0013] In the second aspect of the embodiments of the present invention, a multi-source matching positioning system is provided, including:

[0014] An initialization module, configured to receive single-frame real-time feature element information, IMU information, GNSS information, and VCU information, and initialize the positioning system based on the IMU information and GNSS information;

[0015] A map acquisition module, configured to acquire high-precision map data of a corresponding local range based on the current position information;

[0016] An element verification module, configured to verify whether the current single-frame real-time feature element is valid and save the corresponding quality inspection results;

[0017] A first matching fusion module, configured to perform first matching fusion according to the single-frame real-time feature element information, IMU information, GNSS information, VCU information, and high-precision map data to obtain the horizontal and vertical correction information, attitude information, and matching confidence of the current position;

[0018] A 3D reconstruction module, configured to obtain local scene element information of the corresponding position through 3D reconstruction based on the corrected position information and attitude information after the first matching fusion;

[0019] A second matching fusion module, configured to perform second matching fusion based on the local scene element information, the corrected position information after the first matching fusion, and the high-precision map information if the local scene elements meet the predetermined matching requirements;

[0020] An abnormal correction module, configured to correct the abnormal matching results according to the horizontal and vertical correction information, matching confidence, and single-frame real-time feature quality inspection results output by the first matching fusion and the second matching fusion;

[0021] A fusion positioning module, configured to fuse the corrected matching results with the IMU information, GNSS information, and VCU information to obtain the final speed, position, and attitude information.

[0022] In the third aspect of the embodiments of the present invention, an electronic device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the steps of the method described in the first aspect of the embodiments of the present invention are implemented.

[0023] In the fourth aspect of the embodiments of the present invention, a computer-readable storage medium is provided. The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the method provided in the first aspect of the embodiments of the present invention are implemented.

[0024] In the embodiments of the present invention, by respectively matching and fusing the real-time feature element information sensed by the camera and the local scene element information of the three-dimensional reconstruction with the high-precision map, and based on the results of the two matching fusions and the single-frame element quality inspection results, the abnormal matching results are corrected. Based on the corrected matching results and combined with data such as IMU and GNSS, multi-source matching positioning is achieved, thereby solving the problem that the matching positioning of the single-frame perception of the camera fails in specific scenarios. Through multi-source matching positioning, the accuracy, reliability, and real-time performance of the positioning can be guaranteed when the camera perception fails, meeting the high-precision positioning requirements in different scenarios. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following will briefly introduce the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other drawings can be obtained based on these drawings.

[0026] Figure 1 It is a schematic flowchart of a multi-source matching positioning method provided by an embodiment of the present invention;

[0027] Figure 2 It is a schematic structural diagram of a multi-source matching positioning system provided by an embodiment of the present invention;

[0028] Figure 3 It is a schematic structural diagram of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0029] In order to make the objectives, features, and advantages of the present invention more obvious and understandable, the following will clearly and completely describe the technical solutions in the embodiments of the present invention with reference to the drawings in the embodiments of the present invention. Obviously, the embodiments described below are only some embodiments of the present invention, rather than all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts belong to the scope of protection of the present invention.

[0030] It should be understood that the term "including" and other similar expressions in the specification or claims of the present invention and the above-mentioned drawings mean covering non-exclusive inclusion. For example, a process, method, system, or device that includes a series of steps or units is not limited to the listed steps or units. In addition, "first" and "second" are used to distinguish different objects and are not used to describe a specific order.

[0031] Please refer to Figure 1 , a schematic flowchart of a multi-source matching and positioning method provided by an embodiment of the present invention, including:

[0032] S101. Receive single-frame real-time feature element information, IMU information, GNSS information, and VCU information, and initialize the positioning system based on the IMU information and GNSS information;

[0033] The single-frame real-time feature element information refers to the real-time perception result of the vehicle-mounted camera on the road, and the feature elements may include lane lines, traffic signs, ground arrows, stop lines, etc. The IMU (Inertial Measurement Unit) information is the three-axis attitude angle and acceleration information of the object sensed by the inertial measurement unit; the GNSS (Global Navigation Satellite System) information is the navigation satellite positioning information; the VCU (Vehicle control unit) information is the vehicle control decision information output by the vehicle control unit, such as vehicle acceleration, deceleration and other control information.

[0034] Based on the attitude information measured by GNSS positioning and IMU, the speed, position, attitude, etc. of the positioning system can be initialized. Combining the real-time IMU information and GNSS information, the real-time positioning of the vehicle can be calculated.

[0035] S102. Based on the current position information, obtain high-precision map data of the corresponding local area;

[0036] According to the vehicle positioning calculated by the positioning system, send the current position information to the high-precision map engine to obtain high-precision map data within a certain range around the current position. The high-precision map of the local area contains not only basic map information but also the absolute position information (i.e., longitude and latitude) of each traffic element (feature element).

[0037] S103. Verify whether the current single-frame real-time feature element is valid and save the corresponding quality inspection result;

[0038] The quality inspection verifies the validity of the real-time feature elements collected by the vehicle-mounted camera to filter out significantly abnormal perception results, that is, to filter out the feature elements with abnormal perception and avoid feature element fusion matching

[0039] The influence of matching.

[0040] Among them, according to the change trend of historical single-frame feature element information and the change trend of vehicle movement, comprehensively judge whether the current single-frame real-time feature element is valid.

[0041] The change trend of the historical single-frame feature element information is the data of the historical camera's perception process 0 of the feature element, such as the historical perception frame data of the same scene position. The vehicle movement change trend refers to the vehicle's

[0042] operating information such as movement speed, acceleration, and heading angle. Based on the historical element perception data and vehicle movement data, it can be judged whether the perception result of the current single-frame feature element is valid. For example, in the case of occlusion of traffic elements, it can be determined that the perception result of the single-frame real-time feature element is invalid.

[0043] S104. Perform a primary matching fusion based on the single-frame real-time feature element information, IMU information, GNSS information, VCU information, and 5 high-precision map data to obtain the lateral and longitudinal correction information, attitude

[0044] information, and matching confidence of the current position;

[0045] Based on the IMU information, GNSS information, VCU information, etc., by matching the feature elements perceived in real-time single frames with the elements in the high-precision map, the vehicle position, attitude and other information can be corrected,

[0046] to obtain the vehicle positioning, attitude and other information after the primary fusion matching, as well as the feature element matching confidence. 0S105. Transmit the corrected position information and attitude information after the primary matching fusion to the real-time three-dimensional re

[0047] construction module, and obtain the local scene element information of the corresponding position through three-dimensional reconstruction;

[0048] Based on the position and attitude after the primary fusion matching, combined with the feature element perception result, three-dimensional reconstruction can be performed to obtain the local scene element information of the current position.

[0049] S106. If the local scene elements meet the predetermined matching requirements, perform a secondary matching fusion based on the local scene element information, 5 the position information corrected by the primary matching fusion, and the high-precision map information;

[0050] Among them, by comparing the local camera perception result with the high-precision map information, judge whether the local scene elements meet the predetermined matching requirements. Since the local scene elements obtained by three-dimensional reconstruction are based on the perception result of the vehicle-mounted camera, the single-frame element perception of the vehicle-mounted camera can be directly compared with the elements at the corresponding position in the high-precision map.

[0051] If the matching requirements are met, such as the similarity of feature comparison reaching a predetermined value, it can be considered that the two match. Then, the local scene features of the 3D reconstruction can be fused and matched with the feature elements in the high-precision map by combining the position information after the first fusion match. Through the second matching fusion, the comparison of feature elements can be carried out under the camera view (or vehicle coordinate system) to ensure the reliability of the matching result.

[0052] S107. Correct the abnormal matching results according to the horizontal and vertical correction information, matching confidence, and single-frame real-time feature element quality inspection results output by the first matching fusion and the second matching fusion;

[0053] Based on the position information, attitude information, and matching confidence after the two fusion matches, correct the abnormal perception results of the feature elements in the quality inspection, so as to perform fusion positioning through the corrected single-frame perception results.

[0054] S108. Fuse the corrected matching results with the IMU information, GNSS information, and VCU information to obtain the final speed, position, and attitude information.

[0055] Based on the corrected camera perception matching results, combined with the real-time IMU information, GNSS information, and VCU information, fusion positioning can be carried out to obtain accurate speed, position, attitude, and other information.

[0056] In this embodiment, comprehensively considering the single-frame real-time feature element information and the local scene feature information of the 3D reconstruction, respectively matching with the high-precision map, and realizing fusion positioning based on the complementarity between multi-source heterogeneous data, avoiding the problems realized by the camera in specific scenarios, ensuring the accuracy, reliability, and real-time performance of positioning, and meeting the high-precision positioning requirements in different scenarios.

[0057] It should be understood that the sequence numbers of the steps in the above embodiments do not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation to the implementation process of the embodiments of the present invention.

[0058] Figure 2 FIG. is a schematic structural diagram of a multi-source matching positioning system provided by an embodiment of the present invention. The system includes:

[0059] An initialization module 210, configured to receive single-frame real-time feature element information, IMU information, GNSS information, and VCU information, and initialize the positioning system based on the IMU information and GNSS information;

[0060] A map acquisition module 220, configured to acquire high-precision map data of a corresponding local range based on the current position information;

[0061] The element verification module 230 is used to verify whether the current single-frame real-time feature elements are valid and save the corresponding quality inspection results;

[0062] Among them, according to the change trend of historical single-frame feature element information and the change trend of vehicle movement, it is comprehensively judged whether the current single-frame real-time feature elements are valid.

[0063] The primary matching and fusion module 240 is used to perform primary matching and fusion based on the single-frame real-time feature element information, IMU information, GNSS information, VCU information, and high-precision map data to obtain the lateral and longitudinal correction information, attitude information, and matching confidence of the current position;

[0064] The 3D reconstruction module 250 is used to obtain the local scene element information of the corresponding position through 3D reconstruction based on the corrected position information and attitude information after primary matching and fusion;

[0065] The secondary matching and fusion module 260 is used to perform secondary matching and fusion based on the local scene element information, the position information corrected by primary matching and fusion, and the high-precision map information if the local scene elements meet the predetermined matching requirements;

[0066] Among them, the secondary matching and fusion module includes:

[0067] The comparison and judgment unit is used to judge whether the local scene elements meet the predetermined matching requirements by comparing the local camera perception results with the high-precision map information.

[0068] The abnormal correction module 270 is used to correct the abnormal matching results according to the lateral and longitudinal correction information, matching confidence, and single-frame real-time feature element quality inspection results output by primary matching and fusion and secondary matching and fusion;

[0069] The fusion positioning module 280 is used to fuse the corrected matching results with the IMU information, GNSS information, and VCU information to obtain the final speed, position, and attitude information.

[0070] Those skilled in the art can clearly understand that for the convenience and conciseness of description, the specific working processes of the above-described systems and modules can refer to the corresponding processes in the foregoing method embodiments, and will not be elaborated herein.

[0071] Figure 3 is a schematic structural diagram of an electronic device provided by an embodiment of the present invention. The electronic device is used for multi-source fusion matching and positioning. As Figure 3 shown, the electronic device 3 of this embodiment includes: a memory 310, a processor 320, and a system bus 330. The memory 310 includes a program 3101 that can run thereon. Those skilled in the art can understand, Figure 3The structure of the electronic device shown does not limit the electronic device, which may include more or fewer components than shown, or combine certain components, or have different component arrangements.

[0072] The following will specifically introduce each component of the electronic device in conjunction with Figure 3 :

[0073] The memory 310 can be used to store software programs and modules. The processor 320 executes various functional applications and data processing of the electronic device by running the software programs and modules stored in the memory 310. The memory 310 mainly includes a program storage area and a data storage area. Among them, the program storage area can store an operating system, application programs required for at least one function (such as a sound playback function, an image playback function, etc.); the data storage area can store data created according to the use of the electronic device (such as cached data, etc.). In addition, the memory 310 can include a high-speed random access memory, and can also include a non-volatile memory, such as at least one magnetic disk storage device, a flash memory device, or other volatile solid-state storage devices.

[0074] A runnable program 3101 containing a network request method is included in the memory 310. The runnable program 3101 can be divided into one or more modules / units. The one or more modules / units are stored in the memory 310 and executed by the processor 320 to perform multi-source data fusion matching positioning, etc. The one or more modules / units can be a series of computer program instruction segments capable of completing specific functions, and these instruction segments are used to describe the execution process of the computer program 3101 in the electronic device 3. For example, the computer program 3101 can be divided into functional modules such as an initialization module, a map acquisition module, a feature verification module, a primary matching fusion module, a 3D reconstruction module, a secondary matching fusion module, and a fusion positioning module.

[0075] The processor 320 is the control center of the electronic device, connecting various parts of the entire electronic device through various interfaces and lines. By running or executing the software programs and / or modules stored in the memory 310, and calling the data stored in the memory 310, it executes various functions of the electronic device and processes data, thereby monitoring the overall state of the electronic device. Optionally, the processor 320 can include one or more processing units; preferably, the processor 320 can integrate an application processor and a modem processor. Among them, the application processor mainly processes the operating system, application programs, etc., and the modem processor mainly processes wireless communication. It can be understood that the above modem processor may not be integrated into the processor 320 either.

[0076] The system bus 330 is used to connect the various functional components inside the computer, and can transmit data information, address information, and control information. Its types can be, for example, PCI bus, ISA bus, CAN bus, etc. The instructions of the processor 320 are transmitted to the memory 310 through the bus, and the memory 310 feeds back data to the processor 320. The system bus 330 is responsible for the data and instruction interaction between the processor 320 and the memory 310. Of course, the system bus 330 can also be connected to other devices, such as network interfaces, display devices, etc.

[0077] In the embodiment of the present invention, the executable program executed by the processor 320 included in the electronic device includes:

[0078] Receiving single-frame real-time feature element information, IMU information, GNSS information, and VCU information, and initializing the positioning system based on the IMU information and GNSS information;

[0079] Based on the current position information, obtaining high-precision map data of the corresponding local range;

[0080] Verifying whether the current single-frame real-time feature element is valid, and saving the corresponding quality inspection result;

[0081] Performing a first matching fusion based on the single-frame real-time feature element information, IMU information, GNSS information, VCU information, and high-precision map data to obtain the lateral and longitudinal correction information, attitude information, and matching confidence of the current position;

[0082] Transmitting the corrected position information and attitude information after the first matching fusion to the real-time three-dimensional reconstruction module, and obtaining the local scene element information of the corresponding position through three-dimensional reconstruction;

[0083] If the local scene elements meet the predetermined matching requirements, performing a second matching fusion based on the local scene element information, the corrected position information after the first matching fusion, and the high-precision map information;

[0084] Correcting the abnormal matching results according to the lateral and longitudinal correction information, matching confidence, and single-frame real-time feature element quality inspection results output by the first matching fusion and the second matching fusion;

[0085] Fusing the corrected matching results with the IMU information, GNSS information, and VCU information to obtain the final speed, position, and attitude information.

[0086] Those skilled in the art can clearly understand that for the convenience and simplicity of description, the specific working processes of the systems, devices, and modules described above can refer to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0087] In the above embodiments, the descriptions of the respective embodiments have their own focuses. For parts not described or recorded in detail in a certain embodiment, reference may be made to the relevant descriptions of other embodiments.

[0088] As described above, the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that: they can still modify the technical solutions recorded in the foregoing embodiments, or

[0089] equivalently replace some of the technical features therein; and these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A multi-source matching and positioning method, characterized in that, Including: Receiving single-frame real-time feature element information, IMU information, GNSS information, and VCU information, and initializing the positioning system based on the IMU information and GNSS information; Obtaining high-precision map data for the corresponding local range based on the current position information; Verifying whether the current single-frame real-time feature element is valid and saving the corresponding quality inspection result; Performing a first matching and fusion based on the single-frame real-time feature element information, IMU information, GNSS information, VCU information, and high-precision map data to obtain the lateral and longitudinal correction information, attitude information, and matching confidence of the current position; Transmitting the corrected position information and attitude information after the first matching and fusion to the real-time three-dimensional reconstruction module, and obtaining the local scene element information of the corresponding position through three-dimensional reconstruction; If the local scene elements meet the predetermined matching requirements, performing a second matching and fusion based on the local scene element information, the position information corrected by the first matching and fusion, and the high-precision map information; Correcting the abnormal matching results according to the lateral and longitudinal correction information, matching confidence, and single-frame real-time feature element quality inspection results output by the first matching and fusion and the second matching and fusion; Fusing the corrected matching results with the IMU information, GNSS information, and VCU information to obtain the final speed, position, and attitude information.

2. The method according to claim 1, characterized in that, The verifying whether the current single-frame real-time feature element is valid and saving the corresponding quality inspection result includes: Comprehensively judging whether the current single-frame real-time feature element is valid according to the change trend of historical single-frame feature element information and the change trend of vehicle movement.

3. The method according to claim 1, characterized in that, The if the local scene elements meet the predetermined matching requirements, performing a second matching and fusion based on the local scene element information, the position information corrected by the first matching and fusion, and the high-precision map information includes: Judging whether the local scene elements meet the predetermined matching requirements by comparing the local camera perception results with the high-precision map information.

4. A multi-source matching and positioning system, characterized in that, At least including: An initialization module for receiving single-frame real-time feature element information, IMU information, GNSS information, and VCU information, and initializing the positioning system based on the IMU information and GNSS information; A map acquisition module for obtaining high-precision map data for the corresponding local range based on the current position information; An element verification module for verifying whether the current single-frame real-time feature element is valid and saving the corresponding quality inspection result; A first matching and fusion module for performing a first matching and fusion based on the single-frame real-time feature element information, IMU information, GNSS information, VCU information, and high-precision map data to obtain the lateral and longitudinal correction information, attitude information, and matching confidence of the current position; A three-dimensional reconstruction module for obtaining the local scene element information of the corresponding position through three-dimensional reconstruction based on the position information and attitude information corrected after the first matching and fusion; A second matching and fusion module for performing a second matching and fusion based on the local scene element information, the position information corrected by the first matching and fusion, and the high-precision map information if the local scene elements meet the predetermined matching requirements; Anomaly correction module, configured to correct the abnormal matching results according to the horizontal and vertical correction information, matching confidence, and single-frame real-time feature quality inspection results output by the primary matching fusion and the secondary matching fusion; Fusion positioning module, configured to fuse the corrected matching results with the IMU information, GNSS information, and VCU information to obtain the final speed, position, and attitude information.

5. The system according to claim 4, characterized in that, Verifying whether the current single-frame real-time feature elements are valid and saving the corresponding quality inspection results includes: Comprehensively judging whether the current single-frame real-time feature elements are valid according to the change trend of historical single-frame feature element information and the change trend of vehicle movement.

6. The system according to claim 4, characterized in that, The secondary matching fusion module includes: Comparison and judgment unit, configured to judge whether the local scene elements meet the predetermined matching requirements by comparing the local camera perception results with the high-precision map information.

7. An electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the steps of a multi-source matching positioning method according to any one of claims 1 to 3.

8. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed, it implements the steps of a multi-source matching positioning method according to any one of claims 1 to 3.

Images