High-precision map updating method and device
By integrating sensors such as inertial measurement units and wheel speedometers into vehicles, and combining them with camera and LiDAR data for high-precision map updates, the problems of high cost and poor quality in high-precision map updates have been solved, achieving low-cost, high-precision real-time map updates.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- ECARX (HUBEI) TECHCO LTD
- Filing Date
- 2023-01-16
- Publication Date
- 2026-07-07
AI Technical Summary
Existing technologies for updating high-precision maps suffer from high costs or poor map quality, making frequent and timely updates impossible.
By acquiring vehicle pose information and multiple frames of road images, dead reckoning is performed using sensors such as inertial measurement units and wheel speedometers. Feature extraction and fusion are then performed by combining camera and lidar data. Road vector feature information is registered with observation feature information, and once the conditions are met, the updated information is uploaded to the cloud for map updates.
It achieves low-cost, high-precision map updates, ensuring the real-time nature and accuracy of map information, and improves the reliability of updates by using crowdsourcing.
Smart Images
Figure CN116045964B_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of map updating technology, and more specifically, to a high-precision map updating method and apparatus. Background Technology
[0002] High-definition maps (HD maps), also known as high-precision electronic maps, differ from traditional navigation maps. In addition to providing road-level navigation information, they can also provide lane-level navigation information. In terms of both the richness and accuracy of information, HD maps are far superior to traditional navigation maps, and are therefore widely used in scenarios such as vehicle navigation and autonomous driving.
[0003] Because actual roads can change due to construction and rerouting, high-precision maps need to be constantly updated and maintained to maintain their real-time nature, or what the industry calls "freshness." There are two main approaches: one is to use specialized data collection vehicles equipped with high-precision equipment to periodically collect data for map updates. The advantage of this approach is high-quality updates, but the disadvantage is that it is expensive and cannot be updated frequently and promptly. The other approach is to equip ordinary vehicles with ordinary sensors and update the map through crowdsourcing. The advantages of this approach are low cost and timely updates, but the disadvantages are lower update quality and higher technical difficulty.
[0004] There is currently no effective solution to the above problems. Summary of the Invention
[0005] This application provides a high-precision map updating method and apparatus to at least solve the technical problems of high cost or poor map quality when updating high-precision maps in related technologies.
[0006] According to one aspect of the embodiments of this application, a high-precision map updating method is provided, comprising: acquiring the first pose information of a vehicle at a first acquisition time, and acquiring a local high-precision map corresponding to the first pose information from a cloud module, wherein the local high-precision map includes road vector feature information; acquiring multiple frames of road images and vehicle dead reckoning information acquired within a preset time period, and determining fused observation feature information based on the multiple frames of road images and vehicle dead reckoning information, wherein the vehicle dead reckoning information is used for dead reckoning; registering the road vector feature information and the fused observation feature information to obtain the registered pose information of the vehicle; and uploading the fused observation feature information to the cloud module when the registered pose information meets preset conditions, so as to update the local high-precision map.
[0007] Optionally, obtaining the first pose information of the vehicle at the first acquisition time includes: when the first acquisition time is the initial acquisition time, obtaining the first pose information of the vehicle from the Global Navigation Satellite System; or, obtaining the first pose information input by the target object in the human-machine interface of the vehicle; when the first acquisition time is not the initial acquisition time, obtaining the first deadlock information of the vehicle at the first acquisition time, and obtaining the second pose information and the second deadlock information of the vehicle at the second acquisition time, and determining the first pose information of the vehicle at the first acquisition time based on the second pose information, the first deadlock information and the second deadlock information; wherein, the second acquisition time is the acquisition time preceding the first acquisition time.
[0008] Optionally, acquiring multiple frames of road images and vehicle dead center information collected within a preset time period includes: determining each third acquisition time within a preset time period prior to the first acquisition time; acquiring road images collected by a first type of sensor at each third acquisition time, wherein the first type of sensor includes one of the following: a camera, a lidar; acquiring vehicle dead center information collected by a second type of sensor at each third acquisition time, wherein the second type of sensor includes one of the following: an inertial measurement unit, a wheel speedometer, a vehicle speedometer.
[0009] Optionally, the fused observation feature information is determined based on multiple frames of road images and vehicle dead reckoning information, including: for each frame of road image, extracting first observation feature information from the road image, and converting the first observation feature information in the sensor coordinate system into second observation feature information in the vehicle coordinate system based on the extrinsic parameters of the first type of sensor; determining the relative pose between the vehicle dead reckoning information collected at the third acquisition time corresponding to the road image and the vehicle dead reckoning information collected at the first acquisition time, and converting the second observation feature information into third observation feature information based on the relative pose; and fusing the third observation feature information corresponding to each frame of road image to obtain fused observation feature information.
[0010] Optionally, the road vector feature information and the fused observation feature information are registered to obtain the vehicle's registration pose information, including: converting the fused observation feature information in the vehicle coordinate system into the fourth observation feature information in the world coordinate system based on the first pose information; establishing a cost function based on the first projection error between the road vector feature information and the fourth observation feature information; and determining the vehicle's registration pose information by minimizing the cost function.
[0011] Optionally, when the registration pose information meets preset conditions, the fused observation feature information is sent to the cloud module, including: converting the fused observation feature information in the vehicle coordinate system into the fifth observation feature information in the world coordinate system based on the registration pose information; determining the second projection error between the road vector feature information and the fifth observation feature information, and determining the confidence level of the registration pose information based on the second projection error; and sending the fused observation feature information to the cloud module when the confidence level is greater than a preset confidence threshold and the second projection error is greater than a preset error threshold.
[0012] Optionally, the fused observation feature information is uploaded to the cloud module to update the local high-precision map, including: uploading the fused observation feature information to the cloud module, wherein the cloud module is used to manage multiple sets of received fused observation feature information, count the frequency of occurrence of the same fused observation feature information, determine the fused observation feature information with the highest frequency of occurrence as the target fused observation feature information, and update the local high-precision map based on the target fused observation feature information.
[0013] Optionally, after obtaining the vehicle's registration pose information, the method further includes: determining the vehicle's target positioning information at the first acquisition time based on the registration pose information; and displaying a local high-precision map and target positioning information in the vehicle's human-machine interface.
[0014] According to another aspect of the embodiments of this application, a high-precision map updating device is also provided, comprising: a first acquisition module, configured to acquire the first pose information of a vehicle at a first acquisition time, and acquire a local high-precision map corresponding to the first pose information from a cloud module, wherein the local high-precision map includes road vector feature information; a second acquisition module, configured to acquire multiple frames of road images and vehicle dead reckoning information acquired within a preset time period, and determine fused observation feature information based on the multiple frames of road images and vehicle dead reckoning information, wherein the vehicle dead reckoning information is used for dead reckoning; a registration module, configured to register the road vector feature information and the fused observation feature information to obtain the registered pose information of the vehicle; and an update module, configured to upload the fused observation feature information to the cloud module to update the local high-precision map when the registered pose information meets preset conditions.
[0015] According to another aspect of the embodiments of this application, an in-vehicle device is also provided, the in-vehicle device including: a memory and a processor, wherein the memory stores a computer program, and the processor is configured to execute the above-described high-precision map update method through the computer program.
[0016] In this embodiment, the vehicle's first pose information at the first acquisition time is first obtained, and a local high-precision map corresponding to the first pose information is obtained from the cloud module. This local high-precision map includes road vector feature information. Then, multiple frames of road images and vehicle dead reckoning information collected within a preset time period are acquired, and fused observation feature information is determined based on the multiple frames of road images and vehicle dead reckoning information. Among them, vehicle dead reckoning information is used for dead reckoning. Next, the road vector feature information and fused observation feature information are registered to obtain the vehicle's registered pose information. Finally, when the registered pose information meets preset conditions, the fused observation feature information is uploaded to the cloud module to update the local high-precision map. This process involves extracting features from road images and vehicle dead reckoning information to obtain fused observation feature information. This information is then registered with road vector feature information in a high-precision map. This process can obtain real-time, accurate registration pose information for vehicles without the need for expensive acquisition equipment. When this registration pose information meets preset conditions, the accuracy of the map update information is guaranteed. At this point, the map update information is uploaded to the cloud, where it can then perform crowdsourced updates to the high-precision map based on multiple sets of map update information. This application effectively solves the technical problems of high cost or poor map quality when updating high-precision maps in related technologies. Attached Figure Description
[0017] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments of this application and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings:
[0018] Figure 1 This is a flowchart illustrating an optional high-precision map update method according to an embodiment of this application;
[0019] Figure 2 This is a schematic diagram of an optional high-precision map and vehicle positioning according to an embodiment of this application;
[0020] Figure 3 This is a schematic diagram of an optional process for determining the registration pose according to an embodiment of this application;
[0021] Figure 4 This is a schematic diagram illustrating an optional vehicle-cloud interaction according to an embodiment of this application;
[0022] Figure 5 This is a schematic diagram of an optional high-precision map updating device according to an embodiment of this application. Detailed Implementation
[0023] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, and not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative effort should fall within the scope of protection of the present application.
[0024] It should be noted that the terms "first," "second," etc., used in the specification, claims, and drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.
[0025] To better understand the embodiments of this application, the following is a translation and explanation of some nouns or terms that appear in the description of the embodiments of this application:
[0026] Localization technology is one of the fundamental and core technologies for robotic applications such as autonomous driving, providing robots with position and orientation information. Based on their principles, localization technologies can be categorized into geometric localization, dead reckoning (DR), and feature-based localization.
[0027] Geometric positioning involves measuring distance or angle using a reference device at a known location, and then determining one's own position through geometric calculations. This includes technologies such as GNSS (Global Navigation Satellite System), UWB (Ultra Wide Band), Bluetooth, and 5G, providing absolute positioning information. In intelligent vehicle applications, GNSS technology is the most widely used. GNSS positioning, based on satellite positioning technology, is divided into point positioning, differential GPS positioning, and RTK (Real-Time Kinematic) GPS positioning. Point positioning provides a positioning accuracy of 3–10 meters, differential GPS provides 0.5–2 meters, and RTK GPS provides centimeter-level accuracy. Its limitations include reliance on positioning facilities, susceptibility to signal blockage and reflection, and failure in environments such as tunnels and overpasses.
[0028] Dead reckoning calculates the position at the next moment from the previous position, based on motion data from sensors such as IMUs (Inertial Measurement Units) and wheel speedometers, providing relative positioning information. For example, when calculating the relative pose from point a to point b, let the pose of point a be T. a The pose of point b is T. b Then the relative pose T between a and b is determined. ba =T a-inverse *T b T here a-inverse It refers to T a The inverse matrix. A limitation of dead reckoning is that the positioning error accumulates and increases with the reckoning distance.
[0029] Feature-based localization first acquires several features of the surrounding environment, such as base station IDs, Wi-Fi fingerprints, images, and LiDAR point clouds. Then, it matches the observed features with a pre-established feature map to determine the location within the map, providing absolute positioning information. The direct factors affecting feature-based localization are the quantity, quality, and discriminative power of the features. Its limitation is that positioning accuracy and stability decrease when scene and environmental factors affect feature observation.
[0030] Coordinate systems: In positioning technology, the world coordinate system, the carrier coordinate system, and the sensor coordinate system are usually involved.
[0031] The world coordinate system is denoted as W, and it maintains a fixed relationship with the actual geographical location. The Earth-Centered, Earth-Fixed coordinate system (ECEF) is usually used.
[0032] The carrier coordinate system is denoted as B, centered at a fixed position on the carrier. For the vehicle, this is the vehicle coordinate system, such as centered at the rear axle. The vehicle pose is the 6Dof (Degree of Freedom) pose of the vehicle coordinate system in the world coordinate system, denoted as T. WB .
[0033] The sensor coordinate system is denoted by S, also known as the observation coordinate system. All measurement data acquired by the sensor is based on the sensor coordinate system. Since the sensor is usually fixed to a carrier and undergoes rigid body motion with the carrier, there is a fixed transformation relationship T between the sensor coordinate system and the carrier coordinate system. BS Also known as sensor extrinsic parameters.
[0034] Example 1
[0035] According to an embodiment of this application, a high-precision map update method applicable to vehicle-mounted devices is provided. It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions. Furthermore, although a logical order is shown in the flowchart, in some cases, the steps shown or described may be executed in a different order than that shown here.
[0036] Figure 1 This is a flowchart illustrating an optional high-precision map update method according to an embodiment of this application, such as... Figure 1 As shown, the method includes at least steps S102-S108, wherein:
[0037] Step S102: Obtain the first pose information of the vehicle at the first acquisition time, and obtain the local high-precision map corresponding to the first pose information from the cloud module. The local high-precision map includes road vector feature information.
[0038] Since the embodiments of this application primarily rely on the real-time positioning of the vehicle for map updates, the aforementioned first acquisition time mainly refers to the latest acquisition time. As an optional implementation, the first pose information of the vehicle at the first acquisition time can be obtained in the following way:
[0039] When the first acquisition time is the initial acquisition time, the vehicle's first attitude information can be obtained from the Global Navigation Satellite System, denoted as T. WB1 However, since global satellite navigation systems are easily affected by signal blockage and reflection, they may fail in scenarios such as tunnels and overpasses. Therefore, it is also possible to obtain the first pose information of the target object input in the vehicle's human-machine interface, that is, to determine the current position and pose information of the vehicle through user assistance. The specific input method can be text input or voice input, or it can be the specification of a specific location on the human-machine interface.
[0040] If the first acquisition time is not the initial acquisition time, the first deadlock information of the vehicle at the first acquisition time can be obtained, and the second pose information and second deadlock information of the vehicle at the second acquisition time can be obtained. Based on the second pose information, the first deadlock information and the second deadlock information, the first pose information of the vehicle at the first acquisition time can be determined; wherein, the second acquisition time is the acquisition time preceding the first acquisition time.
[0041] For example, the second pose information obtained is T WB2 Based on the first and second dead reckoning information, the relative pose of the vehicle at the first and second data acquisition times is calculated as T. B1B2 Then the first pose information is determined to be T. WB1 =T WB2 *T B1B2 .
[0042] After obtaining the first pose information, a local high-precision map corresponding to the first pose information can be retrieved from the cloud module. The cloud module stores a complete high-precision map, which stores road information in vector format, including but not limited to storing road surface object information such as lampposts, road signs, and curbs as point, line, and area vectors, as well as road marking information such as solid lines, dashed lines, arrows, and text. Figure 2 This is a schematic diagram of a typical high-precision map.
[0043] Specifically, during vehicle operation, the on-board equipment can send a request to the cloud module to obtain a high-precision map. This request includes the vehicle's current first pose information. After receiving the request, the cloud module can send a local high-precision map corresponding to the first pose information to the on-board equipment, such as a local high-precision map within a 1-kilometer radius of the vehicle's current location.
[0044] Step S104: Obtain multiple frames of road images and vehicle dead reckoning information collected within a preset time period, and determine the fused observation feature information based on the multiple frames of road images and vehicle dead reckoning information, wherein the vehicle dead reckoning information is used for dead reckoning.
[0045] Considering that single-frame road images may not provide comprehensive feature information due to occlusion, limited sensor detection distance, etc., this application proposes to stitch and fuse the observation features of multiple road images for subsequent pose calibration and map updating.
[0046] The preset time period is a preset duration of time preceding (and including) the first acquisition moment. This preset duration can be set manually and is not specifically limited here. The preset time period includes multiple acquisition moments at fixed time intervals, which are related to the acquisition frequency of the acquisition device.
[0047] As an optional implementation, multiple frames of road images and vehicle dead center information collected within a preset time period can be obtained first by: determining each third acquisition time within a preset time period prior to the first acquisition time; acquiring road images collected by a first type of sensor at each third acquisition time, wherein the first type of sensor includes one of the following: a camera, a lidar; acquiring vehicle dead center information collected by a second type of sensor at each third acquisition time, wherein the second type of sensor includes one of the following: an inertial measurement unit, a wheel speedometer, a vehicle speedometer.
[0048] Subsequently, for each frame of road image, the first observation feature information in the road image can be extracted through methods such as detection, segmentation, and recognition. Then, based on the extrinsic parameters of the first type of sensor, the first observation feature information in the sensor coordinate system is converted into second observation feature information in the vehicle coordinate system. For example, the first observation feature information in the sensor coordinate system is P. S The extrinsic parameter of the first type of sensor is T. BS Then, the second observation feature information in the vehicle coordinate system can be obtained as P. B =P S *T BS .
[0049] To stitch together the observation feature information of multiple road images, it is necessary to ensure that they are all in the same coordinate system. Therefore, for each road image, the relative pose between the vehicle dead reckoning information collected at the third acquisition time and the vehicle dead reckoning information collected at the first acquisition time can be determined, and the second observation feature information can be converted into the third observation feature information based on the relative pose. Finally, the third observation feature information corresponding to each road image is fused to obtain the fused observation feature information.
[0050] For example, the second observation feature information corresponding to each third acquisition time is {P1, P2, ..., P...} n The corresponding vehicle positioning information is {T1, T2, ..., T}. n}, where n represents the latest first acquisition time, and for the i-th third acquisition time, the relative pose of the vehicle at that acquisition time and the first acquisition time is T. ni =T i-inverse *T n Then the third observation feature information at the third acquisition time is nP. i =T ni *P i The final fused observation feature information is
[0051] Step S106: Register the road vector feature information and the fused observation feature information to obtain the vehicle's registered pose information.
[0052] The registration process involves finding an optimal pose T. WB-best This is the process of minimizing the distance between the semantic observations corresponding to the road vector feature information and the fused observation feature information.
[0053] As an optional implementation, the road vector feature information and the fused observation feature information can be registered in the following way: the first pose information is used as the initial value for optimization, and the fused observation feature information in the vehicle coordinate system is converted into the fourth observation feature information in the world coordinate system based on the first pose information; then, a cost function is established based on the first projection error between the road vector feature information and the fourth observation feature information; and the vehicle's registration pose information is determined by minimizing the cost function.
[0054] For example, suppose the road elements in the road vector feature information are {M1, M2, ..., M}. n}, where each observation element in the fused observation feature information is {F1, F2, ..., F}. n}, then the first pose information T WB1 As initial values for optimization, construct the cost function:
[0055] f(T WB =SUM(DIST(T) WB *F i M i ))
[0056] In the formula, T WB *F i This is the fourth observation feature information, DIST(T) WB *F i M i ) represents the observed element F i With road element M i The first projection error between them, SUM(DIST(T) WB *F i M i )) represents the summation of the first projection error over each element.
[0057] The optimization process can be represented as:
[0058] T WB-best =argmin(f(T) WB ))
[0059] In the formula, argmin(f(T) WB )) indicates finding the optimal T WB-best Make the cost function f(T) WB The value of ) is the smallest.
[0060] Through the above process, the initial low-precision first pose information T can be obtained. WB1 The optimized, high-precision registration pose information T is obtained. WB-best .
[0061] Figure 3A schematic diagram of an optional process for determining registration pose information is shown. When the current time is the initial acquisition moment, the first pose information is obtained through the Global Navigation Satellite System or user input. When the current time is not the initial acquisition moment, the first pose information is obtained by dead reckoning based on the pose information from the previous acquisition moment, and the corresponding local high-precision map is acquired. Using the first pose information as the initial value, feature registration is performed on road vector feature information and fused observation feature information to obtain the vehicle's registration pose information.
[0062] As an optional implementation, after obtaining the vehicle's registration pose information, the target positioning information of the vehicle at the first acquisition time can be determined based on the registration pose information, and a local high-precision map and target positioning information can be displayed in the vehicle's human-machine interface, such as... Figure 2 As shown, the pentagram represents the vehicle's current location.
[0063] Step S108: When the registration pose information meets the preset conditions, the fused observation feature information is uploaded to the cloud module to update the local high-precision map.
[0064] Specifically, the fused observation feature information in the vehicle coordinate system can be converted into the fifth observation feature information in the world coordinate system based on the registration pose information. Then, the second projection error between the road vector feature information and the fifth observation feature information is determined, and the confidence level of the registration pose information is determined based on the second projection error. When the confidence level is greater than the preset confidence level threshold and the second projection error is greater than the preset error threshold, the fused observation feature information is sent to the cloud module.
[0065] An optional formula for calculating confidence level is as follows:
[0066] Conf = SUM(DIST(T) WB-best *F i M i ))
[0067] In the formula, Conf represents the confidence level, and T WB-best *F i For the fifth observation feature information, DIST(T) WB-best *F i M i SUM(DIST(T)) represents the second projection error. WB-best *F i M i )) represents the summation of the second projection error over each element.
[0068] To ensure the accuracy of map updates, fused observation feature information is only uploaded to the cloud module when the registration pose information meets preset conditions. Specifically, when the confidence level Conf is greater than the preset confidence threshold CT, it indicates that the registration pose information is reliable and the current vehicle positioning is accurate. When the second projection error DIST(T) is greater than the preset confidence threshold CT, it indicates that the registration pose information is reliable and the current vehicle positioning is accurate. WB-best *F i M i If the error value is greater than the preset error threshold DT, it indicates that map element M... i A change has occurred: the high-precision map needs to be updated. At this point, F needs to be... i As a new map element, it is updated to the high-precision map and designated MF. i .
[0069] Considering that map updates based on single-vehicle, single-trip observations may be unreliable due to factors such as occlusion, this application adopts a crowdsourcing approach for updating high-precision maps in the cloud.
[0070] As an optional implementation, the cloud module manages the received multiple sets of fused observation feature information, counts the frequency of occurrence of the same fused observation feature information, determines the fused observation feature information with the highest frequency as the target fused observation feature information, and updates the local high-precision map based on the target fused observation feature information.
[0071] Specifically, the cloud module organizes map update data from multiple vehicles uploading the same local high-precision map within a certain time interval, and denotes the set of map update data from multiple trips as SET{MF ij}, where i represents the map element index, j represents the map update data of the j-th pass, and the final map update data determined by the cloud module is:
[0072] MF i-max =MAX{SET{MF ij}}
[0073] In the formula, MAX{SET{MF ij}} indicates that multiple values are taken from the map update data, and this is done with respect to the index j.
[0074] Figure 4This diagram illustrates a complete interaction between the vehicle side and the cloud side to update a high-precision map. The process involves several steps: When the current time is the initial data acquisition moment, the first pose information is obtained through the Global Navigation Satellite System (GNSS) or user input. When the current time is not the initial data acquisition moment, the first pose information is obtained through dead reckoning based on data collected by the first and second type sensors. The onboard device sends a request to the cloud module to obtain a high-precision map, which includes the vehicle's current first pose information. Upon receiving the request, the cloud module sends a local high-precision map corresponding to the first pose information to the onboard device, including road vector feature information. Feature extraction, dead reckoning, and feature fusion are performed based on data collected by the first and second type sensors to obtain fused observation feature information. The road vector feature information and the fused observation feature information are registered to obtain the vehicle's registered pose information, which can be used for the vehicle's Location Based Services (LBS). When the registered pose information meets preset conditions, the fused observation feature information is uploaded to the cloud module as map update data. The cloud module statistically fuses the received map update data from multiple trips to determine the target map update data for crowdsourced updates of the high-precision map.
[0075] In this embodiment, the vehicle's first pose information at the first acquisition time is first obtained, and a local high-precision map corresponding to the first pose information is obtained from the cloud module. This local high-precision map includes road vector feature information. Then, multiple frames of road images and vehicle dead reckoning information collected within a preset time period are acquired, and fused observation feature information is determined based on the multiple frames of road images and vehicle dead reckoning information. Among them, vehicle dead reckoning information is used for dead reckoning. Next, the road vector feature information and fused observation feature information are registered to obtain the vehicle's registered pose information. Finally, when the registered pose information meets preset conditions, the fused observation feature information is uploaded to the cloud module to update the local high-precision map. This process involves extracting features from road images and vehicle dead reckoning information to obtain fused observation feature information. This information is then registered with road vector feature information in a high-precision map. This process can obtain real-time, accurate registration pose information for vehicles without the need for expensive acquisition equipment. When this registration pose information meets preset conditions, the accuracy of the map update information is guaranteed. At this point, the map update information is uploaded to the cloud, where it can then perform crowdsourced updates to the high-precision map based on multiple sets of map update information. This application effectively solves the technical problems of high cost or poor map quality when updating high-precision maps in related technologies.
[0076] Example 2
[0077] According to an embodiment of this application, a high-precision map updating apparatus for implementing the high-precision map updating method in Embodiment 1 is also provided, such as... Figure 5As shown, the high-precision map updating device includes at least a first acquisition module 51, a second acquisition module 52, a registration module 53, and an update module 54, wherein:
[0078] The first acquisition module 51 is used to acquire the first pose information of the vehicle at the first acquisition time, and to acquire the local high-precision map corresponding to the first pose information from the cloud module. The local high-precision map includes road vector feature information.
[0079] Since the embodiments of this application mainly rely on the real-time positioning of the vehicle for map updates, the aforementioned first acquisition time mainly refers to the latest acquisition time. As an optional implementation, the first acquisition module can obtain the first pose information of the vehicle at the first acquisition time in the following way:
[0080] When the first acquisition time is the initial acquisition time, the first acquisition module can obtain the vehicle's first position information from the global satellite navigation system. However, since the global satellite navigation system is easily affected by signal blockage and reflection, it will fail in scenarios such as tunnels and overpasses. Therefore, it can also obtain the first position information of the target object input in the vehicle's human-machine interface. That is, the current position information of the vehicle is determined through user assistance. The specific input method can be text input or voice input, or it can be the specification of a specific position on the human-machine interface.
[0081] When the first acquisition time is not the initial acquisition time, the first acquisition module can acquire the first dead center information of the vehicle at the first acquisition time, and acquire the second pose information and second dead center information of the vehicle at the second acquisition time. Based on the second pose information, the first dead center information and the second dead center information, the first pose information of the vehicle at the first acquisition time is determined; wherein, the second acquisition time is the acquisition time preceding the first acquisition time.
[0082] After obtaining the first pose information, the first acquisition module can retrieve the local high-precision map corresponding to the first pose information from the cloud module. The cloud module stores a complete high-precision map, which stores road information in the form of vector information, including but not limited to storing information on road objects such as lampposts, road signs, and curbs in vector information such as points, lines, and areas, as well as road marking information such as solid lines, dashed lines, arrows, and text.
[0083] Specifically, during vehicle operation, the first acquisition module can send a request to the cloud module to obtain a high-precision map. This request includes the vehicle's current first pose information. After receiving the request, the cloud module can send a local high-precision map corresponding to the first pose information to the first acquisition module.
[0084] The second acquisition module 52 is used to acquire multiple frames of road images and vehicle dead reckoning information collected within a preset time period, and to determine fused observation feature information based on the multiple frames of road images and vehicle dead reckoning information, wherein the vehicle dead reckoning information is used for dead reckoning.
[0085] Considering that single-frame road images may not provide comprehensive feature information due to occlusion, limited sensor detection distance, etc., this application proposes to stitch and fuse the observation features of multiple road images for subsequent pose calibration and map updating.
[0086] The preset time period is a preset duration of time preceding (and including) the first acquisition moment. This preset duration can be set manually and is not specifically limited here. The preset time period includes multiple acquisition moments at fixed time intervals, which are related to the acquisition frequency of the acquisition device.
[0087] As an optional implementation, the second acquisition module may first acquire multiple frames of road images and vehicle dead center information collected within a preset time period by: determining each third acquisition time within a preset time period prior to the first acquisition time; acquiring road images collected by a first type of sensor at each third acquisition time, wherein the first type of sensor includes one of the following: a camera, a lidar; acquiring vehicle dead center information collected by a second type of sensor at each third acquisition time, wherein the second type of sensor includes one of the following: an inertial measurement unit, a wheel speedometer, a vehicle speedometer.
[0088] Subsequently, for each frame of road image, the second acquisition module can extract the first observation feature information in the road image through detection, segmentation, recognition and other methods, and convert the first observation feature information in the sensor coordinate system into the second observation feature information in the vehicle coordinate system based on the extrinsic parameters of the first type of sensor.
[0089] To stitch together the observation feature information of multiple road images, it is necessary to ensure that they are all in the same coordinate system. Therefore, for each road image, the second acquisition module can determine the relative pose between the vehicle dead reckoning information collected at the third acquisition time and the vehicle dead reckoning information collected at the first acquisition time, and convert the second observation feature information into the third observation feature information based on the relative pose. Finally, the third observation feature information corresponding to each road image is fused to obtain the fused observation feature information.
[0090] The registration module 53 is used to register road vector feature information and fused observation feature information to obtain the vehicle's registration pose information.
[0091] The registration process involves finding an optimal pose that minimizes the distance between the semantic observations corresponding to the road vector feature information and the fused observation feature information.
[0092] As an optional implementation, the registration module can register road vector feature information and fused observation feature information in the following way: using the first pose information as the initial value for optimization, the fused observation feature information in the vehicle coordinate system is converted into the fourth observation feature information in the world coordinate system based on the first pose information; then, a cost function is established based on the first projection error between the road vector feature information and the fourth observation feature information; and the vehicle's registration pose information is determined by minimizing the cost function.
[0093] As an optional implementation, the high-precision map update device in this application embodiment further includes a real-time positioning module, which is used to determine the target positioning information of the vehicle at the first acquisition time based on the registration pose information after obtaining the vehicle's registration pose information, and to display the local high-precision map and target positioning information in the vehicle's human-machine interface.
[0094] The update module 54 is used to upload the fused observation feature information to the cloud module when the registration pose information meets the preset conditions, so as to update the local high-precision map.
[0095] Specifically, the update module can first convert the fused observation feature information in the vehicle coordinate system into the fifth observation feature information in the world coordinate system based on the registration pose information; then determine the second projection error between the road vector feature information and the fifth observation feature information, and determine the confidence level of the registration pose information based on the second projection error; when the confidence level is greater than the preset confidence threshold and the second projection error is greater than the preset error threshold, the fused observation feature information is sent to the cloud module.
[0096] Considering that map updates based on single-vehicle, single-trip observations may be unreliable due to factors such as occlusion, this application adopts a crowdsourcing approach for updating high-precision maps in the cloud.
[0097] As an optional implementation, the cloud module manages the received multiple sets of fused observation feature information, counts the frequency of occurrence of the same fused observation feature information, determines the fused observation feature information with the highest frequency as the target fused observation feature information, and updates the local high-precision map based on the target fused observation feature information.
[0098] It should be noted that each module in the high-precision map update device in this application embodiment corresponds one-to-one with each implementation step of the high-precision map update method in embodiment 1. Since embodiment 1 has been described in detail, some details not shown in this embodiment can be referred to embodiment 1, and will not be elaborated further here.
[0099] Example 3
[0100] According to an embodiment of this application, a non-volatile storage medium is also provided, which includes a stored program, wherein the device where the non-volatile storage medium is located executes the high-precision map update method in Embodiment 1 by running the program.
[0101] Specifically, the device containing the non-volatile storage medium executes the following steps by running the program: acquiring the first pose information of the vehicle at the first acquisition time, and acquiring a local high-precision map corresponding to the first pose information from the cloud module, wherein the local high-precision map includes road vector feature information; acquiring multiple frames of road images and vehicle dead reckoning information acquired within a preset time period, and determining fused observation feature information based on the multiple frames of road images and vehicle dead reckoning information, wherein the vehicle dead reckoning information is used for dead reckoning; registering the road vector feature information and the fused observation feature information to obtain the vehicle's registered pose information; when the registered pose information meets preset conditions, uploading the fused observation feature information to the cloud module to update the local high-precision map.
[0102] According to an embodiment of this application, a processor is also provided for running a program, wherein the program executes the high-precision map update method in embodiment 1 during runtime.
[0103] Specifically, the program executes the following steps during runtime: acquiring the vehicle's first pose information at the first acquisition time, and obtaining a local high-precision map corresponding to the first pose information from the cloud module, wherein the local high-precision map includes road vector feature information; acquiring multiple frames of road images and vehicle dead reckoning information acquired within a preset time period, and determining fused observation feature information based on the multiple frames of road images and vehicle dead reckoning information, wherein the vehicle dead reckoning information is used for dead reckoning; registering the road vector feature information and the fused observation feature information to obtain the vehicle's registered pose information; when the registered pose information meets preset conditions, uploading the fused observation feature information to the cloud module to update the local high-precision map.
[0104] According to an embodiment of this application, an in-vehicle device is also provided, the in-vehicle device including: a memory and a processor, wherein the memory stores a computer program, and the processor is configured to execute the high-precision map update method of embodiment 1 through the computer program.
[0105] Specifically, the processor is configured to execute the following steps via a computer program: acquire the first pose information of the vehicle at the first acquisition time, and acquire a local high-precision map corresponding to the first pose information from the cloud module, wherein the local high-precision map includes road vector feature information; acquire multiple frames of road images and vehicle dead reckoning information acquired within a preset time period, and determine fused observation feature information based on the multiple frames of road images and vehicle dead reckoning information, wherein the vehicle dead reckoning information is used for dead reckoning; register the road vector feature information and the fused observation feature information to obtain the vehicle's registered pose information; when the registered pose information meets preset conditions, upload the fused observation feature information to the cloud module to update the local high-precision map.
[0106] The sequence numbers of the embodiments in this application are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.
[0107] In the above embodiments of this application, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments.
[0108] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. The device embodiments described above are merely illustrative; for example, the division of units can be a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the displayed or discussed mutual couplings, direct couplings, or communication connections may be through some interfaces; indirect couplings or communication connections between units or modules may be electrical or other forms.
[0109] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.
[0110] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.
[0111] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as a USB flash drive, read-only memory (ROM), random access memory (RAM), portable hard drive, magnetic disk, or optical disk.
[0112] The above are merely preferred embodiments of this application. It should be noted that those skilled in the art can make various improvements and modifications without departing from the principles of this application, and these improvements and modifications should also be considered within the scope of protection of this application.
Claims
1. A high-precision map update method, characterized in that, include: The first pose information of the vehicle at the first acquisition time is obtained, and a local high-precision map corresponding to the first pose information is obtained from the cloud module, wherein the local high-precision map includes road vector feature information; Acquire multiple frames of road images and vehicle dead reckoning information collected within a preset time period, and determine fused observation feature information based on the multiple frames of road images and the vehicle dead reckoning information, wherein the vehicle dead reckoning information is used for dead reckoning. The registration pose information of the vehicle is obtained by registering the road vector feature information and the fused observation feature information, including: converting the fused observation feature information in the vehicle coordinate system into a fourth observation feature information in the world coordinate system based on the first pose information; establishing a cost function based on the first super-projection error between the road vector feature information and the fourth observation feature information; and determining the registration pose information of the vehicle by minimizing the cost function. When the registration pose information meets the preset conditions, the fused observation feature information is uploaded to the cloud module to update the local high-precision map.
2. The method according to claim 1, characterized in that, Obtain the first pose information of the vehicle at the first data acquisition moment, including: When the first acquisition time is the initial acquisition time, the first pose information of the vehicle is obtained from the global satellite navigation system; or, the first pose information input by the target object in the human-machine interface of the vehicle is obtained. When the first acquisition time is not the initial acquisition time, the first deadlock information of the vehicle at the first acquisition time is obtained, and the second pose information and second deadlock information of the vehicle at the second acquisition time are obtained. The first pose information of the vehicle at the first acquisition time is determined based on the second pose information, the first deadlock information and the second deadlock information; wherein, the second acquisition time is the previous acquisition time of the first acquisition time.
3. The method according to claim 1, characterized in that, Acquire multiple frames of road images and vehicle dead center information collected within a preset time period, including: Determine each third acquisition time within the preset time period prior to the first acquisition time; The road images acquired by a first type of sensor at each of the third acquisition times are obtained, wherein the first type of sensor includes one of the following: a camera, a lidar; The vehicle dead reckoning information collected by the second type of sensor at each of the third acquisition times is acquired, wherein the second type of sensor includes one of the following: an inertial measurement unit, a wheel speedometer, and a vehicle speedometer.
4. The method according to claim 3, characterized in that, Based on the multi-frame road images and the vehicle dead reckoning information, the fused observation feature information is determined, including: For each frame of the road image, the first observation feature information in the road image is extracted, and the first observation feature information in the sensor coordinate system is converted into the second observation feature information in the vehicle coordinate system according to the extrinsic parameters of the first type of sensor. The relative pose between the vehicle dead reckoning information acquired at the third acquisition time corresponding to the road image and the vehicle dead reckoning information acquired at the first acquisition time is determined, and the second observation feature information is converted into the third observation feature information based on the relative pose. The third observation feature information corresponding to each frame of the road image is fused to obtain the fused observation feature information.
5. The method according to claim 1, characterized in that, When the registration pose information meets preset conditions, the fused observation feature information is sent to the cloud module, including: Based on the registration pose information, the fused observation feature information in the vehicle coordinate system is converted into the fifth observation feature information in the world coordinate system; The second projection error between the road vector feature information and the fifth observation feature information is determined, and the confidence level of the registration pose information is determined based on the second projection error. When the confidence level is greater than a preset confidence level threshold and the second projection error is greater than a preset error threshold, the fused observation feature information is sent to the cloud module.
6. The method according to claim 1, characterized in that, Uploading the fused observation feature information to the cloud module to update the local high-precision map includes: The fused observation feature information is uploaded to the cloud module, wherein the cloud module is used to manage multiple sets of the received fused observation feature information, count the frequency of occurrence of the same fused observation feature information, determine the fused observation feature information with the highest frequency of occurrence as the target fused observation feature information, and update the local high-precision map based on the target fused observation feature information.
7. The method according to claim 1, characterized in that, After obtaining the registration pose information of the vehicle, the method further includes: The target positioning information of the vehicle at the first acquisition time is determined based on the registration pose information; The local high-precision map and the target positioning information are displayed in the human-machine interface of the vehicle.
8. A high-precision map updating device, characterized in that, include: The first acquisition module is used to acquire the first pose information of the vehicle at the first acquisition time, and to acquire the local high-precision map corresponding to the first pose information from the cloud module, wherein the local high-precision map includes road vector feature information. The second acquisition module is used to acquire multiple frames of road images and vehicle dead reckoning information collected within a preset time period, and to determine fused observation feature information based on the multiple frames of road images and the vehicle dead reckoning information, wherein the vehicle dead reckoning information is used for dead reckoning. The registration module is used to register the road vector feature information and the fused observation feature information to obtain the registration pose information of the vehicle, including: converting the fused observation feature information in the vehicle coordinate system into a fourth observation feature information in the world coordinate system based on the first pose information; establishing a cost function based on the first super-projection error between the road vector feature information and the fourth observation feature information; and determining the registration pose information of the vehicle by minimizing the cost function. The update module is used to upload the fused observation feature information to the cloud module when the registration pose information meets the preset conditions, so as to update the local high-precision map.
9. A vehicle-mounted device, characterized in that, include: A memory and a processor, wherein the memory stores a computer program, and the processor is configured to execute the high-precision map update method according to any one of claims 1 to 7 through the computer program.