Map trajectory verification method, device, mobile device and storage medium

Abstract

The embodiment of the present invention provides a map trajectory verification method, equipment, mobile device and storage medium. The method includes: collecting map construction data and verification data of driving on the map respectively; using the verification data to perform track calculation to determine the entire verification trajectory composed of multiple track sub-segments; obtaining a first trajectory under satellite positioning coordinates in the multiple track sub-segments; obtaining an estimated second trajectory under the satellite positioning coordinate system; and performing map trajectory verification based on the first trajectory and the second trajectory. The verification trajectory calculation method based on pose graph optimization in the embodiment of the present invention can prevent positioning loss and deflection problems in the verification trajectory, as well as the failure of real-time positioning due to reasons such as discontinuity of the starting point of the verification trajectory and being out of the map. Even if the GPS of the verification trajectory fails or part of the trajectory is outside the map, a smooth trajectory result can be obtained, and it is guaranteed that the trajectory inside the map can be correctly matched.

Description

Technical Field

[0001] The present invention relates to the field of autonomous driving, and in particular to a map trajectory verification method, equipment, mobile device and storage medium. Background Art

[0002] Autonomous driving technology is a key trend in the development of intelligent vehicle technology. It is generally categorized into five levels of autonomous driving intelligence, one to five. In most cases, Level 4 vehicles rely on no human intervention and have achieved significant scale in low-speed applications such as cleaning, sanitation, and logistics.

[0003] Level 4 autonomous vehicles generally require high-precision maps for autonomous navigation, control, and planning. Therefore, before operating a vehicle, it is necessary to pre-map the target scene using onboard sensors. A key issue in the map-building process is how to evaluate map quality. In practical applications, it is rare to obtain the three-dimensional structure of the scene in advance, making it difficult to compare the map with the ground truth to calculate the error. Therefore, more practical engineering methods are needed to evaluate maps.

[0004] To obtain the corresponding map, you can use the following method: while collecting the map, you can also record a verification trajectory. After completing the map construction, use the resulting map as a model to evaluate the positioning performance of the verification trajectory in the map. When positioning failure, height failure, etc. frequently occur in the verification path, it will be considered an error in the map construction. The calculation method of the verification trajectory can directly use the real-time fusion positioning method on the vehicle side, sort the detection data in time series, and use the fusion positioning result as the output of the verification trajectory.

[0005] To verify the quality of a map, you can usually use:

[0006] 1. Use the same vehicle to circle the map multiple times, using the results to cross-validate the map quality. This method ensures that the same road section is captured multiple times, avoiding the uncertainty caused by a single capture.

[0007] 2. Use the same vehicle to collect both mapping data and validation data. Mapping data is used for map construction, while validation data is used only to verify map accuracy. This approach draws on the concept of training and test sets in machine learning to avoid duplication and overfitting. In practice, the validation trajectory does not have to be an exact duplicate of the mapping trajectory; even branches from different road sections can be used. Different methods exist for calculating the validation trajectory, including simulated real-time positioning and offline verification.

[0008] 3. Use different vehicles to conduct map crowdsourcing to verify the consistency of map data. This method is mainly used in traditional electronic navigation maps and can quickly update existing map data.

[0009] 4. Use CAD models and floor plans of the real scene itself to compare map quality.

[0010] During the implementation of the present invention, the inventors discovered that the above technology has at least the following problems:

[0011] 1. While effective, multiple circling solutions can make the data collection process cumbersome. For passenger vehicles, data collection relies primarily on the driver driving the vehicle, so multiple circling is acceptable. However, for low-speed vehicles, data collection relies primarily on personnel following the vehicle. Multiple circling significantly increases vehicle range and data collection time, making this solution unfeasible.

[0012] 2. Separately collecting mapping and verification data is a relatively low-cost solution. This solution is more practical for low-speed autonomous driving. However, in complex routes, it is difficult for data collectors to remember the shapes of all existing trajectories. This can lead to mismatches between verification data and collected data, or out-of-map issues, making it impossible for real-time positioning to calculate the shape of the entire verification trajectory.

[0013] 3. The crowdsourcing method using different vehicles requires a large number of autonomous vehicles. In traditional electronic navigation map production, ordinary vehicles can be used to update the map, but the number of autonomous vehicles currently does not support such work.

[0014] 4. Using scene CAD models and floor plans requires obtaining real scene data in advance, but this data is also difficult to obtain, making this solution basically unfeasible in practice. Summary of the Invention

[0015] In order to at least solve the problem of infeasibility of verifying map quality in the prior art, an embodiment of the present invention provides a map trajectory verification method, comprising:

[0016] respectively collecting map construction data and verification data of driving in the map;

[0017] performing dead reckoning using the verification data to determine an entire verification trajectory comprised of a plurality of trajectory subsegments;

[0018] Among the plurality of track subsegments, for a first track subsegment of a valid satellite positioning signal, using the valid satellite positioning data and the first track subsegment to perform track optimization to obtain a first track under satellite positioning coordinates;

[0019] For the first track subsegment of the invalid satellite positioning signal, using a known first relative position between the end point of collection of the map construction data and the starting point of the verification track, the first track subsegment is transformed into the satellite positioning coordinate system to obtain an estimated second track in the satellite positioning coordinate system;

[0020] Map trajectory verification is performed based on the first trajectory and the second trajectory.

[0021] In a second aspect, an embodiment of the present invention provides a map trajectory verification execution device, characterized by comprising:

[0022] a data collection module for collecting map construction data and verification data of driving in the map;

[0023] a verification trajectory determination module, configured to perform dead reckoning using the verification data to determine an entire verification trajectory consisting of a plurality of trajectory sub-segments;

[0024] a first trajectory determination module configured to, for a first trajectory subsegment of a valid satellite positioning signal among the plurality of trajectory subsegments, perform trajectory optimization using the valid satellite positioning data and the first trajectory subsegment to obtain a first trajectory under satellite positioning coordinates;

[0025] a second trajectory determination module configured to transform, for a first track subsegment with an invalid satellite positioning signal, the first track subsegment into the satellite positioning coordinate system using a known first relative position between an end point of collection of the map construction data and a starting point of the verification trajectory, to obtain an estimated second trajectory in the satellite positioning coordinate system;

[0026] A verification module is used to perform map trajectory verification based on the first trajectory and the second trajectory.

[0027] In a third aspect, an electronic device is provided, comprising: at least one processor, and a memory communicatively connected to the at least one processor, wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can perform the steps of the map trajectory verification method of any embodiment of the present invention.

[0028] In a fourth aspect, an embodiment of the present invention provides a mobile device, comprising a main body and the electronic device according to any embodiment of the present invention mounted on the main body.

[0029] In a fifth aspect, an embodiment of the present invention provides a storage medium on which a computer program is stored, characterized in that when the program is executed by a processor, the steps of the map trajectory verification method of any embodiment of the present invention are implemented.

[0030] In a sixth aspect, an embodiment of the present invention further provides a computer program product, which, when run on a computer, enables the computer to execute the map trajectory verification method described in any one of the embodiments of the present invention.

[0031] The beneficial effects of the embodiments of the present invention are as follows: based on the low-cost map trajectory verification of separately collected mapping and verification data, a method of separately collecting mapping data and verification data to verify the validity of the map is realized, which solves the problem that the verification data and the collected data do not overlap or are out of map. In addition, the verification trajectory calculation method based on pose graph optimization can prevent the real-time algorithm from experiencing positioning loss and deflection when local signals are missing. It also solves the problem of real-time positioning failure in the real-time positioning calculation scheme due to discontinuous starting points of the verification trajectory, out of map, etc. Even if the GPS of the verification trajectory fails or part of the trajectory is outside the map, a smooth trajectory result can be obtained, and it is guaranteed that the trajectory within the map can be correctly matched. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following is a brief introduction to the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0033] Figure 1 This is a flow chart of a map trajectory verification method provided by one embodiment of the present invention;

[0034] Figure 2 This is an overall structural diagram of a map trajectory verification method provided by one embodiment of the present invention;

[0035] Figure 3 Schematic diagram of a pose graph containing a mapping trajectory and a verification trajectory of a map trajectory verification method provided by one embodiment of the present invention;

[0036] Figure 4 This is a schematic structural diagram of a map trajectory verification execution device provided by one embodiment of the present invention;

[0037] Figure 5 A schematic structural diagram of an embodiment of an electronic device for map trajectory verification provided by one embodiment of the present invention. DETAILED DESCRIPTION

[0038] To make the objectives, technical solutions, and advantages of the embodiments of the present invention more clear, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments of the present invention without making any creative efforts shall fall within the scope of protection of the present invention.

[0039] Those skilled in the art will appreciate that the embodiments of the present application may be implemented as a system, apparatus, device, method, or computer program product. Therefore, the present disclosure may be implemented in the following forms: entirely in hardware, entirely in software (including firmware, resident software, microcode, etc.), or in a combination of hardware and software.

[0040] For ease of understanding, the technical terms involved in this application are explained below:

[0041] The "mobile device" referred to in this application includes but is not limited to vehicles with six autonomous driving technology levels, L0-L5, as established by the Society of Automotive Engineers International (SAE International) or the Chinese national standard "Automotive Driving Automation Classification".

[0042] In some embodiments, the mobile device may be a vehicle or a robotic device having the following functions:

[0043] (1) Passenger-carrying function, such as family cars and buses;

[0044] (2) Cargo carrying function, such as ordinary trucks, box trucks, trailer trucks, closed trucks, tank trucks, flatbed trucks, container trucks, dump trucks, special structure trucks, etc.;

[0045] (3) Tool functions, such as logistics delivery vehicles, automated guided vehicles (AGVs), patrol cars, cranes, hoists, excavators, bulldozers, forklifts, road rollers, loaders, off-road engineering vehicles, armored engineering vehicles, sewage treatment vehicles, sanitation vehicles, vacuum trucks, floor scrubbers, sprinkler trucks, sweeping robots, food delivery robots, shopping guide robots, lawn mowers, golf carts, etc.;

[0046] (4) Entertainment functions, such as amusement vehicles, amusement park self-driving devices, balance vehicles, etc.;

[0047] (5) Special rescue functions, such as fire trucks, ambulances, power repair trucks, engineering rescue trucks, etc.

[0048] like Figure 1FIG2 is a flowchart of a map trajectory verification method provided by an embodiment of the present invention, comprising the following steps:

[0049] S11: Collecting map construction data and verification data of driving in the map respectively;

[0050] S12: performing dead reckoning using the verification data to determine an entire verification trajectory consisting of a plurality of trajectory sub-segments;

[0051] S13: For a first track subsegment of a valid satellite positioning signal, among the plurality of track subsegments, optimize the track using the valid satellite positioning data and the first track subsegment to obtain a first track under satellite positioning coordinates;

[0052] S14: For the first track subsegment of the invalid satellite positioning signal, using a known first relative position between the acquisition end location of the map construction data and the starting point of the verification track, transform the first track subsegment into the satellite positioning coordinate system to obtain an estimated second track in the satellite positioning coordinate system;

[0053] S15: Perform map trajectory verification based on the first trajectory and the second trajectory.

[0054] In this embodiment, considering actual usage, separately collecting mapping and verification data is a relatively low-cost solution. This method is improved on this basis to solve the possible problems of verification data not overlapping with the collected map construction data and being out of map. This method can calculate the shape of the entire verification trajectory for matching verification.

[0055] In step S11, since this method is based on the separate collection of map construction data and verification data of driving within the map, the separately collected map construction data (in laser point cloud format) and verification data are combined. The verification data is collected by a separate vehicle and includes sensor data recorded by the vehicle's sensors. In one embodiment, the sensor data includes at least inertial measurement unit data and wheel speedometer data.

[0056] In this implementation, sensor data includes GPS (Global Positioning System) data, IMU (Inertial Measurement Unit) data, wheel speedometer data, and lidar point cloud data. For example, GPS data (if available during the acquisition process) can be used to obtain the vehicle's trajectory in the GPS coordinate system during the acquisition process. Wheel speedometer data and IMU data can be used to estimate the vehicle's actual driving trajectory.

[0057] In step S12, the verification data from step S11 is unpacked and then time-synchronized with the aforementioned data examples to ensure that each data point at the same time accurately reflects the vehicle's data at that point in time. Dead Reckoning (DR) is then performed using the IMU data and wheel speedometer data to obtain the calculated verification trajectory. The calculated verification trajectory is described by a series of keyframes, each of which is a frame of point cloud data. This is why the verification data also includes LiDAR point cloud data, which is reliable in the short term and describes the vehicle's general motion shape. (Longer term, GPS loss may affect the trajectory shape.) To avoid this, the entire trajectory is divided into multiple track subsegments. This not only facilitates matching the trajectory with map construction data but also ensures relatively accurate track subsegments over short distances. This ensures that the trajectory corresponding to each track subsegment accurately describes the vehicle's motion within that subsegment.

[0058] Step S13 takes into account the fact that, due to the obstruction of buildings and obstacles during the acquisition process (for example, dense high-rise buildings may block the GPS signal), the verification trajectory may have some track segments with valid GPS signals while others do not. To address these different situations, two different matching methods are used.

[0059] The satellite positioning signal described in the steps is obtained by GPS, GNSS or BeiDou. The satellite positioning signal below takes the GPS signal as an example.

[0060] As an embodiment, for the first track subsegment of the valid satellite positioning signal, performing track optimization using the valid satellite positioning data and the first track subsegment includes:

[0061] Using valid satellite positioning data and the first track trajectory sub-segment for smoothing;

[0062] Matching is performed based on the smoothed first flight track sub-segment with the point cloud in the map construction data to correct abnormal data in the first flight track sub-segment.

[0063] In this embodiment, since the satellite positioning data is valid, the satellite positioning data can be used to perform trajectory smoothing on the first track trajectory sub-segment, and the satellite positioning data can be used to correct the abnormal points in the first track trajectory sub-segment to obtain a smooth first track trajectory sub-segment in the satellite positioning data coordinate system.

[0064] Then, based on the smoothed first track sub-segment, nearby point cloud data in the map construction data is found, and the first track sub-segment is matched with the point cloud. Since relatively good initial track values ​​already exist at this point, it can be expected that the map matching algorithm will be successful in most areas. This step outputs the matching results of each key frame of the verification trajectory with the map. Specifically, since the first track sub-segment has a valid GPS signal, the approximate location of the track of this sub-segment in the map construction data can be known. The point cloud data near this location is selected and matched with the key frames in the track sub-segment of this sub-segment to obtain a matching result.

[0065] Furthermore, matching the smoothed first track sub-segment with the point cloud in the map construction data includes:

[0066] Selecting a plurality of matching points to be checked in the first track sub-segment,

[0067] Based on the number of threads of the central processing unit, multi-thread parallel matching is performed on the multiple matching points to be checked.

[0068] In this embodiment, the matching algorithm for the first track subsegment with the map construction data can utilize a multi-threaded parallel matching approach. In this multi-threaded parallel matching approach, a large number of matching points to be checked within the first track subsegment can be specified at once. The algorithm creates multiple threads based on the number of CPU threads on the computer execution device and distributes the matching calculations across these threads. The matching algorithm is implemented using the Normal Distribution Transform (NDT), and the algorithm receives and records matching results with higher NDT scores.

[0069] Regarding step S14, during the verification data collection process of the vehicle while driving, some track segments do not have valid GPS signals due to the obstruction of buildings, obstacles, etc. during the collection process (for example, dense high-rise buildings, there will be no GPS signal in such areas).

[0070] For first track subsegments with invalid satellite positioning signals, since the track's position in the satellite positioning coordinate system is unknown, only the starting point of each second track subsegment and the end point of map construction data collection are known. In other words, the second track subsegment does not have a valid satellite positioning signal, but the relative position of the second track subsegment to the starting point of the entire verification track is known (hereinafter referred to as the "second relative position"). If the starting point of the entire verification track can be found in the map, the approximate position of the second track subsegment in the map construction data can be determined through coordinate conversion and the second relative position between the second track subsegment and the starting point of the entire verification track, thereby obtaining an estimated second track in the satellite positioning coordinate system.

[0071] To facilitate finding the starting point of the entire verification trajectory, as one embodiment, step S11 separately collects map construction data and verification data of driving on the map, including: starting verification data collection at the location where map construction data collection ends. That is, through collection constraints, the starting point of the entire verification trajectory is ensured to be as continuous as possible with the location where map construction data collection ends. This collection constraint ensures that the location where map construction data collection ends and the starting point of the entire verification trajectory have a known first relative position: the starting point of the entire verification trajectory is located within a preset area around the location where map construction data collection ends. In specific implementations, due to factors such as vehicle size and sensor collection errors, it is not possible to ensure that the starting point of the verification trajectory completely coincides with the location where map construction data collection ends. The only guarantee is that the starting point of the verification trajectory is within a preset area around the location where map construction data collection ends, such as a circular area with a radius of 5 to 20 meters centered at the location where map construction data collection ends.

[0072] Furthermore, if the starting point of the verification trajectory is located within a predetermined area around the final location of map data collection, a grid search can be performed to locate the specific location of the starting point of the verification trajectory. In one embodiment, a grid search can be performed using the final location of map data collection as the origin to determine the location and orientation of the starting point of the verification trajectory.

[0073] In this embodiment, a grid search is used to determine the starting point. As long as the position and orientation of the starting point of the verification track can be found in the map construction data, the entire verification track can be transformed into this coordinate system for subsequent processing.

[0074] Specifically, a grid search involves performing a grid search with the final location of map data collection as the origin, iterating through possible initial values ​​and attempting to match the map data with the point cloud under various initial values. In practice, a range (for example, 20 meters), a step size (0.5 meters), an angular range (360 degrees), and an angular step size (15 degrees) are given. All possible initial values ​​are then generated to determine the position and orientation of the validation trajectory's starting point. With the position and orientation of the validation trajectory's starting point, and the first trajectory subsegment being part of the entire validation trajectory, the second relative position of each subsegment in the first trajectory subsegment relative to the validation trajectory's starting point is determined. This allows the first trajectory subsegment to be transformed into the satellite positioning coordinate system. These ranges and step sizes are determined empirically. Generally speaking, the grid search converges to the correct value within close proximity. This ensures the reliability of the validation trajectory calculation in scenarios with invalid satellite positioning signals. If constraints are imposed on the verification data collection process, the starting point of the entire verification trajectory can be located near the location where the map data collection ended. Therefore, by using the location where the map data collection ended as the origin and traversing all possible points near it, the starting point of the entire verification trajectory can be found, further improving the performance of the grid search.

[0075] Furthermore, based on the position and orientation of the starting point of the verification trajectory, the second track sub-segments can be transformed into the satellite positioning coordinate system using the second relative position between each second track sub-segment and the starting point of the verification trajectory.

[0076] For step S15, the first trajectory under the satellite positioning coordinates and the estimated second trajectory under the satellite positioning coordinate system are obtained through the above steps, and the first trajectory and the second trajectory are matched with the map construction data. In order to further ensure the accuracy of map verification, after obtaining the matching results of the verification trajectory and the map construction data, the trajectory determined in the above steps, the GPS readings at the time of valid satellite positioning signals and the trajectory and map construction data matching results are used as constraints to calculate the pose graph optimization. Among them, pose graph optimization is a robust optimization method used to solve optimization problems with pose as optimization variables and constraints, and it can also ensure strong optimization capabilities after some constraints fail. Abnormal data in the first trajectory and the second trajectory can be excluded, and the overall steps are as follows: Figure 2 shown.

[0077] This further ensures that, for example, when the satellite positioning signal is invalid, a track subsegment that should have been matched successfully is misjudged as an anomaly due to missing GPS data, thereby improving the accuracy of map track verification overall.

[0078] In one embodiment, the performing map trajectory verification based on the first trajectory and the second trajectory includes:

[0079] Performing map matching processing based on the first trajectory and the second trajectory to obtain an initial estimated trajectory T0;

[0080] Based on the initial estimated trajectory T0, posture processing is performed to determine a first optimized estimated trajectory T1;

[0081] The map matching process and the pose graph optimization process are performed on the i-th optimized estimated trajectory to determine the i+1-th optimized estimated trajectory until the n-th optimized estimated trajectory fully matches the map, where i is a natural number greater than 1 and less than n.

[0082] The map trajectory is verified using the nth optimized estimated trajectory. In this embodiment, if there is a GPS missing in some areas of the map construction data, then even if the above-mentioned step S14 is processed, the estimated trajectory set may not be accurate enough, so that its matching with the map construction data will also be inaccurate with a certain probability. At this time, further matching can be performed based on the obtained pose graph. In other words, the pose graph is used to optimize the inaccurate trajectory set, and the key frames in the optimized estimated trajectory set are further matched with the map to obtain a new pose graph. In other words, this forms a cyclic iterative matching process, and the number of iterations is i as described above. If the match is sufficient and the optimization converges (reaches convergence or reaches the maximum predetermined number of iterations n), the calculation of the verification trajectory is considered to be completely completed, and the trajectory result is output; otherwise, the map match is continued to be found based on the smoothed result, and further iterative optimization is performed. In this way, a more accurate matching result can be obtained.

[0083] Finally, as an implementation manner, performing map trajectory verification based on the first trajectory and the second trajectory includes:

[0084] Matching each key frame of each trajectory in the first trajectory and the second trajectory with the map construction data to obtain a matching result between the key frame and the map;

[0085] When the matching result reaches a preset threshold, the map construction data is verified;

[0086] When the matching result does not reach a preset threshold, an alarm is issued for the key frame that is not successfully matched.

[0087] Output the matching results between keyframes and maps and the entire verification trajectory consisting of multiple track segments to assist manual review.

[0088] In the output mode of this embodiment, considering the purpose of this method, the entire verification trajectory, each key frame of each trajectory in the trajectory set, and the verified status of the map construction data are output.

[0089] For example, if the goal is simply to verify map quality, only the keyframe-map matching results can be output. Specifically, when the keyframe-map matching rate reaches a preset threshold, the quality of the map data is good. Otherwise, for ease of use, an alarm will be issued for keyframes that failed to match, prompting the map data developer to manually review the map quality of the corresponding area, thus improving their user experience.

[0090] If the cartographer is very familiar with the map construction data and has the baseline trajectory of the map construction data in his memory, then the entire verification trajectory composed of multiple track sub-segments can also be output. The cartographer can determine at a glance whether the track estimated by the verification data is accurate, which improves the cartographer's efficiency.

[0091] It can be seen from this implementation that, based on the low-cost map trajectory verification of separately collected mapping and verification data, a method of separately collecting mapping data and verification data to verify the validity of the map is realized, which solves the problem of verification data not coinciding with collected data and being out of map. In addition, the verification trajectory calculation method based on pose graph optimization can prevent the real-time algorithm from losing positioning and deflecting when local signals are missing. It also solves the problem of real-time positioning failure due to discontinuous starting points of the verification trajectory and being out of map in the real-time positioning calculation scheme. Even if the GPS of the verification trajectory fails or part of the trajectory is outside the map, a smooth trajectory result can be obtained, and it is guaranteed that the trajectory inside the map can be correctly matched.

[0092] As an embodiment, the pose graph optimization process includes at least one pose graph constraint, wherein the pose graph constraint includes: satellite positioning signal reading constraint, track relative motion constraint, map matching constraint, and local keyframe constraint.

[0093] In this embodiment, various graph optimization processes can be used to optimize the trajectory. Specifically:

[0094] For satellite positioning signal reading constraints (using GPS as an example), if GPS determines that it is in a usable state, that is, a valid satellite positioning signal as described in the above steps, GPS can be used as a pose prior and applied to the pose of each keyframe. However, due to GPS multipath effects, this can lead to reading errors when the GPS solution is fixed. Therefore, a robust kernel function (for example, a Cauchy kernel can be used) is added to all GPS constraints. The kernel function threshold is given by GPS accuracy (for example, 15cm can be used in practice, but the specific value is for illustration only and is not limited here). This invalidates anomalous GPS data caused by GPS multipath effects and prevents subsequent use of this data.

[0095] Track relative motion constraints: Tracks provide accurate short-term relative motion constraints to constrain the relative motion between adjacent keyframes. Because tracks can exhibit abnormalities such as wheel slip, these constraints are also subject to a robust kernel. This invalidates IMU and tachometer data resulting from such anomalies (thus eliminating verification tracks derived from erroneous IMU and tachometer data), preventing them from being used in subsequent steps.

[0096] Map matching constraints give the matching relationship between each keyframe and each local sub-map in the map construction data, which can be used to constrain the position constraints of the keyframe relative to the map. However, it should be noted that a keyframe in the track may also be matched with multiple local maps at the same time, so there may be multiple constraints of this type for each keyframe. Specifically, by setting map matching constraints, the XYZ coordinate values ​​of the point cloud corresponding to each keyframe in the track are constrained. For example, if the coordinate values ​​of the point cloud corresponding to each keyframe in a certain track sub-segment run out of the constrained XYZ coordinate values, then the xyz coordinate data that does not meet the constraints will be invalid and will no longer be used subsequently.

[0097] For areas where satellite positioning signals are missing, that is, where the satellite positioning signals are invalid, local keyframe constraints will also constrain the keyframes in the track and the local map to be on the same height level. Specifically, since the XY coordinates are no longer accurate due to the lack of satellite positioning signals, the height (i.e., Z coordinate value) of the point cloud in the keyframe where GPS information is missing is constrained by setting a height constraint. The Z coordinate data that does not meet the constraint is marked as invalid, and these data will not be used subsequently, so that their height is within the constraint range. For example, the height interval of the map layer is [0, 3m], and the height of the point cloud in a keyframe where GPS information is missing is 3.5m, which does not meet the constraint and will not be used subsequently. Simply put, for keyframes in track trajectories where GPS information is missing, local keyframe constraints can be used, and for keyframes in track trajectories where GPS information is not missing, map matching constraints can be used.

[0098] Finally, after joint optimization of multiple constraints on the pose graph, outliers in the estimated trajectory are eliminated, such as Figure 3 As shown in the figure, the middle track is the construction track of the map data, the two sides are the verification tracks, and the intersection line in the middle is the adjacent constraint between the construction track and the verification track. After solving the pose graph optimization, we can obtain an estimated track based on these constraints, thereby improving the accuracy of map track verification.

[0099] like Figure 4FIG2 is a schematic structural diagram of a map trajectory verification execution device provided by an embodiment of the present invention. The system can execute the map trajectory verification method described in any of the above embodiments and is configured in a terminal.

[0100] The present embodiment provides a map trajectory verification execution device 10 including: a data acquisition module 11 , a verification trajectory determination module 12 , a first trajectory determination module 13 , a second trajectory determination module 14 and a verification module 15 .

[0101] Among them, the data acquisition module 11 is used to respectively collect map construction data and verification data of driving in the map; the verification trajectory determination module 12 is used to use the verification data to perform track calculation and determine the entire verification trajectory composed of multiple track sub-segments; the first trajectory determination module 13 is used to use the valid satellite positioning data and the first track sub-segment to perform trajectory optimization for the first track sub-segment with a valid satellite positioning signal among the multiple track sub-segments, so as to obtain a first trajectory under satellite positioning coordinates; the second trajectory determination module 14 is used to use the known first relative position between the collection end point of the map construction data and the starting point of the verification trajectory for the first track sub-segment with an invalid satellite positioning signal, to transform the first track sub-segment to the satellite positioning coordinate system, so as to obtain an estimated second trajectory under the satellite positioning coordinate system; the verification module 15 is used to perform map trajectory verification based on the first trajectory and the second trajectory.

[0102] An embodiment of the present invention further provides a non-volatile computer storage medium storing computer executable instructions, wherein the computer executable instructions can execute the map trajectory verification method in any of the above method embodiments;

[0103] As an embodiment, the non-volatile computer storage medium of the present invention stores computer-executable instructions, and the computer-executable instructions are configured as follows:

[0104] respectively collecting map construction data and verification data of driving in the map;

[0105] performing dead reckoning using the verification data to determine an entire verification trajectory comprised of a plurality of trajectory subsegments;

[0106] Among the plurality of track subsegments, for a first track subsegment of a valid satellite positioning signal, using the valid satellite positioning data and the first track subsegment to perform track optimization to obtain a first track under satellite positioning coordinates;

[0107] For the first track subsegment of the invalid satellite positioning signal, using a known first relative position between the end point of collection of the map construction data and the starting point of the verification track, the first track subsegment is transformed into the satellite positioning coordinate system to obtain an estimated second track in the satellite positioning coordinate system;

[0108] Map trajectory verification is performed based on the first trajectory and the second trajectory.

[0109] A non-volatile computer-readable storage medium can be used to store non-volatile software programs, non-volatile computer executable programs, and modules, such as the program instructions / modules corresponding to the methods described in the embodiments of the present invention. One or more program instructions stored in the non-volatile computer-readable storage medium, when executed by a processor, perform the map trajectory verification method described in any of the above method embodiments.

[0110] An embodiment of the present invention also provides an electronic device, comprising: at least one processor, and a memory communicatively connected to the at least one processor, wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to execute a map trajectory verification method applied to a trajectory prediction model of a mobile device.

[0111] In some embodiments, the present invention further provides a mobile device comprising a main body and an electronic device according to any of the preceding embodiments mounted on the main body. The mobile device may be an unmanned vehicle, such as an unmanned sweeper, unmanned floor scrubber, unmanned logistics vehicle, unmanned passenger vehicle, unmanned sanitation vehicle, unmanned minibus / bus, truck, mining vehicle, etc., or a robot.

[0112] In some embodiments, an embodiment of the present invention further provides a computer program product, which, when executed on a computer, enables the computer to execute any one of the map trajectory verification methods for a trajectory prediction model applied to a mobile device described in the embodiments of the present invention.

[0113] Figure 5 This is a hardware structure diagram of an electronic device for a map trajectory verification method provided by another embodiment of the present application. Figure 5 As shown, the device includes:

[0114] One or more processors 510 and memory 520, Figure 5 The device for the map trajectory verification method may further include: an input device 530 and an output device 540.

[0115] The processor 510, the memory 520, the input device 530 and the output device 540 may be connected via a bus or other means. Figure 5 The bus connection is taken as an example.

[0116] Memory 520, as a non-volatile computer-readable storage medium, can be used to store non-volatile software programs, non-volatile computer executable programs, and modules, such as the program instructions / modules corresponding to the map trajectory verification method in the embodiments of this application. Processor 510 executes the non-volatile software programs, instructions, and modules stored in memory 520 to execute various server functional applications and data processing, thereby implementing the map trajectory verification method in the above-mentioned method embodiment.

[0117] The memory 520 may include a program storage area and a data storage area, wherein the program storage area may store an operating system and application programs required for at least one function; the data storage area may store data, etc. In addition, the memory 520 may include a high-speed random access memory and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other non-volatile solid-state storage device. In some embodiments, the memory 520 may optionally include a memory remotely located relative to the processor 510, and these remote memories may be connected to the mobile device via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0118] The input device 530 can receive input digital or character information. The output device 540 can include a display device such as a display screen.

[0119] The one or more modules are stored in the memory 520 and, when executed by the one or more processors 510 , perform the map trajectory verification method in any of the above method embodiments.

[0120] The above-mentioned product can execute the method provided in the embodiment of this application, and has the functional modules and beneficial effects corresponding to the execution method. For technical details not fully described in this embodiment, please refer to the method provided in the embodiment of this application.

[0121] The non-volatile computer-readable storage medium may include a program storage area and a data storage area, wherein the program storage area may store an operating system and application programs required for at least one function; the data storage area may store data created based on the use of the device, etc. In addition, the non-volatile computer-readable storage medium may include high-speed random access memory and may also include non-volatile memory, such as at least one disk storage device, flash memory device, or other non-volatile solid-state memory device. In some embodiments, the non-volatile computer-readable storage medium may optionally include a memory remotely located relative to the processor, and these remote memories may be connected to the device via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0122] An embodiment of the present invention also provides an electronic device, comprising: at least one processor, and a memory communicatively connected to the at least one processor, wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the steps of the map trajectory verification method of any embodiment of the present invention.

[0123] The electronic devices of the embodiments of the present application exist in various forms, including but not limited to:

[0124] (1) Mobile communication devices: These devices are characterized by their mobile communication capabilities and their primary purpose is to provide voice and data communications. These terminals include smartphones, multimedia phones, feature phones, and low-end phones.

[0125] (2) Ultra-mobile personal computer devices: These devices fall under the category of personal computers and have computing and processing capabilities, and generally also have mobile Internet access. These terminals include PDAs, MIDs, and UMPC devices, such as tablet computers.

[0126] (3) Portable entertainment devices: These devices can display and play multimedia content. They include audio and video players, handheld game consoles, e-books, smart toys, and portable car navigation devices.

[0127] (4) Other mobile devices with data processing capabilities.

[0128] In this document, relational terms such as first and second are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the terms "include" and "comprise" include not only those elements, but also other elements not explicitly listed, or elements inherent to such process, method, article or device. In the absence of further limitations, the elements defined by the statement "include..." do not exclude the presence of other identical elements in the process, method, article or device that includes the elements.

[0129] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, i.e., they may be located in one location or distributed across multiple network units. Some or all of the modules may be selected based on actual needs to achieve the objectives of the present embodiment. Persons of ordinary skill in the art will be able to understand and implement the present invention without inventive effort.

[0130] Through the above description of the embodiments, those skilled in the art will clearly understand that each embodiment can be implemented using software plus a necessary general-purpose hardware platform, or of course, hardware. Based on this understanding, the essence of the above technical solution, or the portion that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a magnetic disk, or an optical disk, and includes a number of instructions for causing a computer device (such as a personal computer, server, or network device) to execute the methods described in each embodiment or certain portions of the embodiments.

[0131] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the various embodiments of the present invention.

Claims

1. A map trajectory verification method, comprising: respectively collecting map construction data and verification data of driving in the map; The map construction data adopts the laser point cloud format; performing dead reckoning using the verification data to determine an entire verification trajectory comprised of a plurality of trajectory subsegments; Among the plurality of track subsegments, for a first track subsegment of a valid satellite positioning signal, using the valid satellite positioning data and the first track subsegment to perform track optimization to obtain a first track under satellite positioning coordinates; For the second track subsegment of the invalid satellite positioning signal, transform the second track subsegment into the satellite positioning coordinate system using the known first relative position between the acquisition end location of the map construction data and the starting point of the verification trajectory to obtain an estimated second trajectory in the satellite positioning coordinate system; Performing map trajectory verification based on the first trajectory and the second trajectory; The performing map trajectory verification based on the first trajectory and the second trajectory includes: Performing map matching processing based on the first trajectory and the second trajectory to obtain an initial estimated trajectory T0; Based on the initial estimated trajectory T0, performing pose graph optimization processing to determine a first optimized estimated trajectory T1; Performing the map matching process and the pose graph optimization process on the i-th optimized estimated trajectory to determine the i+1-th optimized estimated trajectory, until the n-th optimized estimated trajectory fully matches the map, where i is a natural number greater than 1 and less than n; Use the nth optimized estimated trajectory to verify the map trajectory; The pose graph optimization process includes at least one pose graph constraint, wherein the pose graph constraint includes: satellite positioning signal reading constraint, track relative motion constraint, map matching constraint, and local key frame constraint.

2. The method according to claim 1, characterized in that The performing dead reckoning using the verification data comprises: Dead reckoning is performed using vehicle-mounted driving sensor data in the verification data, wherein the sensor data at least includes: inertial measurement unit data and wheel speed meter data.

3. The method according to claim 1, characterized in that For the first flight path subsegment of the valid satellite positioning signal, performing trajectory optimization using the valid satellite positioning data and the first flight path subsegment includes: Smoothing the first track sub-segment using valid satellite positioning data; Matching is performed based on the smoothed first flight track sub-segment with the point cloud in the map construction data to correct abnormal data in the first flight track sub-segment.

4. The method according to claim 1, wherein Collect map construction data and verification data of driving in the map separately, including: Starting to collect the verification data at the location where the collection of the map construction data ends; Then, the known first relative position between the collection end location of the map construction data and the starting point of the verification trajectory is: the starting point of the verification trajectory is located in a preset area around the collection end location of the map construction data.

5. The method according to claim 4, characterized in that For the first track subsegment of the invalid satellite positioning signal, transforming the first track subsegment into the satellite positioning coordinate system by using a known first relative position between the acquisition end location of the map construction data and the starting point of the verification track comprises: Performing a grid search with the map construction data collection end point as the origin to determine the position and orientation of the starting point of the verification trajectory; Based on the position and orientation of the starting point of the verification trajectory, the first flight track sub-segment is transformed into the satellite positioning coordinate system using a second relative position between the first flight track sub-segment and the starting point of the verification trajectory.

6. The method according to claim 3, characterized in that The matching of the smoothed first track sub-segment with the point cloud in the map construction data comprises: Selecting a plurality of matching points to be checked in the first track subsegment; Based on the number of threads of the central processing unit, multi-thread parallel matching is performed on the multiple matching points to be checked.

7. The method according to claim 1, characterized in that The entire verification trajectory is composed of point cloud data corresponding to each key frame.

8. The method according to claim 1, characterized in that Satellite positioning signals are obtained by GPS, GNSS or BeiDou.

9. The method according to claim 1, characterized in that The performing map trajectory verification based on the first trajectory and the second trajectory includes: Matching each key frame of each trajectory in the first trajectory and the second trajectory with the map construction data to obtain a matching result between the key frame and the map; When the matching result reaches a preset threshold, the map construction data is verified; When the matching result does not reach a preset threshold, an alarm is issued for the key frame that is not successfully matched.

10. The method according to claim 9, characterized in that The performing map trajectory verification based on the first trajectory and the second trajectory further includes: The matching results between the keyframe and the map and the entire verification trajectory consisting of multiple track subsegments are output to assist manual review.

11. A map trajectory verification execution device, characterized in that: include: a data collection module for collecting map construction data and verification data of driving in the map; The map construction data adopts the laser point cloud format; a verification trajectory determination module, configured to perform dead reckoning using the verification data to determine an entire verification trajectory consisting of a plurality of trajectory sub-segments; a first trajectory determination module configured to, for a first trajectory subsegment of a valid satellite positioning signal among the plurality of trajectory subsegments, perform trajectory optimization using the valid satellite positioning data and the first trajectory subsegment to obtain a first trajectory under satellite positioning coordinates; a second trajectory determination module configured to transform, for a first track subsegment with an invalid satellite positioning signal, the first track subsegment into the satellite positioning coordinate system using a known first relative position between an end point of collection of the map construction data and a starting point of the verification trajectory, to obtain an estimated second trajectory in the satellite positioning coordinate system; a verification module, configured to perform map trajectory verification based on the first trajectory and the second trajectory; The performing map trajectory verification based on the first trajectory and the second trajectory includes: Performing map matching processing based on the first trajectory and the second trajectory to obtain an initial estimated trajectory T0; Based on the initial estimated trajectory T0, performing pose graph optimization processing to determine a first optimized estimated trajectory T1; Performing the map matching process and the pose graph optimization process on the i-th optimized estimated trajectory to determine the i+1-th optimized estimated trajectory, until the n-th optimized estimated trajectory fully matches the map, where i is a natural number greater than 1 and less than n; Use the nth optimized estimated trajectory to verify the map trajectory; The pose graph optimization process includes at least one pose graph constraint, wherein the pose graph constraint includes: satellite positioning signal reading constraint, track relative motion constraint, map matching constraint, and local key frame constraint.

12. An electronic device comprising: At least one processor, and a memory communicatively connected to the at least one processor, wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the steps of the method according to any one of claims 1 to 10. 13 . A mobile device comprising a body and the electronic device according to claim 12 mounted on the body.

14. A storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the steps of the method according to any one of claims 1 to 10 are implemented.

Images