Track generation method and device, computing equipment, storage medium and program product

By detecting the initial trajectory of the movable device and processing the visual odometry, the problem of low map accuracy caused by independent processing errors of sensor data is solved, and the generated trajectory is more accurate.

CN118913247APending Publication Date: 2024-11-08XIAN NAVINFO INFORMATION TECH CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202410954770.3
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-07-16
Publication Date
2024-11-08

Smart Images

  • Figure CN118913247A_ABST
    Figure CN118913247A_ABST
Patent Text Reader

Abstract

The embodiment of the invention provides a track generation method and device, computing equipment, a storage medium and a program product, and relates to the technical field of maps. The method comprises the following steps: acquiring work order data of the mobile equipment in a preset time period, and determining at least one piece of hopping section data in the work order data; performing pose estimation processing on the jump section data to obtain visual odometer VO section data corresponding to the jump section data; and according to the VO segment data and the work order data, generating fusion trajectory data of the mobile device. According to the method, high-quality tracks can be generated, and map precision is improved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the field of map technology, and in particular to a trajectory generation method, apparatus, computing device, storage medium and program product. Background Art

[0002] High-precision maps provide rich positioning and navigation information during the movement of movable devices such as vehicles and robots; high-precision maps can be drawn by generating the movement trajectory of movable devices.

[0003] In the prior art, the sensor data collected by each sensor is first processed independently to obtain corresponding independent processing results, and then these independent processing results are fused through a fusion algorithm to obtain the movement trajectory of the movable device for generating a map.

[0004] However, in the above process, there are errors in the independent processing of sensor data, which leads to a large difference between the movement trajectory obtained after the fusion processing and the actual trajectory, which in turn leads to low accuracy of the drawn map. Summary of the invention

[0005] The embodiments of the present application provide a trajectory generation method, apparatus, computing device, storage medium and program product to solve the technical problem that errors in independent processing of sensor data result in a large difference between the obtained movement trajectory and the actual trajectory.

[0006] In a first aspect, an embodiment of the present application provides a trajectory generation method, comprising:

[0007] Acquire work order data of the movable device within a preset time period, and determine at least one jump segment data in the work order data; wherein the work order data represents the initial movement trajectory of the movable device within the preset time period; the jump segment data represents the trajectory segment where the trajectory jump occurs in the initial movement trajectory of the movable device within the preset time period;

[0008] Performing pose estimation processing on the jump segment data to obtain visual odometer VO segment data corresponding to the jump segment data; wherein the VO segment data represents the moving position and moving direction of the movable device in three-dimensional space;

[0009] According to each of the VO segment data and the work order data, fused trajectory data of the movable device is generated; wherein the fused trajectory data represents the final movement trajectory of the movable device within the preset time period; and the fused trajectory data is used to generate a map.

[0010] In a possible implementation, generating fused trajectory data of the movable device according to each of the VO segment data and the work order data includes:

[0011] The combined navigation data in the work order data and each of the VO segment data are fused to obtain the fused trajectory data; wherein the combined navigation data is obtained by fusion processing based on the navigation satellite system data and the inertial navigation system data; the combined navigation data includes initial trajectory points at multiple collection times within a preset time period.

[0012] In a possible implementation, the VO segment data includes VO trajectory points at multiple collection times; the combined navigation data in the work order data and each of the VO segment data are fused to obtain the fused trajectory data, including:

[0013] According to the combined navigation data, the VO segment data is scaled and restored to obtain restored segment data; wherein the restored segment data includes a restored trajectory point corresponding to each of the VO trajectory points;

[0014] Determine the start anchor point and the end anchor point corresponding to the recovery segment data from the combined navigation data; wherein the start anchor point is the first initial trajectory point in the neighbor segment data corresponding to the recovery segment data; the neighbor segment data is all non-jump segment data in the combined navigation data whose collection time is adjacent to the collection time corresponding to the recovery segment data; the non-jump segment data represents a trajectory segment in which no jump occurs in the initial moving trajectory of the movable device; the end anchor point is the last initial trajectory point in the neighbor segment data;

[0015] According to the starting anchor point, the restored segment data is aligned to obtain aligned segment data; wherein the aligned segment data includes an aligned trajectory point corresponding to each restored trajectory point;

[0016] The aligned segment data is subjected to rotation and translation processing according to the start anchor point and the end anchor point to obtain the fused trajectory data.

[0017] In a possible implementation, performing scale restoration processing on the VO segment data according to the combined navigation data to obtain restored segment data includes:

[0018] According to the video data in the work order data, the target segment data corresponding to the VO segment data is determined from the combined navigation data; wherein the video data includes images of the driving scene in which the movable device is located at multiple acquisition times; the target segment data includes a target trajectory point corresponding to each of the VO trajectory points; the target trajectory point is an initial trajectory point whose first serial number in the combined navigation data is consistent with the second serial number of the VO trajectory point; the first serial number is an image frame number whose acquisition time in the video data is consistent with the acquisition time of the initial trajectory point; the second serial number is an image frame number whose acquisition time in the video data is consistent with the acquisition time of the VO trajectory point;

[0019] Determine the speed information of the target trajectory point corresponding to the VO trajectory point, which is the speed information of the VO trajectory point; wherein the speed information represents the moving speed of the movable device at the corresponding acquisition time;

[0020] The VO segment data is scaled and restored according to the speed information of each of the initial trajectory points to obtain the restored segment data.

[0021] In a possible implementation, performing alignment processing on the restored segment data according to the start anchor point to obtain aligned segment data includes:

[0022] Determine the combined pose data and VO pose data corresponding to the starting anchor point; wherein the combined pose data is the position and posture of the starting anchor point relative to the first initial track point in the combined navigation data; the VO pose data is the position and posture of the starting anchor point relative to the first recovery track point in the recovery segment data corresponding to the starting anchor point;

[0023] Determine relative posture data between the combined posture data and the VO posture data according to the combined posture data and the VO posture data;

[0024] According to the relative posture data, the restored segment data is aligned and transformed to obtain the aligned segment data.

[0025] In a possible implementation, performing rotation and translation processing on the aligned segment data according to the start anchor point and the end anchor point to obtain the fused trajectory data includes:

[0026] Determine a first vector and a second vector corresponding to the alignment segment data according to the start anchor point and the end anchor point; wherein the first vector is a vector between a first alignment track point and a last alignment track point in the alignment segment data; and the second vector is a vector between the start anchor point and the end anchor point;

[0027] Generate a rotation matrix between the first vector and the second vector according to the first vector and the second vector;

[0028] According to the rotation matrix, the aligned segment data is rotated to obtain rotated segment data; wherein the rotated segment data includes a rotation trajectory point corresponding to each aligned trajectory point;

[0029] According to the combined navigation data, each of the rotation segment data is translated to obtain the fused trajectory data.

[0030] In a possible implementation manner, determining at least one jump segment data in the work order data includes:

[0031] Performing trajectory jump detection processing on the combined navigation data and the navigation satellite system data in the work order data to obtain a detection result; wherein the combined navigation data is obtained by fusing the navigation satellite system data and the inertial navigation system data; the detection result includes at least one jump point; the jump point is a critical trajectory point where a trajectory jump occurs in the initial moving trajectory of the movable device;

[0032] According to each of the jump points, the combined navigation data is segmented to obtain each of the jump segment data and at least one non-jump segment data; wherein the non-jump segment data represents a trajectory segment in which no jump occurs in the initial moving trajectory of the movable device.

[0033] In a possible implementation manner, segmenting the integrated navigation data according to each of the jump points to obtain each of the jump segment data and at least one non-jump segment data includes:

[0034] According to each of the jump points, the combined navigation data is segmented to obtain at least one original jump segment data and each of the non-jump segment data corresponding to the combined navigation data; wherein the original jump segment data represents the original trajectory segment where the trajectory jump occurs in the initial moving trajectory of the movable device within a preset time period;

[0035] The original jump segment data is expanded according to the neighbor segment data of the original jump segment data to obtain the jump segment data; wherein the neighbor segment data is the non-jump segment data whose collection time is adjacent to the collection time of the original jump segment data.

[0036] In a possible implementation, the method further includes:

[0037] Performing a trajectory quality inspection process on the fused trajectory data to obtain an inspection result of the fused trajectory data;

[0038] If it is determined that the inspection result indicates that the trajectory quality corresponding to the fused trajectory data is qualified, a map is generated according to the fused trajectory data.

[0039] In a second aspect, an embodiment of the present application provides a trajectory generation device, comprising:

[0040] A determination module, used to obtain work order data of a movable device within a preset time period, and determine at least one jump segment data in the work order data; wherein the work order data represents an initial movement trajectory of the movable device within the preset time period; and the jump segment data represents a trajectory segment where a trajectory jump occurs in the initial movement trajectory of the movable device within the preset time period;

[0041] A processing module, used to perform posture estimation processing on the jump segment data to obtain visual odometer VO segment data corresponding to the jump segment data; wherein the VO segment data represents the moving position and moving direction of the movable device in three-dimensional space;

[0042] A generation module is used to generate fused trajectory data of the movable device according to each of the VO segment data and the work order data; wherein the fused trajectory data represents the final movement trajectory of the movable device within the preset time period; and the fused trajectory data is used to generate a map.

[0043] In a possible implementation, the generation module is specifically used to: fuse the combined navigation data in the work order data and each of the VO segment data to obtain the fused trajectory data; wherein the combined navigation data is obtained by fusing the navigation satellite system data and the inertial navigation system data; and the combined navigation data includes initial trajectory points at multiple collection times within a preset time period.

[0044] In a possible implementation, the VO segment data includes VO trajectory points at multiple acquisition times; the generating module is specifically used to: perform scale recovery processing on the VO segment data according to the combined navigation data to obtain recovery segment data; wherein the recovery segment data includes the recovery trajectory point corresponding to each of the VO trajectory points; determine the start anchor point and the end anchor point corresponding to the recovery segment data from the combined navigation data; wherein the start anchor point is the first initial trajectory point in the neighbor segment data corresponding to the recovery segment data; the neighbor segment data is all non-jump segment data whose acquisition time in the combined navigation data is adjacent to the acquisition time corresponding to the recovery segment data; the non-jump segment data represents the trajectory segment in which no jump occurs in the initial moving trajectory of the movable device; the end anchor point is the last initial trajectory point in the neighbor segment data; according to the start anchor point, align the recovery segment data to obtain aligned segment data; wherein the aligned segment data includes the alignment trajectory point corresponding to each of the recovery trajectory points; according to the start anchor point and the end anchor point, rotate and translate the aligned segment data to obtain the fused trajectory data.

[0045] In a possible implementation, the generation module is specifically used to: determine the target segment data corresponding to the VO segment data from the combined navigation data based on the video data in the work order data; wherein the video data includes images of the driving scenes in which the movable device is located at multiple acquisition times; the target segment data includes a target trajectory point corresponding to each of the VO trajectory points; the target trajectory point is an initial trajectory point whose first serial number in the combined navigation data is consistent with the second serial number of the VO trajectory point; the first serial number is the image frame number whose acquisition time in the video data is consistent with the acquisition time of the initial trajectory point; the second serial number is the image frame number whose acquisition time in the video data is consistent with the acquisition time of the VO trajectory point; determine the speed information of the target trajectory point corresponding to the VO trajectory point, which is the speed information of the VO trajectory point; wherein the speed information represents the moving speed of the movable device at the corresponding acquisition time; and perform scale recovery processing on the VO segment data based on the speed information of each of the initial trajectory points to obtain the recovered segment data.

[0046] In a possible implementation, the generation module is specifically used to: determine the combined pose data and VO pose data corresponding to the starting anchor point; wherein the combined pose data is the position and posture of the starting anchor point relative to the first initial trajectory point in the combined navigation data; the VO pose data is the position and posture of the starting anchor point relative to the first recovery trajectory point in the recovery segment data corresponding to the starting anchor point; based on the combined pose data and the VO pose data, determine the relative pose data between the combined pose data and the VO pose data; based on the relative pose data, perform alignment transformation processing on the recovery segment data to obtain the aligned segment data.

[0047] In a possible implementation, the generation module is specifically used to: determine the first vector and the second vector corresponding to the alignment segment data according to the start anchor point and the end anchor point; wherein the first vector is the vector between the first alignment trajectory point and the last alignment trajectory point in the alignment segment data; the second vector is the vector between the start anchor point and the end anchor point; generate a rotation matrix between the first vector and the second vector according to the first vector and the second vector; rotate the alignment segment data according to the rotation matrix to obtain rotated segment data; wherein the rotated segment data includes a rotation trajectory point corresponding to each of the alignment trajectory points; and translate each of the rotation segment data according to the combined navigation data to obtain the fused trajectory data.

[0048] In a possible implementation, the determination module is specifically used to: perform trajectory jump detection processing on the combined navigation data and navigation satellite system data in the work order data to obtain a detection result; wherein the combined navigation data is obtained based on a fusion processing of the navigation satellite system data and the inertial navigation system data; the detection result includes at least one jump point; the jump point is a critical trajectory point where a trajectory jump occurs in the initial moving trajectory of the movable device; according to each of the jump points, the combined navigation data is segmented to obtain each of the jump segment data and at least one non-jump segment data; wherein the non-jump segment data represents a trajectory segment in which no jump occurs in the initial moving trajectory of the movable device.

[0049] In a possible implementation, the determination module is specifically used to: segment the combined navigation data according to each of the jump points to obtain at least one original jump segment data and each of the non-jump segment data corresponding to the combined navigation data; wherein the original jump segment data represents the original trajectory segment where the trajectory jump occurs in the initial moving trajectory of the movable device within a preset time period; and expand the original jump segment data according to the neighbor segment data of the original jump segment data to obtain the jump segment data; wherein the neighbor segment data is the non-jump segment data in each of the non-jump segment data whose collection time is adjacent to the collection time of the original jump segment data.

[0050] In a possible implementation, the device is further used to: perform a trajectory quality inspection process on the fused trajectory data to obtain an inspection result of the fused trajectory data; if it is determined that the inspection result indicates that the trajectory quality corresponding to the fused trajectory data is qualified, generate a map according to the fused trajectory data.

[0051] In a third aspect, an embodiment of the present application provides a computing device, including: a memory, a processor;

[0052] The memory stores computer-executable instructions;

[0053] The processor executes the computer-executable instructions stored in the memory, so that the processor executes the above first aspect and / or various possible implementations of the first aspect.

[0054] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, in which computer-executable instructions are stored. When the computer-executable instructions are executed by a processor, they are used to implement the first aspect above and / or various possible implementations of the first aspect.

[0055] In a fifth aspect, an embodiment of the present application provides a computer program product, including a computer program, which, when executed by a processor, implements the above first aspect and / or various possible implementation methods of the first aspect.

[0056] The trajectory generation method, apparatus, computing device, storage medium and program product provided in the embodiments of the present application detect the initial trajectory of the movable device during movement, find out the trajectory segments where the trajectory jumps, perform the visual odometer VO algorithm on these trajectory segments where the trajectory jumps, calculate the corresponding visual trajectory segments, and generate a high-quality final trajectory by loosely coupling the initial trajectory and the visual trajectory segments, thereby improving the map accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0057] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0058] Figure 1 A schematic diagram of an application scenario provided for this application;

[0059] Figure 2 A schematic diagram of a flow chart of a trajectory generation method provided in an embodiment of the present application;

[0060] Figure 3 A schematic diagram of a flow chart of a trajectory loosely coupled calculation process provided in an embodiment of the present application;

[0061] Figure 4 A schematic diagram of a flow chart of another trajectory generation method provided in an embodiment of the present application;

[0062] Figure 5 A schematic diagram of a trajectory fusion process provided in an embodiment of the present application;

[0063] Figure 6 A schematic diagram of an extension segment provided in an embodiment of the present application;

[0064] Figure 7 A sample schematic diagram of trceval.txt provided in an embodiment of the present application;

[0065] Figure 8 A schematic diagram of a trajectory jump situation provided in an embodiment of the present application;

[0066] Fig. 9 A schematic diagram of a posture optimization process provided in an embodiment of the present application;

[0067] Fig.10 A schematic diagram of a trajectory fusion effect provided in an embodiment of the present application;

[0068] Fig.11 A schematic diagram of a process flow of trajectory quality evaluation provided in an embodiment of the present application;

[0069] Fig.12 A schematic diagram of the structure of a trajectory generating device provided in an embodiment of the present application;

[0070] Fig.13 A schematic diagram of the structure of a computing device provided in an embodiment of the present application.

[0071] The above drawings have shown clear embodiments of the present application, which will be described in more detail later. These drawings and text descriptions are not intended to limit the scope of the present application in any way, but to illustrate the concept of the present application to those skilled in the art by referring to specific embodiments. DETAILED DESCRIPTION

[0072] Here, exemplary embodiments will be described in detail, and examples thereof are shown in the accompanying drawings. When the following description refers to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present application. Instead, they are only examples of devices and methods consistent with some aspects of the present application.

[0073] First, the terms involved in this application are explained:

[0074] Global Navigation Satellite System: GNSS is the abbreviation of Global Navigation Satellite System. It is a technology that provides global positioning, navigation and timing services through a satellite system deployed in the Earth's orbit. The GNSS system uses a group of satellites distributed in the Earth's orbit to send positioning signals to receivers on the ground. The receiver determines its own position, speed and time information by analyzing these signals.

[0075] Inertial Measurement Unit: IMU is the abbreviation of Inertial Measurement Unit, which is the core component of the inertial navigation system. It consists of a variety of inertial sensors used to measure and report in real time the specific force, angular velocity and, in some cases, the direction of the object around the magnetic field. The main function of IMU is to provide the carrier's motion and orientation data, which is essential for determining the carrier's precise position, velocity and attitude. IMU usually includes the following types of sensors: accelerometer, which is used to measure the acceleration of the carrier in three spatial axes, that is, the acceleration along the X, Y and Z axes. Acceleration data can be used to calculate velocity and displacement by integration; gyroscope, which is used to measure the angular velocity of the carrier around the three spatial axes, that is, the rotation rate around the X, Y and Z axes. By integrating the angular velocity data, the rotation angle of the carrier can be obtained, thereby determining its direction; magnetometer (in some IMUs), which is used to measure the direction of the carrier relative to the earth's magnetic field, providing information about the direction of the carrier in the earth's magnetic field. The data of IMU is fused and solved by appropriate data processing algorithms (such as Kalman filters) to provide accurate navigation solutions. IMUs are widely used in a variety of fields, including aerospace, robotics, sports science, mobile and video game control, and the automotive industry. They are essential for systems that require precise motion tracking and navigation. The accuracy of the IMU's output information largely determines the accuracy of the entire inertial navigation system, so the accuracy and reliability of the IMU has a direct impact on the performance of the entire system.

[0076] Inertial Navigation System: INS is the abbreviation of Inertial Navigation System. Inertial Navigation System is an autonomous navigation system that does not rely on external information and does not radiate energy to the outside, like a radio navigation system. INS mainly measures the acceleration of the carrier in the inertial reference frame through internal gyroscopes and accelerometers, and integrates these measurements over time to infer information such as the carrier's speed, yaw angle and position in the navigation coordinate system. The core components of the inertial navigation system include: Inertial Measurement Unit (IMU), which usually includes accelerometers and gyroscopes to measure the acceleration and angular velocity of the carrier; computing unit, which provides filtering algorithm processing and fusion of raw sensor data to calculate the carrier's attitude (position and direction) and speed. There are two main ways to implement inertial navigation systems: platform inertial navigation systems, which use an electromechanical platform to stabilize the measuring axis of the accelerometer and provide a baseline for the three-degree-of-freedom linear acceleration; strapdown inertial navigation systems (SINS), which do not use an electromechanical platform, but instead mount the inertial elements directly on the carrier, and perform complex calculations and transformations through a "mathematical" platform established in the computer to obtain the carrier's speed and position. INS is widely used in aviation, aerospace, submarines, mobile robots, self-driving cars and other fields to provide positioning, orientation and speed information. It is particularly suitable for situations where GPS signals are unavailable, such as tunnels or indoor environments. At this time, INS can be extrapolated based on the last known GPS signal and continue to provide navigation information.

[0077] GNSS / INS Integration Navigation-GNSS / INS integrated navigation: Modern INS systems can be used in conjunction with global navigation satellite systems (GNSS) such as GPS to form GPS-assisted INS to reduce the drift error accumulated by INS over time and improve navigation accuracy. The fusion of the two sensors GNSS / INS is referred to as integrated navigation. The working principle of GNSS / INS integration is as follows: GNSS provides high-precision absolute position and time information, while INS provides continuous navigation capabilities when GNSS is not available. When GNSS signals are available, INS can use this information for calibration to reduce its own accumulated errors over time; in some applications, such as safety-critical applications, even if GNSS signals are interfered or blocked, the integrated navigation system can still rely on INS to continue to provide navigation information, thereby increasing the reliability of the system; when GNSS signals are available, INS can be calibrated to improve accuracy. At the same time, INS can provide faster response than GNSS in a short period of time, especially when the carrier is performing high-speed or large maneuvers; integrated navigation systems usually use advanced data processing algorithms (such as Kalman filters) to fuse GNSS and INS data to provide more accurate and reliable navigation solutions than single systems.

[0078] Visual odometry: Visual odometry (VO) is a technique that uses camera images to estimate the distance and direction a mobile robot or autonomous vehicle has moved relative to its previous position. It belongs to the field of computer vision and robotic navigation, and mainly relies on extracting feature points from a continuous sequence of images and tracking the position changes of these feature points in the image sequence. The key advantages of visual odometry are: it does not rely on external signals, so it can work in indoor environments or urban canyons where GPS signals are not available; under good visual conditions, visual odometry can provide very accurate motion estimates. Visual odometry is widely used in fields such as robotic navigation, autonomous vehicles, mobile device positioning, and augmented reality. It can be combined with other sensors (such as lidar, IMU or GPS) to form a multi-sensor fusion system to improve the accuracy and robustness of navigation.

[0079] Loosely Coupled: Loosely Coupled is a sensor fusion method in which data from different sensors are combined at a higher level, usually after each sensor independently provides its own navigation solution. In a loosely coupled system, the data from each sensor is processed independently, and then their outputs (such as position, velocity, attitude, etc.) are combined at a higher level to form a comprehensive navigation solution. Loosely coupled systems are suitable for applications that do not have extremely high real-time requirements, or when synchronization and coordination between sensors are difficult during the system integration phase. Loose coupling often occurs in situations where cost and flexibility are more important, or when there are challenges in sensor integration, such as some consumer drones, indoor navigation systems, etc.

[0080] Trajectory: refers to the position and posture. The trajectory belongs to 6 degrees of freedom, including rotation and transformation.

[0081] Work order: A work order is a business processing unit. The collection time of each work order is 1 minute, the video collection frequency is 15 frames per second, the IMU collection frequency is 1000 frames per second, and the GNSS collection frequency is 1 frame per second.

[0082] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use, processing, transmission, provision, disclosure and application of the relevant data comply with relevant laws, regulations and standards, take necessary confidentiality measures, do not violate public order and good morals, and provide corresponding operation entrances for users to choose to authorize or refuse.

[0083] It should be noted that the trajectory generation method, device, removable device, storage medium and program product of the present application can be used in the field of map technology, and can also be used in any field other than map technology. The application field of the trajectory generation method, device, removable device, storage medium and program product of the present application is not limited.

[0084] High-precision maps provide rich positioning and navigation information during the movement of movable devices such as vehicles and robots; high-precision maps can be drawn by generating the movement trajectory of movable devices.

[0085] In one example, the sensor data collected by each sensor is first processed independently to obtain corresponding independent processing results, and then these independent processing results are fused through a fusion algorithm to obtain the movement trajectory of the movable device for generating a map.

[0086] However, in the above method, loose coupling may not be able to fully utilize the complementary information between different sensors; independent processing may cause errors to be amplified before fusion; the generated trajectory pose is very different from the actual trajectory and cannot be directly applied to automatic map generation.

[0087] In response to the above technical problems, this application proposes the following technical concept: by detecting the initial trajectory of the movable device during movement, finding the trajectory segments where the trajectory jumps, and separately performing the visual odometer VO algorithm on these trajectory segments where the trajectory jumps, the corresponding visual trajectory segments are calculated, and by loosely coupling the initial trajectory and the visual trajectory segments, a high-quality final trajectory is generated. In this way, the problem that independent processing in the prior art may cause errors to be amplified before fusion, resulting in a large difference between the generated trajectory pose and the real trajectory, can be solved.

[0088] The technical solution of the present application and how the technical solution of the present application solves the above-mentioned technical problems are described in detail below with specific embodiments. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments. The embodiments of the present application will be described below in conjunction with the accompanying drawings.

[0089] Figure 1 This is a schematic diagram of an application scenario provided by this application. Figure 1 As shown, the specific application scenario of the present application includes a movable device such as a driving recorder in a vehicle on a city road, which includes a computing device 101 and a data acquisition device 102. The data acquisition device 102 collects work order data representing the initial movement trajectory of the driving recorder. The computing device 101 obtains the work order data and processes the work order data to obtain the movement trajectory of the vehicle where the driving recorder is located for crowdsourcing mapping.

[0090] Figure 2 A schematic diagram of a trajectory generation method provided in an embodiment of the present application is shown in FIG. Figure 2 As shown, the method includes:

[0091] 201. Obtain work order data of a movable device within a preset time period, and determine at least one jump segment data in the work order data; wherein the work order data represents an initial moving trajectory of the movable device within the preset time period; and the jump segment data represents a trajectory segment where a trajectory jump occurs in the initial moving trajectory of the movable device within the preset time period.

[0092] Exemplarily, the execution subject of this embodiment may be a computing device. For ease of description, the device is referred to as a computing device below. The device may be a physical device or a virtual device that executes the trajectory generation method. Figure 3 A schematic diagram of a process flow of a trajectory loosely coupled calculation process provided in an embodiment of the present application, such as Figure 3 As shown, a mobile device such as a collection vehicle will generate work orders in chronological order during collection. Each work order includes GNSS data gps.txt received by the GNSS receiver in the driving recorder, IMU data collected by the IMU sensor 102, inertial navigation system INS data, video data video.mp4 collected by the rolling shutter camera, collection time data time.txt, and generated metadata meta.json. The device can obtain the work order data containing these work orders to characterize the initial movement trajectory of the mobile device within a preset time period. The overall process control ArkRunner module deployed in the device, the A The rkRunner module is responsible for the overall process control to call each module. Based on the ArkRunner module, the trajectory evaluation TrcEval (INS mode) module deployed in the device is called. Based on the trajectory jump detection method in the TrcEval module, the input file Input, that is, the acquired work order data, is detected and processed to check whether the trajectory posture has jumped, and the trajectory segment where the trajectory jump occurs in the initial moving trajectory of the movable device within the preset time period is determined as a jump segment, so that one or more jump segment data can be obtained, and these jump segment data are written to trceval_ins.txt for processing.

[0093] 202. Perform pose estimation processing on the jump segment data to obtain visual odometer VO segment data corresponding to the jump segment data; wherein the VO segment data represents the moving position and moving direction of the movable device in three-dimensional space.

[0094] For example, in combination Figure 3, the ArkRunner module in the device calls the pre-deployed visual odometer VO module. Based on the posture estimation processing technology allowed in the VO module, each frame of the video data in the work order data can be used to perform posture estimation processing on each jump segment data in trceval_ins.txt, and the visual trajectory segment corresponding to each jump segment data is obtained, and aligned with the corresponding trajectory of the original jump segment data, that is, the VO segment data imgpost_vo_{segid}.txt is obtained. Each VO segment data represents the moving position and moving direction of the movable device in the corresponding three-dimensional space; for example, using the PnP (Perspective-n-Point) algorithm, the posture (position and direction) of the movable device in the camera space is estimated according to the three-dimensional coordinates in each jump segment data obtained and the corresponding two-dimensional image points in the video data. It is worth noting that after the posture estimation processing, each VO segment data obtained after the processing can also be subjected to quality evaluation processing to obtain VO segment data trceval_vo_{segid}.txt with qualified quality.

[0095] 203. Generate fused trajectory data of the movable device according to each VO segment data and the work order data; wherein the fused trajectory data represents the final movement trajectory of the movable device within a preset time period; and the fused trajectory data is used to generate a map.

[0096] For example, in combination Figure 3 The overall process control ArkRunner module in the device, based on the loosely coupled algorithm Trcfusion module, fuses the obtained VO segment data imgpost_vo_{segid}.txt and the work order data to obtain the fusion segment, and uses the fusion segment to backfill the original initial movement trajectory to generate the final movement trajectory of the movable device within the preset time period with high quality, which is the fusion trajectory data imgpost_fusion.txt; then, the obtained fusion trajectory data imgpost_fusion.txt is converted and output to obtain the file vec02 for map generation processing, such as crowdsourcing mapping of urban road scenes.

[0097] In this embodiment, a trajectory generation method is provided, which detects the initial trajectory of a movable device during movement, finds out the trajectory segments where the trajectory jumps, performs a visual odometer VO algorithm on these trajectory segments where the trajectory jumps, calculates the corresponding visual trajectory segments, and generates a high-quality final trajectory by loosely coupling the initial trajectory and the visual trajectory segments, thereby improving the map accuracy.

[0098] Figure 4 A flow chart of another trajectory generation method provided in an embodiment of the present application is shown as follows: Figure 4As shown, the method includes:

[0099] 301. Obtain work order data of movable equipment within a preset time period.

[0100] Exemplarily, this step may refer to step 201 and will not be described in detail here.

[0101] 302. Perform trajectory jump detection processing on the combined navigation data and the navigation satellite system data in the work order data to obtain a detection result; wherein the combined navigation data is obtained by fusion processing based on the navigation satellite system data and the inertial navigation system data; the detection result includes at least one jump point; the jump point is a critical trajectory point where a trajectory jump occurs in the initial moving trajectory of the movable device.

[0102] Exemplarily, the work order data acquired in the device includes navigation satellite system data GNSS and inertial navigation system INS data. In order to reduce the error accumulated over time in the inertial navigation system data itself and improve the data accuracy, the navigation satellite system data and the inertial navigation system data are fused and processed by using advanced data processing algorithms (such as Kalman filter) or graph optimization-based methods to obtain the combined navigation data contained in the work order data to characterize the mobile position, attitude and speed information of the movable device in a preset time period. The total process control ArkRunner module deployed in the device calls the trajectory evaluation TrcEval (INS mode) module deployed in the device, and detects and processes the combined navigation data based on the trajectory jump detection method in the TrcEval module, checks whether the trajectory posture has jumped, and finds the critical trajectory points where the position, attitude, etc. of each trajectory point in the combined navigation data jump, that is, obtains one or more jump points included in the detection result for processing. It is worth adding that each track point in the combined navigation data imgpost_ins.txt corresponds to and is associated with each frame of the video data video.mp4. Each track point corresponds to a line of information, each line of information corresponds to a posture, and each line of information contains multiple fields, such as seq field, time field, and other fields. The seq field is the image frame number after de-framing in the corresponding video.mp4. For example, if seq is 0, it means that this line corresponds to the posture corresponding to the seq 0 frame of the image. The time field is the timestamp of the camera exposure time of the corresponding image in the acquisition. The other fields are used to record the position, posture, and speed information of the IMU sensor in the ENU coordinate system at the exposure time of the image frame of the corresponding image.

[0103] 303. According to each jump point, the combined navigation data is segmented to obtain each jump segment data and at least one non-jump segment data; wherein the non-jump segment data represents a trajectory segment in which no jump occurs in the initial moving trajectory of the movable device.

[0104] For example, the TrcEval module in the device divides the combined navigation data imgpost_ins.txt into multiple segments according to each jump point and records them with seq. Some of these segments are good quality segments, i.e., non-jump segment data, to characterize the track segments without jumps in the initial moving track of the movable device, and some are segments with track jumps, i.e., jump segments, and the information of these segments will be written into trceval_ins.txt. Thus, more accurate jump segment data can be obtained so that more accurate tracks can be obtained in subsequent processing.

[0105] In one example, step 303 includes the following steps:

[0106] The first step of step 303 is to segment the combined navigation data according to each jump point to obtain at least one original jump segment data and each non-jump segment data corresponding to the combined navigation data; wherein the original jump segment data represents the original trajectory segment where the trajectory jump occurs in the initial moving trajectory of the movable device within a preset time period.

[0107] The second step of step 303 is to expand the original jump segment data according to the neighbor segment data of the original jump segment data to obtain the jump segment data; wherein the neighbor segment data is the non-jump segment data whose collection time is adjacent to the collection time of the original jump segment data.

[0108] For example, Figure 5 A schematic diagram of a trajectory fusion process provided in an embodiment of the present application is shown in FIG. Figure 5As shown, the TrcEval module in the device divides the combined navigation data imgpost_ins.txt into multiple segments according to each jump point and records it with seq to obtain trceval.txt. Among the various trajectory segments in the trceval.txt, some are segments with good quality, that is, non-jump segment data SingleStatus.txt file, to characterize the trajectory segments without jump in the initial moving trajectory of the movable device. The SingleStatus.txt file can generate a single frame information file based on the GPS data in the work order data, which records whether the trajectory of the corresponding image frame is normal, GNSS information such as the number of satellites, horizontal precision factor (HypertensiveDisease Of Pregnancy, HDOP), GPS speed, GPS heading, etc.; Among the various trajectory segments in the trceval.txt, some are original trajectory segments with trajectory jumps, that is, original jump segment data. Figure 6 A schematic diagram of an extension segment provided in an embodiment of the present application, such as Figure 6 As shown, based on the TrcEval module, the device determines the non-jump segment data whose collection time is adjacent to the collection time of the original jump segment data from each non-jump segment data SingleStatus.txt file for each original jump segment data, as the neighbor segment data of the original jump segment data, and performs expansion processing on the original jump segment data according to the neighbor segment data of the original jump segment data to obtain the jump segment data corresponding to the original jump segment data. It is worth noting that the purpose of expansion is to better initialize the VO module, and to form an intersection between segments, so as to facilitate the adjustment of the VO segment data, so as to better use it in the subsequent fusion stage to obtain a more accurate trajectory.

[0109] For example, the TrcEval module in the device divides the integrated navigation data imgpost_ins.txt into multiple segments according to each jump point and records them with seq to obtain trceval.txt. Figure 7 A sample schematic diagram of trceval.txt provided in an embodiment of the present application is shown in FIG. Figure 7As shown in the figure, the status column in trceval.txt indicates whether there is a trajectory jump in this segment, that is, the Sequence Record, which contains whether the trajectory position of each segment has a jump. 0 indicates that this trajectory segment has no jump, and 1 indicates that this trajectory segment has a jump. Each Seq record usually also contains the sequence itself and some metadata, such as the sequence ID, description, source, etc. The Seq record also contains annotation information about the sequence; each segment has a segid (SegmentIdentifier) ​​as the segment ID to distinguish different segments. In the trceval.txt, for each trajectory segment without jump, seq_start and seq_end record the frame number of the corresponding start and end images of the segment in the video data video.mp4; for each original segment with jump, it needs to be extended 25 meters forward and backward. The extended segment is the extended segment, that is, the final jump segment. Each extended segment is recorded in the two fields of seq_start and seq_end, and bad_start and bad_end record the original segment with jump.

[0110] 304. Perform pose estimation processing on the jump segment data to obtain visual odometer VO segment data corresponding to the jump segment data; wherein the VO segment data represents the moving position and moving direction of the movable device in the three-dimensional space.

[0111] Exemplarily, this step may refer to step 202 and will not be described in detail here.

[0112] 305. The combined navigation data in the work order data and the data of each VO segment are fused to obtain fused trajectory data; wherein the combined navigation data is obtained by fusion processing based on the navigation satellite system data and the inertial navigation system data; the combined navigation data includes initial trajectory points at multiple collection times within a preset time period.

[0113] Exemplarily, the work order data acquired by the device includes navigation satellite system data GNSS and inertial navigation system INS data. In order to improve data accuracy, the navigation satellite system data and the inertial navigation system data are fused by using advanced data processing algorithms to obtain combined navigation data contained in the work order data; the combined navigation data includes initial trajectory points at multiple acquisition times within a preset time period, and the acquisition time is consistent with the acquisition time corresponding to each image in the video data in the work order data or the camera exposure time of the image. The TrcFusion module of the device is responsible for fusing each VO segment data imgpost_vo_{segid}.txt and the combined navigation data imgpost_ins.txt, that is, posture optimization processing, to obtain the final fused trajectory data imgpost_fusion.txt, that is, the input of the TrcFusion module is imgpost_ins.txt, imgpost_vo_{segid}.txt, trceval_ins.txt, and the output is imgpost_fusion.txt. Furthermore, when GNSS signals are lost, jumped, degraded, or unavailable on urban roads such as in canyons with tall buildings or in tunnels, the accurate trajectory and pose of the vehicle are generated during its driving process, laying the foundation for crowdsourcing mapping of urban road scenes.

[0114] In one example, the VO segment data includes VO trajectory points at multiple acquisition times; step 305 includes the following steps:

[0115] The first step of step 305 is to perform scale restoration processing on the VO segment data according to the combined navigation data to obtain restored segment data; wherein the restored segment data includes the restored trajectory point corresponding to each VO trajectory point.

[0116] The second step of step 305 is to determine the start anchor point and the end anchor point corresponding to the recovery segment data from the combined navigation data; wherein the start anchor point is the first initial trajectory point in the neighbor segment data corresponding to the recovery segment data; the neighbor segment data is all non-jump segment data in the combined navigation data whose collection time is adjacent to the collection time corresponding to the recovery segment data; the non-jump segment data represents the trajectory segment in which no jump occurs in the initial moving trajectory of the movable device; and the end anchor point is the last initial trajectory point in the neighbor segment data.

[0117] The third step of step 305 is to align the restored segment data according to the starting anchor point to obtain aligned segment data; wherein the aligned segment data includes an aligned trajectory point corresponding to each restored trajectory point.

[0118] The fourth step of step 305 is to perform rotation and translation processing on the aligned segment data according to the start anchor point and the end anchor point to obtain fused trajectory data.

[0119] Exemplarily, each obtained VO segment data includes VO trajectory points at multiple acquisition moments, and the acquisition moment is consistent with the acquisition moment corresponding to each image in the video data in the work order data or the camera exposure moment of the image. In the trajectory fusion process, based on the preset fusion trajectory algorithm, according to the position, speed and posture information of each initial trajectory point in the combined navigation data imgpost_ins.txt, each VO trajectory point in each VO segment data is scaled and restored, that is, the YO segment data is restored from the scale in the virtual space to the real size in the real world, and then the restored trajectory point corresponding to each VO trajectory point in the restored segment data is obtained. For each obtained recovery segment data, a trajectory segment in which no jump occurs in the initial moving trajectory of the movable device, that is, each non-jump segment data, is obtained from the combined navigation data, and all non-jump segment data whose collection time is adjacent to the collection time corresponding to the recovery segment data are determined from each non-jump segment data as the neighbor segment data corresponding to the recovery segment data, and the first initial trajectory point in the neighbor segment data corresponding to the recovery segment data is determined as the start anchor point corresponding to the recovery segment data, and the last initial trajectory point in the neighbor segment data corresponding to the recovery segment data is determined as the end anchor point corresponding to the recovery segment data; for example, Figure 8 A schematic diagram of a trajectory jump situation provided in an embodiment of the present application is shown in FIG. Figure 8As shown, it includes the original GNSS trajectory gps.txt, the integrated navigation data imgpost_ins.txt, and the recovery segment data imgpost_vo_1.txt. For each trajectory segment without jumps, seq_start and seq_end record the frame numbers of the corresponding start and end images of the segment in the video data video.mp4; for each original segment with jumps, it needs to be extended 25 meters forward and backward, and the extended segment is the extended segment, that is, the final jump segment. Each extended segment is recorded in the two fields seq_start and seq_end. Bad_start and bad_end record the original segment with jumps. In the interval [seq_start, bad_start], it is recorded as start_valid_anchors, and the first element is recorded as the start anchor point start_anchor; in the interval [bad_end, seq_end], it is recorded as end_valid_anchors, and the last element is recorded as the end anchor point end_anchor. The device performs alignment processing on the recovery trajectory points in each recovery segment data according to the starting anchor point corresponding to each recovery segment data, that is, aligns each recovery segment data with the trajectory corresponding to the combined navigation data, and obtains the alignment segment data corresponding to each recovery segment data, and each alignment segment data includes the alignment trajectory point corresponding to each recovery trajectory point in the corresponding recovery segment data. The device performs rotation and translation processing on the alignment trajectory points in the alignment segment data corresponding to each recovery segment data according to the starting anchor point and the ending anchor point corresponding to each recovery segment data, and backfills them to the corresponding position in the combined navigation data, thereby obtaining the final fusion trajectory data.

[0120] In one example, the first step of step 305 includes:

[0121] Step 1. Determine the target segment data corresponding to the VO segment data from the combined navigation data according to the video data in the work order data; wherein the video data includes images of the driving scene in which the movable device is located at multiple acquisition times; the target segment data includes a target trajectory point corresponding to each VO trajectory point; the target trajectory point is an initial trajectory point whose first serial number in the combined navigation data is consistent with the second serial number of the VO trajectory point; the first serial number is an image frame number whose acquisition time in the video data is consistent with the acquisition time of the initial trajectory point; the second serial number is an image frame number whose acquisition time in the video data is consistent with the acquisition time of the VO trajectory point.

[0122] Step 2: Determine the speed information of the target trajectory point corresponding to the VO trajectory point, which is the speed information of the VO trajectory point; wherein the speed information represents the moving speed of the movable device at the corresponding acquisition time.

[0123] Step 3: According to the velocity information of each initial trajectory point, the VO segment data is scaled and restored to obtain the restored segment data.

[0124] Specifically, Fig. 9 A schematic diagram of a posture optimization process provided in an embodiment of the present application is shown in FIG. Fig. 9 As shown, the fusion process is specifically a posture optimization process. In this posture optimization process, for the data collected by the camera and other IMU / INS sensors in the work order data, the Enu to Camera stage is entered. The preset sensor coordinate system, such as the IMU / INS coordinate system / local navigation coordinate system, where each initial track point in the combined navigation data is located can be converted to the camera coordinate system for subsequent calculations. Then the SetVOInfo stage is entered. For each VO segment data, the device determines the image frame number whose acquisition time in the video data is consistent with the acquisition time of each initial track point in the combined navigation data, i.e., the first serial number, according to the video data in the work order data. The video data includes images of the driving scene where the movable device is located at multiple acquisition times. At the same time, the image frame number whose acquisition time in the video data is consistent with the acquisition time of each VO track point in the VO segment data is determined as the serial number of each VO track point in the VO segment data, i.e., the second serial number. The initial track point whose first serial number in the combined navigation data is consistent with the second serial number of each VO track point is determined as the corresponding target track point of each VO track point, i.e., each VO is obtained. The target segment data corresponding to the segment data, each target trajectory point has speed information to characterize the moving speed of the movable device at the corresponding acquisition time, and the speed information of the target trajectory point corresponding to each VO trajectory point is assigned to the speed information of each corresponding VO trajectory point, wherein, for each VO trajectory point with abnormal speed information, the adjacent VO trajectory points with normal speed information of the VO trajectory point are determined, and the speed information of the adjacent VO trajectory points of the VO trajectory point is interpolated, and the result is the speed information of the VO trajectory point, and then enters the ScaleVoWithSpeed ​​stage, according to the speed information of each initial trajectory point in the combined navigation data, the scale recovery processing is performed on each VO trajectory point in each VO segment data to obtain the corresponding recovery segment data.

[0125] In one example, the third step of step 305 includes:

[0126] Step 1, determine the combined pose data and VO pose data corresponding to the starting anchor point; wherein the combined pose data is the position and posture of the starting anchor point relative to the first initial trajectory point in the combined navigation data; the VO pose data is the position and posture of the starting anchor point relative to the first recovery trajectory point in the recovery segment data corresponding to the starting anchor point.

[0127] Step 2: Determine the relative posture data between the combined posture data and the VO posture data based on the combined posture data and the VO posture data.

[0128] Step 3: According to the relative posture data, the restored segment data is aligned and transformed to obtain the aligned segment data.

[0129] Specifically, combined Fig. 9 After the SearchAnchors stage, the start anchor point and the end anchor point corresponding to each recovery segment data are determined, and then the ConvertVo stage is entered. That is, for the start anchor point start_anchor corresponding to each recovery segment data, the position and posture of each start anchor point relative to the first initial trajectory point in the combined navigation data are determined according to the combined navigation data, and the combined pose data P_(ins,start_anchor)∈SE(3) corresponding to the start anchor point is obtained. Then, the position and posture of each start anchor point relative to the first recovery trajectory point in the recovery segment data corresponding to the start anchor point are determined, that is, the VO pose data P_(vo,start_anchor)∈SE(3) corresponding to the start anchor point is obtained. According to the preset calculation algorithm, Calculate the combined pose data P_(ins,start_anchor)∈SE(3) corresponding to each start anchor point and the VO pose data P_(vo,start_anchor)∈SE(3) corresponding to the start anchor point to obtain the relative pose data E_(start_anchor) between the two, where: It is an inverse combination operator, which accepts two postures and gives a relative posture. According to each relative posture data E_(start_anchor) calculated, it performs alignment transformation processing on each restored trajectory point in each restored segment data to obtain the corresponding aligned segment data.

[0130] In one example, the fourth step of step 305 includes:

[0131] Step 1: Determine the first vector and the second vector corresponding to the alignment segment data according to the start anchor point and the end anchor point; wherein the first vector is the vector between the first alignment trajectory point and the last alignment trajectory point in the alignment segment data; and the second vector is the vector between the start anchor point and the end anchor point.

[0132] Step 2: Generate a rotation matrix between the first vector and the second vector based on the first vector and the second vector.

[0133] Step 3: Rotate the aligned segment data according to the rotation matrix to obtain rotated segment data; wherein the rotated segment data includes a rotation trajectory point corresponding to each aligned trajectory point.

[0134] Step 4: According to the combined navigation data, the data of each rotation segment is translated to obtain fused trajectory data.

[0135] Specifically, combined Fig. 9 After the ConvertVo stage, each alignment segment data is obtained. When the start anchor point and the end anchor point corresponding to each recovery segment are different, the RotateVo stage is entered to execute the RotateVo function. That is, according to vo_vector = vo_t_end_anchor-vo_t_start_anchor, the vector between the first alignment trajectory point vo_t_start_anchor and the last alignment trajectory point vo_t_end_anchor in each alignment segment data is determined as the first vector vo_vector corresponding to each alignment segment data; according to ins_vector = ins_t_end_anchor-ins_t_start_anchor, the vector between the start anchor point ins_t_start_anchor and the end anchor point ins_t_end_anchor corresponding to each recovery segment is determined as the second vector ins_vector corresponding to the alignment segment data corresponding to each recovery segment. The rotation matrix between vo_vector and ins_vector can be directly obtained, that is, the rotation matrix R corresponding to each alignment segment data, such as Eigen::Matrix3d rotMatrix = (newline)Eigen::Quaterniond::FromTwoVectors(vo_vector,camera_vector)(newline).toRotationMatrix(), get the rotation matrix R, then according to the rotation matrix R corresponding to each alignment segment data, rotate and translate the alignment trajectory points in each alignment segment data, so as to align and fuse the trajectories corresponding to the full amount and navigation data, Fig.10 A schematic diagram of a trajectory fusion effect provided in an embodiment of the present application is shown in FIG. Fig.10 As shown in the figure, before the transformation, the INS trajectory corresponding to the combined navigation data and the VO trajectory corresponding to the aligned segment data are included, and after the transformation, the final fused trajectory data with higher quality is obtained. It is worth noting that in the scale recovery processing stage, if the trajectory of the combined navigation data is converted from the ENU coordinate system to the camera coordinate system, then the Camera to Enu stage is entered, and the trajectory of the corresponding combined navigation data needs to be converted from the camera coordinate system to the ENU coordinate system.

[0136] 306. Perform a trajectory quality inspection process on the fused trajectory data to obtain an inspection result of the fused trajectory data.

[0137] For example, Fig.11A schematic diagram of a process flow of trajectory quality evaluation provided in an embodiment of the present application is shown in FIG. Fig.11 As shown, after obtaining the fused trajectory data, in order to further improve the trajectory accuracy, the device performs trajectory quality evaluation processing on the initial trajectory segment INS segment and the backfilled jump segment VO segment in the fused trajectory data imgpost_fusion,txt based on the preset trajectory inspection algorithm, and obtains the inspection result to determine whether the fused trajectory data is qualified.

[0138] 307. If it is determined that the inspection result indicates that the trajectory quality corresponding to the fused trajectory data is qualified, a map is generated according to the fused trajectory data.

[0139] Exemplarily, based on the inspection results obtained, if the device determines that the trajectory quality corresponding to the fused trajectory data is qualified, the device processes the fused trajectory data based on the map generation technology to obtain a map. For the fused trajectory data with unqualified trajectory evaluation, the entire work order data corresponding to the fused trajectory data is discarded, and crowdsourcing mapping is no longer performed to avoid reducing the mapping quality.

[0140] In this embodiment, based on the above embodiment, on the one hand, further scale restoration, alignment, and rotation processing are performed on each VO segment data obtained to obtain higher quality fused trajectory data; on the other hand, trajectory detection processing is further performed on the obtained fused trajectory data to screen out higher quality fused trajectory data, thereby obtaining a map with higher accuracy.

[0141] Fig.12 A schematic diagram of the structure of a trajectory generating device provided in an embodiment of the present application is shown in FIG. Fig.12 As shown, the device comprises:

[0142] The determination module 401 is used to obtain the work order data of the movable device within a preset time period, and determine at least one jump segment data in the work order data; wherein the work order data represents the initial movement trajectory of the movable device within the preset time period; the jump segment data represents the trajectory segment where the trajectory jump occurs in the initial movement trajectory of the movable device within the preset time period;

[0143] The processing module 402 is used to perform posture estimation processing on the jump segment data to obtain visual odometer VO segment data corresponding to the jump segment data; wherein the VO segment data represents the moving position and moving direction of the movable device in the three-dimensional space;

[0144] The generation module 403 is used to generate fused trajectory data of the movable device according to each VO segment data and the work order data; wherein the fused trajectory data represents the final movement trajectory of the movable device within a preset time period; and the fused trajectory data is used to generate a map.

[0145] In one possible implementation, the generation module 403 is specifically used to: fuse the combined navigation data in the work order data and each VO segment data to obtain fused trajectory data; wherein the combined navigation data is obtained by fusing the navigation satellite system data and the inertial navigation system data; the combined navigation data includes initial trajectory points at multiple collection times within a preset time period.

[0146] In a possible implementation, the VO segment data includes VO trajectory points at multiple acquisition times; the generation module 403 is specifically used to: perform scale recovery processing on the VO segment data according to the combined navigation data to obtain recovery segment data; wherein the recovery segment data includes the recovery trajectory point corresponding to each VO trajectory point; determine the start anchor point and the end anchor point corresponding to the recovery segment data from the combined navigation data; wherein the start anchor point is the first initial trajectory point in the neighbor segment data corresponding to the recovery segment data; the neighbor segment data is all non-jump segment data whose acquisition time in the combined navigation data is adjacent to the acquisition time corresponding to the recovery segment data; the non-jump segment data represents the trajectory segment in which no jump occurs in the initial moving trajectory of the movable device; the end anchor point is the last initial trajectory point in the neighbor segment data; according to the start anchor point, align the recovery segment data to obtain aligned segment data; wherein the aligned segment data includes the alignment trajectory point corresponding to each recovery trajectory point; according to the start anchor point and the end anchor point, rotate and translate the aligned segment data to obtain fused trajectory data.

[0147] In a possible implementation, the generation module 403 is specifically used to: determine the target segment data corresponding to the VO segment data from the combined navigation data based on the video data in the work order data; wherein the video data includes images of the driving scenes in which the movable device is located at multiple acquisition times; the target segment data includes a target trajectory point corresponding to each VO trajectory point; the target trajectory point is an initial trajectory point whose first serial number in the combined navigation data is consistent with the second serial number of the VO trajectory point; the first serial number is the image frame number whose acquisition time in the video data is consistent with the acquisition time of the initial trajectory point; the second serial number is the image frame number whose acquisition time in the video data is consistent with the acquisition time of the VO trajectory point; determine the speed information of the target trajectory point corresponding to the VO trajectory point, which is the speed information of the VO trajectory point; wherein the speed information represents the moving speed of the movable device at the corresponding acquisition time; and perform scale recovery processing on the VO segment data based on the speed information of each initial trajectory point to obtain the recovered segment data.

[0148] In one possible implementation, the generation module 403 is specifically used to: determine the combined pose data and VO pose data corresponding to the starting anchor point; wherein the combined pose data is the position and posture of the starting anchor point relative to the first initial trajectory point in the combined navigation data; the VO pose data is the position and posture of the starting anchor point relative to the first recovery trajectory point in the recovery segment data corresponding to the starting anchor point; based on the combined pose data and the VO pose data, determine the relative pose data between the combined pose data and the VO pose data; based on the relative pose data, perform alignment transformation processing on the recovery segment data to obtain aligned segment data.

[0149] In a possible implementation, the generation module 403 is specifically used to: determine the first vector and the second vector corresponding to the alignment segment data according to the start anchor point and the end anchor point; wherein the first vector is the vector between the first alignment trajectory point and the last alignment trajectory point in the alignment segment data; the second vector is the vector between the start anchor point and the end anchor point; generate a rotation matrix between the first vector and the second vector according to the first vector and the second vector; rotate the alignment segment data according to the rotation matrix to obtain rotated segment data; wherein the rotated segment data includes a rotation trajectory point corresponding to each alignment trajectory point; and translate each rotation segment data according to the combined navigation data to obtain fused trajectory data.

[0150] In one possible implementation, the determination module 401 is specifically used to: perform trajectory jump detection processing on the combined navigation data and navigation satellite system data in the work order data to obtain a detection result; wherein the combined navigation data is obtained based on the fusion processing of the navigation satellite system data and the inertial navigation system data; the detection result includes at least one jump point; the jump point is a critical trajectory point where a trajectory jump occurs in the initial moving trajectory of the movable device; according to each jump point, the combined navigation data is segmented to obtain each jump segment data and at least one non-jump segment data; wherein the non-jump segment data represents a trajectory segment in which no jump occurs in the initial moving trajectory of the movable device.

[0151] In a possible implementation, the determination module 401 is specifically used to: segment the combined navigation data according to each jump point to obtain at least one original jump segment data and each non-jump segment data corresponding to the combined navigation data; wherein the original jump segment data represents the original trajectory segment where the trajectory jump occurs in the initial moving trajectory of the movable device within a preset time period; and expand the original jump segment data according to the neighbor segment data of the original jump segment data to obtain the jump segment data; wherein the neighbor segment data is the non-jump segment data in each non-jump segment data whose collection time is adjacent to the collection time of the original jump segment data.

[0152] In a possible implementation, the device is further used to: perform a trajectory quality inspection process on the fused trajectory data to obtain an inspection result of the fused trajectory data; if it is determined that the inspection result indicates that the trajectory quality corresponding to the fused trajectory data is qualified, then generate a map based on the fused trajectory data.

[0153] The device of this embodiment can execute the technical solution in the above method. Its specific implementation process and technical principles are the same and will not be repeated here.

[0154] It should be noted that it should be understood that the division of the various modules of the above device is only a division of logical functions. In actual implementation, they can be fully or partially integrated into one physical entity, or they can be physically separated. And these modules can all be implemented in the form of software called by processing elements; they can also be all implemented in the form of hardware; some modules can also be implemented in the form of software called by processing elements, and some modules can be implemented in the form of hardware. Each module can be a separately established processing element, or it can be integrated in a chip of the above device. In addition, it can also be stored in the memory of the above device in the form of program code, and called and executed by a processing element of the above device. The functions of each module. In addition, all or part of these modules can be integrated together, or they can be implemented independently. The processing element here can be an integrated circuit with signal processing capabilities. In the implementation process, each step of the above method or each module above can be completed by an integrated logic circuit of hardware in the processor element or instructions in the form of software.

[0155] Fig.13 A schematic diagram of a computing device provided in an embodiment of the present application is shown in FIG. Fig.13 As shown, the computing device includes: a memory 501, a processor 502; the memory 501; and a memory for storing instructions executable by the processor 502.

[0156] The processor 502 is configured to execute the method provided in the above embodiment.

[0157] The computing device 500 further includes a receiver 503 and a transmitter 504. The receiver 503 is used to receive instructions and data sent by other devices, and the transmitter 504 is used to send instructions and data to external devices.

[0158] In the above embodiments, it should be understood that the processor may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), etc. A general-purpose processor may be a microprocessor or any conventional processor. The steps of the method disclosed in the invention may be directly implemented as being executed by a hardware processor, or may be executed by a combination of hardware and software modules in the processor.

[0159] The memory may include a high-speed memory (Random Access Memory, RAM), and may also include a non-volatile memory (Non-volatile Memory, NVM), such as at least one disk memory.

[0160] The bus may be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. The bus may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, the bus in the drawings of the present application is not limited to only one bus or one type of bus.

[0161] The present application also provides a computer program product, including a computer program, which implements the above method when executed by a processor.

[0162] The present application also provides a computer-readable storage medium, in which computer-executable instructions are stored. When a processor executes the computer-executable instructions, the above method is implemented.

[0163] The above-mentioned readable storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk. The readable storage medium can be any available medium that can be accessed by a general or special-purpose computer.

[0164] An exemplary readable storage medium is coupled to a processor so that the processor can read information from the readable storage medium and write information to the readable storage medium. Of course, the readable storage medium can also be a component of the processor. The processor and the readable storage medium can be located in an application specific integrated circuit (Application Specific Integrated Circuits, referred to as: ASIC). Of course, the processor and the readable storage medium can also exist in the device as discrete components.

[0165] The division of units is only a logical function division, and there may be other divisions in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interface, device or unit, which can be electrical, mechanical or other forms.

[0166] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0167] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0168] If the function is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, server, or network device, etc.) to perform all or part of the steps of the methods of each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc. Various media that can store program codes.

[0169] Those skilled in the art can understand that all or part of the steps of implementing the above-mentioned method embodiments can be completed by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When the program is executed, the steps of the above-mentioned method embodiments are executed; and the aforementioned storage medium includes: ROM, RAM, disk or optical disk and other media that can store program codes.

[0170] Finally, it should be noted that those skilled in the art will readily conceive of other embodiments of the present invention after considering the specification and practicing the invention disclosed herein. The present invention is intended to cover any variations, uses or adaptations of the present invention, which follow the general principles of the present invention and include common knowledge or customary technical means in the art not disclosed by the present invention, are not limited to the precise structure described above and shown in the drawings, and may be modified and changed in various ways without departing from the scope thereof. The scope of the present invention is limited only by the appended claims.

Claims

1. A trajectory generation method, characterized in that: The method comprises: Acquire work order data of the movable device within a preset time period, and determine at least one jump segment data in the work order data; wherein the work order data represents the initial movement trajectory of the movable device within the preset time period; the jump segment data represents the trajectory segment where the trajectory jump occurs in the initial movement trajectory of the movable device within the preset time period; Performing pose estimation processing on the jump segment data to obtain visual odometer VO segment data corresponding to the jump segment data; wherein the VO segment data represents the moving position and moving direction of the movable device in three-dimensional space; According to each of the VO segment data and the work order data, fused trajectory data of the movable device is generated; wherein the fused trajectory data represents the final movement trajectory of the movable device within the preset time period; and the fused trajectory data is used to generate a map.

2. The method according to claim 1, characterized in that: Generating fused trajectory data of the movable device according to each of the VO segment data and the work order data, including: The combined navigation data in the work order data and each of the VO segment data are fused to obtain the fused trajectory data; wherein the combined navigation data is obtained by fusion processing based on the navigation satellite system data and the inertial navigation system data; the combined navigation data includes initial trajectory points at multiple collection times within a preset time period.

3. The method according to claim 2, characterized in that The combined navigation data in the work order data and each of the VO segment data are fused to obtain the fused trajectory data, including: According to the combined navigation data, the VO segment data is scaled and restored to obtain restored segment data; wherein the restored segment data includes a restored trajectory point corresponding to each of the VO trajectory points; Determining a start anchor point and an end anchor point corresponding to the recovery segment data from the combined navigation data; wherein the start anchor point is the first initial track point in the neighbor segment data corresponding to the recovery segment data; According to the starting anchor point, the restored segment data is aligned to obtain aligned segment data; wherein the aligned segment data includes an aligned trajectory point corresponding to each restored trajectory point; The aligned segment data is subjected to rotation and translation processing according to the start anchor point and the end anchor point to obtain the fused trajectory data.

4. The method according to claim 3, characterized in that According to the combined navigation data, the VO segment data is scaled and restored to obtain restored segment data, including: According to the video data in the work order data, the target segment data corresponding to the VO segment data is determined from the combined navigation data; wherein the target segment data includes a target track point corresponding to each of the VO track points; the target track point is an initial track point whose first serial number in the combined navigation data is consistent with the second serial number of the VO track point; Determine the speed information of the target trajectory point corresponding to the VO trajectory point, which is the speed information of the VO trajectory point; wherein the speed information represents the moving speed of the movable device at the corresponding acquisition time; The VO segment data is scaled and restored according to the speed information of each of the initial trajectory points to obtain the restored segment data.

5. The method according to claim 3, characterized in that: According to the start anchor point, the restored segment data is aligned to obtain aligned segment data, including: Determine the combined pose data and VO pose data corresponding to the starting anchor point; wherein the combined pose data is the position and posture of the starting anchor point relative to the first initial track point in the combined navigation data; the VO pose data is the position and posture of the starting anchor point relative to the first recovery track point in the recovery segment data corresponding to the starting anchor point; Determine relative posture data between the combined posture data and the VO posture data according to the combined posture data and the VO posture data; According to the relative posture data, the restored segment data is aligned and transformed to obtain the aligned segment data.

6. The method according to claim 3, characterized in that According to the start anchor point and the end anchor point, the aligned segment data is subjected to rotation and translation processing to obtain the fused trajectory data, including: Determine a first vector and a second vector corresponding to the alignment segment data according to the start anchor point and the end anchor point; wherein the first vector is a vector between a first alignment track point and a last alignment track point in the alignment segment data; and the second vector is a vector between the start anchor point and the end anchor point; Generate a rotation matrix between the first vector and the second vector according to the first vector and the second vector; According to the rotation matrix, the aligned segment data is rotated to obtain rotated segment data; wherein the rotated segment data includes a rotation trajectory point corresponding to each aligned trajectory point; According to the combined navigation data, each of the rotation segment data is translated to obtain the fused trajectory data.

7. The method according to claim 1, characterized in that Determining at least one jump segment data in the work order data includes: Performing trajectory jump detection processing on the combined navigation data and the navigation satellite system data in the work order data to obtain a detection result; wherein the combined navigation data is obtained by fusing the navigation satellite system data and the inertial navigation system data; the detection result includes at least one jump point; the jump point is a critical trajectory point where a trajectory jump occurs in the initial moving trajectory of the movable device; According to each of the jump points, the combined navigation data is segmented to obtain each of the jump segment data and at least one non-jump segment data; wherein the non-jump segment data represents a trajectory segment in which no jump occurs in the initial moving trajectory of the movable device.

8. The method according to claim 7, characterized in that According to each of the jump points, the integrated navigation data is segmented to obtain each of the jump segment data and at least one non-jump segment data, including: According to each of the jump points, the combined navigation data is segmented to obtain at least one original jump segment data and each of the non-jump segment data corresponding to the combined navigation data; wherein the original jump segment data represents the original trajectory segment where the trajectory jump occurs in the initial moving trajectory of the movable device within a preset time period; The original jump segment data is expanded according to the neighbor segment data of the original jump segment data to obtain the jump segment data; wherein the neighbor segment data is the non-jump segment data whose collection time is adjacent to the collection time of the original jump segment data.

9. A trajectory generating device, characterized in that: include: A determination module, used to obtain work order data of a movable device within a preset time period, and determine at least one jump segment data in the work order data; wherein the work order data represents an initial movement trajectory of the movable device within the preset time period; and the jump segment data represents a trajectory segment where a trajectory jump occurs in the initial movement trajectory of the movable device within the preset time period; A processing module, used to perform posture estimation processing on the jump segment data to obtain visual odometer VO segment data corresponding to the jump segment data; wherein the VO segment data represents the moving position and moving direction of the movable device in three-dimensional space; A generation module is used to generate fused trajectory data of the movable device according to each of the VO segment data and the work order data; wherein the fused trajectory data represents the final movement trajectory of the movable device within the preset time period; and the fused trajectory data is used to generate a map.

10. A computing device, characterized in that include: Memory, processor; The memory stores computer-executable instructions; The processor executes the computer-executable instructions stored in the memory, so that the processor performs the method according to any one of claims 1 to 8.

11. A computer-readable storage medium / computer program product, characterized in that: The computer-readable storage medium stores computer-executable instructions, which, when executed by a processor, are used to implement the method according to any one of claims 1 to 8; and / or, the computer program product includes a computer program, which, when executed by a processor, implements the method according to any one of claims 1 to 8.