Multi-source fusion positioning processing method and apparatus for vehicle terminal, and device

Abstract

The present application relates to autonomous driving technology. Provided are a multi-source fusion positioning processing method and apparatus for a vehicle terminal, and a device. The method comprises: during a traveling process of a vehicle terminal, acquiring inertial measurement data, wheel speed data, laser point cloud data and other source data of the vehicle terminal in a current frame; performing pre-integration on the inertial measurement data and the wheel speed data so as to generate a pre-integration result; performing feature matching on the laser point cloud data of the current frame and laser point cloud data of the frame previous to the current frame, and if it is determined that the current frame is a key frame, determining a point cloud of the key frame; fusing the pre-integration result, the point cloud of the key frame, and the other source data so as to generate the current pose of the vehicle terminal in the key frame; and on the basis of the current pose in the key frame, the point cloud of the key frame, point clouds of historical key frames, and a preset point cloud base map, determining a target pose of the vehicle terminal. The method of the present application solves the technical problems of low positioning accuracy and poor robustness of a vehicle terminal.

Description

Multi-source fusion positioning processing method, device and equipment of vehicle terminal

[0001] The present application claims priority to the Chinese patent application No. 202411378371.3, filed on September 29, 2024, and entitled "Multi-source fusion positioning processing method, device and equipment of vehicle terminal", the whole content of which is incorporated herein by reference. TECHNICAL FIELD

[0002] The present application relates to the automatic driving technology, and in particular to a multi-source fusion positioning processing method, device and equipment of vehicle terminal. BACKGROUND

[0003] At present, in the automatic driving technology, positioning refers to determining the current position and attitude of the vehicle. Positioning is a crucial link in the automatic driving system, and the main purpose is to determine the accurate pose of the vehicle in the global or local map, so as to make key decisions such as navigation, path planning, obstacle avoidance, etc.

[0004] In the prior art, multiple sensor data are usually used for positioning, including lidar, inertial measurement unit and wheel odometry. First, the relative poses of lidar, IMU and wheel odometry and the robot positioning origin are calibrated. Then the weight proportion of each sensor data in the fusion process is set. Next, the lidar data is used for global search or other methods to obtain an initial pose estimate. Then the IMU data is read and the position is predicted by the error state extended Kalman filter. Subsequently, the wheel odometry data is used to update the position by the error state extended Kalman filter. Finally, the previous pose estimation result is combined with the lidar data, and the iterative closest point matching or other matching method is used to update the position.

[0005] However, in the prior art, the positioning accuracy is relatively low due to the use of filtering scheme, and if the data of one of the sources fails, the error of the positioning result will be large, i.e. the robustness is low. SUMMARY

[0006] The present application provides a multi-source fusion positioning processing method, device and equipment of vehicle terminal to solve the technical problem of low positioning accuracy and poor robustness of vehicle terminal.

[0007] In a first aspect, the present application provides a multi-source fusion positioning processing method of vehicle terminal, comprising:

[0008] During the driving process of the vehicle terminal, inertial measurement data, wheel speed data, laser point cloud data and other source data of the vehicle terminal in the current frame are acquired;

[0009] pre-integrating the inertial measurement data and the wheel speed data to generate a pre-integration result;

[0010] performing feature matching on the laser point cloud data of the current frame and the laser point cloud data of a previous frame of the current frame, and determining a point cloud of the key frame if it is determined that the current frame is a key frame; wherein, there are a plurality of historical point clouds of key frames before the point cloud of the current frame;

[0011] fusing the pre-integration result, the point cloud of the key frame, and other source data to generate a current pose of the vehicle terminal at the key frame; wherein, the current pose includes a position and an attitude of the vehicle terminal;

[0012] determining a target pose of the vehicle terminal according to the current pose of the key frame, the point cloud of the key frame, the historical point cloud of the key frame, and a preset point cloud base map.

[0013] Further, performing feature matching on the laser point cloud data of the current frame and the laser point cloud data of a previous frame of the current frame, and determining a point cloud of the key frame if it is determined that the current frame is a key frame, includes:

[0014] performing feature matching on a surface feature in the laser point cloud data of the current frame and a surface feature in the laser point cloud data of a previous frame of the current frame to generate matching result information;

[0015] if it is determined that a confidence of the matching result information is greater than a preset confidence threshold, then determining that the current frame is a key frame and determining the point cloud of the key frame.

[0016] Further, the other source data includes global satellite navigation data;

[0017] fusing the pre-integration result, the point cloud of the key frame, and other source data to generate a current pose of the vehicle terminal at the key frame, includes:

[0018] fusing the pre-integration result and the point cloud of the key frame to generate an initial pose of the vehicle terminal at the current frame;

[0019] determining a timestamp of the key frame and searching for global satellite navigation data at the timestamp;

[0020] generating a current pose of the vehicle terminal at the current frame according to the initial pose and the global satellite navigation data.

[0021] Further, determining a target pose of the vehicle terminal according to the current pose of the key frame, the point cloud of the key frame, the historical point cloud of the key frame, and a preset point cloud base map, includes:

[0022] stitching the point cloud of the historical key frame to obtain a stitched point cloud;

[0023] loading, according to the current pose of the key frame, a local point cloud within a preset range of the current pose in a preset point cloud base map;

[0024] merging the stitched point cloud and the local point cloud to obtain a target point cloud;

[0025] performing point cloud matching on the point cloud of the key frame and the target point cloud to generate a matching score;

[0026] If it is determined that the matching score is greater than a preset score threshold, it is determined that the point cloud of the key frame is an absolute position constraint of the vehicle terminal; and a target pose of the vehicle terminal is determined according to the absolute position constraint of the vehicle terminal.

[0027] Further, the method further comprises:

[0028] inputting the target pose of the vehicle terminal into a preset pre-integration module, re-integrating, by the pre-integration module, data between a time when the current pose is generated and a current time when a pre-integration result is generated to determine a target pose of the vehicle terminal after re-integration; wherein the data includes inertial measurement data, wheel speed data, laser point cloud data and other source data.

[0029] Further, the method further comprises:

[0030] updating the point cloud of the key frame into the point cloud base map.

[0031] Further, the method further comprises:

[0032] performing smoothing processing on the target pose by a preset smoother based on historical motion information of the vehicle terminal and a preset vehicle kinematics constraint to obtain a smoothed target pose.

[0033] In a second aspect, the application provides a multi-source fusion positioning processing device of a vehicle terminal, comprising:

[0034] an acquisition module configured to acquire, in a driving process of a vehicle terminal, inertial measurement data, wheel speed data, laser point cloud data and other source data of the vehicle terminal at a current frame;

[0035] a first generation module configured to pre-integrate the inertial measurement data and the wheel speed data to generate a pre-integration result;

[0036] The first determining module is configured to perform feature matching on the laser point cloud data of the current frame and the laser point cloud data of the previous frame of the current frame, determine a key frame, and determine a point cloud of the key frame; wherein, there are a plurality of historical point clouds of key frames before the point cloud of the current frame;

[0037] The second generating module is configured to fuse the pre-integration result, the point cloud of the key frame, and other source data to generate a current pose of the vehicle terminal at the key frame; wherein, the current pose comprises a position and an attitude of the vehicle terminal;

[0038] The second determining module is configured to determine a target pose of the vehicle terminal according to the current pose of the key frame, the point cloud of the key frame, the historical point cloud of the key frame, and a preset point cloud base map.

[0039] Further, the first determining module comprises:

[0040] The first matching unit is configured to perform feature matching on a surface feature in the laser point cloud data of the current frame and a surface feature in the laser point cloud data of the previous frame of the current frame to generate matching result information;

[0041] The first determining unit is configured to determine the current frame as a key frame and determine a point cloud of the key frame if it is determined that a confidence of the matching result information is greater than a preset confidence threshold.

[0042] Further, the other source data comprises global satellite navigation data; and the second generating module comprises:

[0043] The first generating unit is configured to fuse the pre-integration result and the point cloud of the key frame to generate an initial pose of the vehicle terminal at the current frame;

[0044] The searching unit is configured to determine a timestamp of the key frame and search for global satellite navigation data at the timestamp;

[0045] The second generating unit is configured to fuse the initial pose and the global satellite navigation data to generate a current pose of the vehicle terminal at the current frame.

[0046] Further, the second determining module comprises:

[0047] The splicing unit is configured to perform splicing processing on the historical point cloud of the key frame to obtain a spliced point cloud;

[0048] The loading unit is configured to load a local point cloud within a preset range of the current pose in a preset point cloud base map according to the current pose of the key frame;

[0049] The merging unit is configured to merge the spliced point cloud and the local point cloud to obtain a target point cloud.

[0050] The second matching unit is configured to perform point cloud matching on the point cloud of the key frame and the target point cloud to generate a matching score.

[0051] The second determining unit is configured to determine that the point cloud of the key frame is an absolute position constraint of the vehicle terminal if it is determined that the matching score is greater than a preset score threshold, and determine a target pose of the vehicle terminal according to the absolute position constraint of the vehicle terminal.

[0052] Further, the apparatus is further specifically configured to:

[0053] The target pose of the vehicle terminal is input to a preset pre-integration module, and data between a time when the current pose is generated and a current time when a pre-integration result is generated is re-integrated by the pre-integration module to determine a re-integrated target pose of the vehicle terminal, wherein the data includes the inertial measurement data, the wheel speed data, the laser point cloud data, and the other source data.

[0054] Further, the apparatus further includes:

[0055] The updating unit is configured to update the point cloud of the key frame to the point cloud base map.

[0056] Further, the apparatus further includes:

[0057] The smoothing module is configured to perform smoothing processing on the target pose by a preset smoother based on historical motion information of the vehicle terminal and a preset vehicle kinematics constraint to obtain a smoothed target pose.

[0058] In a third aspect, the present application provides an electronic device, including a memory and a processor, the memory stores a computer program which can be run on the processor, and the processor implements the method of the first aspect when executing the computer program.

[0059] In a fourth aspect, the present application provides a computer readable storage medium, the computer readable storage medium stores computer execution instructions, and the computer execution instructions are executed by a processor to implement the method of the first aspect.

[0060] In a fifth aspect, the present application provides a computer program product, including a computer program, and the computer program is executed by a processor to implement the method of the first aspect.

[0061] The application provides a multi-source fusion positioning processing method, device and equipment of a vehicle terminal. In the driving process of the vehicle terminal, inertial measurement data, wheel speed data, laser point cloud data and other source data of the vehicle terminal in a current frame are acquired. The inertial measurement data and the wheel speed data are pre-integrated to generate a pre-integration result. The laser point cloud data of the current frame is subjected to feature matching with the laser point cloud data of a previous frame of the current frame. If it is determined that the current frame is a key frame, the point cloud of the key frame is determined. The point cloud of the key frame is present before the point cloud of the current frame. The pre-integration result, the point cloud of the key frame and the other source data are fused to generate a current pose of the vehicle terminal in the key frame. The current pose includes the position and the attitude of the vehicle terminal. According to the current pose of the key frame, the point cloud of the key frame, the point cloud of a historical key frame and a preset point cloud bottom map, a target pose of the vehicle terminal is determined. In the scheme, the inertial measurement data, the wheel speed data, the laser point cloud data and other source data are subjected to multi-sensor fusion positioning through graph optimization, so that a target pose with high reliability is obtained. The scheme has the advantages of high positioning accuracy, strong robustness and smooth trajectory, and solves the technical problems of low positioning accuracy and poor robustness of the vehicle terminal. BRIEF DESCRIPTION OF DRAWINGS

[0062] The accompanying drawings, which are incorporated herein and form a part of the specification, illustrate embodiments consistent with the present disclosure and, together with the description, further serve to explain the principles of the disclosure.

[0063] FIG. 1 is a flow diagram of a multi-source fusion positioning processing method of a vehicle terminal according to an embodiment of the application;

[0064] FIG. 2 is a flow diagram of another multi-source fusion positioning processing method of a vehicle terminal according to an embodiment of the application;

[0065] FIG. 3 is an architecture diagram of a multi-source fusion positioning processing method of a vehicle terminal according to an embodiment of the application;

[0066] FIG. 4 is a structural diagram of a multi-source fusion positioning processing device of a vehicle terminal according to an embodiment of the application;

[0067] FIG. 5 is a structural diagram of another multi-source fusion positioning processing device of a vehicle terminal according to an embodiment of the application;

[0068] FIG. 6 is a structural diagram of an electronic device according to an embodiment of the application;

[0069] FIG. 7 is a block diagram of an electronic device according to an embodiment of the application.

[0070] The present disclosure has been shown and described with reference to the preferred embodiments. Equivalent mechanisms of the present disclosure can be used in place of or in addition to those illustrated and described herein. It is contemplated that the application described herein can be practiced with modification and alteration, and that the application be limited only by the scope of the appended claims and the reasonable interpretation therefrom. DETAILED DESCRIPTION

[0071] The exemplary embodiments will be described in detail herein with reference to the accompanying drawings. In the following description, like reference numerals refer to like elements, unless the context clearly dictates otherwise. The following exemplary embodiments are described with reference to the drawings, in which:

[0072] Currently, in the automatic driving technology, positioning refers to determining the current position and attitude of the vehicle. Positioning is a crucial link in the automatic driving system, and the main purpose is to determine the accurate pose of the vehicle in the global or local map, so as to make key decisions such as navigation, path planning, obstacle avoidance, etc. Multi-source fusion positioning refers to fusing multiple different types of sensor data to provide more accurate and reliable positioning results. Different types of sensors have their own advantages and limitations, and by fusing the data of multiple sensors, the shortcomings of each other can be made up, thereby improving the accuracy and accuracy of positioning. In addition, a single sensor may be disturbed or fail in a specific environment, resulting in inaccurate or unavailable positioning results. Through multi-source fusion positioning, data from different sensors can be compared and integrated to improve the robustness and reliability of the system.

[0073] In one example, multiple sensor data are generally used for positioning, including lidar, inertial measurement unit and wheel odometry. The method first calibrates the relative pose of the lidar, IMU and wheel odometry to the origin of the robot positioning. Then set the weight proportion of each sensor data in the fusion process. Next, use the lidar data to perform global search or other methods to obtain an initial pose estimate. Then read the IMU data and perform position prediction through the error state extended Kalman filter. Subsequently, use the wheel odometry data to perform position update through the error state extended Kalman filter. Finally, combine the previous pose estimation result with the lidar data, and use the iterative closest point matching or other matching method to perform position update. However, in the prior art, due to the use of filtering scheme, the positioning accuracy is relatively low, and the global positioning information cannot be directly obtained through the lidar, inertial measurement unit and wheel odometry, but only through matching with the global map to obtain the global positioning information. Once the point cloud matching is wrong or the prior map is updated, it may lead to completely wrong initialization positioning results.

[0074] In one example, when performing multi-source positioning on the vehicle terminal, first, the multi-dimensional state variable corresponding to the train to be estimated, the initial state prior information of the system and the prior factor are obtained, and a factor graph model is generated. When the inertial sensor IMU data is detected, the IMU pre-integration factor is added to the factor graph model. When the global satellite navigation system GNSS data is detected, the corresponding GNSS factor is added to the factor graph model. The earliest state variable in the factor graph model outside the sliding window length is marginalized and deleted, and a sliding window factor graph model is constructed. Then, based on the sliding window factor graph model, graph optimization is performed to calculate the positioning state information of the train to be estimated. The problem of this method is that when the GNSS data is invalid or incorrect, the use of IMU data alone for pose estimation will cause a large cumulative error in the positioning result for a period of time. In addition, this method also needs the prior information of the initial state of the system. Moreover, the trajectory obtained by fusing GNSS and IMU is not smooth.

[0075] Term explanation:

[0076] RTK: refers to real-time dynamic differential positioning, which is a high-precision GNSS (Global Navigation Satellite System) positioning technology. By transmitting differential data between a reference base station with known coordinates and a mobile station, real-time, centimeter-level positioning accuracy is achieved. RTK is used to obtain GNSS data.

[0077] LIO: refers to laser tube odometry, which refers to fusing laser point cloud and inertial navigation data for positioning.

[0078] The vehicle terminal multi-source fusion positioning processing method, device and equipment provided by the present application aim to solve the above technical problems of the prior art.

[0079] The technical solutions of the present application and how the technical solutions of the present application solve the above technical problems will be described in detail below with specific examples. The following specific examples can be combined with each other, and the same or similar concepts or processes may not be described again in some examples. The embodiments of the present application will be described below with reference to the accompanying drawings.

[0080] FIG. 1 is a flowchart of a vehicle terminal multi-source fusion positioning processing method according to an embodiment of the present application. As shown in FIG. 1, the method comprises:

[0081] Step 101, during the driving of the vehicle terminal, inertial measurement data, wheel speed data, laser point cloud data and other source data of the vehicle terminal in the current frame are obtained.

[0082] Exemplarily, the execution subject of the embodiment can be an electronic device, or a terminal device, or a multi-source fusion positioning processing device or equipment of a vehicle terminal, or other devices or equipment that can execute the embodiment, and no limitation is made thereto. In the embodiment, the execution subject is introduced as an electronic device, and the application scenario of the electronic device can be positioning processing of a vehicle terminal at a port, or positioning processing of a vehicle terminal at other locations, and no limitation is made thereto.

[0083] Firstly, in the driving process of the vehicle terminal, inertial measurement data, wheel speed data, laser point cloud data and other source data of the vehicle terminal in the current frame are acquired. The inertial measurement unit (IMU) is a device that measures the acceleration and angular velocity of an object by using sensors such as accelerometers and gyroscopes, and the inertial measurement data of the vehicle terminal in the current frame, i.e., IMU data, can be acquired by the inertial measurement unit (IMU). The wheel speed data is acquired by a tachometer, and the laser point cloud data is acquired by a laser radar, and the laser point cloud data refers to the point cloud data of the surrounding environment of the vehicle terminal; the other source data includes global satellite navigation data, and no limitation is made thereto, and the global satellite navigation data can be acquired by a preset global navigation satellite system (GNSS).

[0084] Step 102, pre-integrating the inertial measurement data and the wheel speed data to generate a pre-integration result.

[0085] Exemplarily, the inertial measurement data and the wheel speed data are pre-integrated first. The pre-integration provides real-time pose output as the value of the track deduction on one hand, and provides the relative relationship between the estimated states on the other hand, so as to perform pose fusion. The angular reliability of the IMU is relatively high, and the translational reliability of the pose calculated by the tachometer combined with the kinematic model is higher than the rotation, so the pre-integration needs to combine the data of the IMU and the tachometer.

[0086] Step 103, performing feature matching on the laser point cloud data of the current frame and the laser point cloud data of the last frame of the current frame, and if it is determined that the current frame is a key frame, determining the point cloud of the key frame; wherein, there are multiple historical point clouds of key frames before the point cloud of the current frame.

[0087] Exemplarily, the inter-frame point cloud matching of the laser can provide the relative pose between the two, and the point cloud matching needs a predicted pose as the initial value, i.e., the initial position, and the influence of the initial value is large. Using the IMU data and calculating the initial value according to the pose of the last frame is more robust than estimating the initial value in the uniform motion mode, so the laser odometry combines the IMU data.

[0088] Specifically, the laser point cloud data of the current frame is matched with the laser point cloud data of the previous frame of the current frame, and a maximum value is set for the number of feature points in the local area where each point cloud is located, so as to prevent the problem of long calculation time caused by too many feature points. Further, because the number of feature surfaces is much larger than the number of corner points in actual use, the role of corner points in point cloud matching is too small, and the scene point cloud matching of many corner points and few feature surfaces is also difficult to stabilize, and the use of IMU smoothing is better, so the feature points only include surface features and do not include corner points. Considering the computing power, no corner points are used in point cloud matching.

[0089] Specifically, the surface features of the laser point cloud data of the current frame are matched with the surface features of the laser point cloud data of the previous frame of the current frame to generate matching result information. After matching, it is determined whether the current frame is a key frame according to the matching result information. If it is determined that the current frame is a key frame, the point cloud of the key frame is determined, and there are multiple historical point clouds of key frames before the point cloud of the current frame. It should be noted that the point cloud matching of each frame of laser point cloud data will obtain the relative pose, but only the key frame will be provided as the source for pose fusion and subsequent laser positioning. The relative pose relationship between key frames is the result of superimposing multiple frames of relative poses.

[0090] Step 104, fusing the pre-integration result, the point cloud of the key frame and other source data to generate the current pose of the vehicle terminal at the key frame; wherein the current pose includes the position and attitude of the vehicle terminal.

[0091] Exemplarily, the pre-integration result and the point cloud of the key frame are fused to generate the initial pose of the vehicle terminal. If the other source data includes global satellite navigation data, the initial pose and the global satellite navigation data are fused to generate the current pose of the vehicle terminal at the key frame, wherein the current pose includes the position and attitude of the vehicle terminal.

[0092] Step 105, determining the target pose of the vehicle terminal according to the current pose of the key frame, the point cloud of the key frame, the point cloud of the historical key frame and the preset point cloud base map.

[0093] Exemplarily, for the high dynamic scene of the port, the laser positioning needs to update the point cloud base map in real time, and cannot directly use the offline map, so a point cloud base map needs to be established in advance, which includes static objects and dynamic objects. The static part is the containers, walls and ground in the scene, and the dynamic part is the moving objects that need to be updated in real time.

[0094] In this step, since the key frame has been screened, this part will only receive the point cloud of the key frame. When receiving the point cloud of the key frame, first, the point cloud of the historical key frame is spliced according to the current pose of the key frame to obtain the point cloud of the real-time part, that is, the spliced point cloud. Then, according to the predicted current pose of the key frame, the local point cloud within the preset range is loaded from the preset point cloud base map, the spliced point cloud and the local point cloud are merged to obtain the target point cloud, and the target point cloud is used as the target of positioning matching. The point cloud of the key frame is used as the matching source, and the point cloud matching is performed between the current point cloud of the key frame and the target point cloud to obtain the matching score. Whether the matching result is available is determined according to the matching score. If available, the point cloud of the key frame is used as an absolute pose constraint, and the target pose of the vehicle terminal is determined according to the absolute pose constraint. For example, the pre-built base map contains absolute pose information. If the point cloud of the current frame can be matched with the offline point cloud, it is known that the target pose of the current frame in the offline map.

[0095] In the embodiment of the application, in the driving process of the vehicle terminal, inertial measurement data, wheel speed data, laser point cloud data and other source data of the vehicle terminal at a current frame are obtained. The inertial measurement data and the wheel speed data are pre-integrated to generate a pre-integration result. The laser point cloud data of the current frame is matched with the laser point cloud data of the previous frame of the current frame, and if it is determined that the current frame is a key frame, the point cloud of the key frame is determined; wherein, there are multiple historical point clouds of key frames before the point cloud of the current frame. The pre-integration result, the point cloud of the key frame and other source data are fused to generate a current pose of the vehicle terminal at the key frame; wherein, the current pose includes the position and attitude of the vehicle terminal. According to the current pose of the key frame, the point cloud of the key frame, the historical point cloud of the key frame and the preset point cloud base map, the target pose of the vehicle terminal is determined. In the scheme, through graph optimization, multi-sensor fusion positioning is performed on inertial measurement data, wheel speed data, laser point cloud data and other source data, a target pose with high reliability is obtained, and the technical problems of low positioning accuracy and poor robustness of the vehicle terminal are solved.

[0096] FIG. 2 is a flowchart of another multi-source fusion positioning processing method of a vehicle terminal provided by an embodiment of the application, as shown in FIG. 2, the method comprises:

[0097] In the driving process of the vehicle terminal, inertial measurement data, wheel speed data, laser point cloud data and other source data of the vehicle terminal at a current frame are obtained.

[0098] By way of example, this step can refer to step 101 in FIG. 1, and will not be described again.

[0099] Step 202, pre-integrating the inertial measurement data and the wheel speed data to generate a pre-integrated result.

[0100] By way of example, this step can refer to step 102 in FIG. 1, and will not be described again.

[0101] Step 203, performing feature matching on the face features in the laser point cloud data of the current frame and the face features in the laser point cloud data of the previous frame of the current frame to generate matching result information.

[0102] By way of example, in the driving process of the vehicle terminal, the electronic device can collect the current frame in real time, and perform feature matching on the face features in the laser point cloud data of the current frame and the face features in the laser point cloud data of the previous frame of the current frame to generate matching result information. For example, if the current frame is the second frame, the face features in the laser point cloud data of the second frame are matched with the face features in the laser point cloud data of the first frame to generate matching result information. Similarly, the face features in the laser point cloud data of the third frame are matched with the face features in the laser point cloud data of the second frame to generate matching result information, and so on, until the collection of the laser point cloud data is stopped. Therefore, there are multiple historical key frame point clouds before the point cloud of the current frame.

[0103] Step 204, if it is determined that the confidence of the matching result information is greater than a preset confidence threshold, it is determined that the current frame is a key frame, and the point cloud of the key frame is determined; wherein there are multiple historical key frame point clouds before the point cloud of the current frame.

[0104] By way of example, the electronic device can determine whether the current frame is a key frame according to the matching result information, and if it is determined that the confidence of the matching result information is greater than a preset confidence threshold, it is determined that the current frame is a key frame, and the point cloud of the key frame is determined. Alternatively, if it is determined that the time difference and / or the moving distance between the adjacent next key frame and the previous key frame is greater than a preset threshold, it is determined that the current frame is a key frame, and the point cloud of the key frame is determined. The determination method is not limited.

[0105] Step 205, fusing the smoothed pre-integrated result and the point cloud of the key frame to generate an initial pose of the vehicle terminal at the current frame.

[0106] By way of example, the pose fusion of the electronic device uses a graph optimization framework with a sliding window to process data for the sliding window based on the key frame of the laser radar. When a key frame of the laser radar is added, the point cloud of the added key frame and the smoothed pre-integrated result are fused to create a state to be estimated by the vehicle terminal, i.e., the initial pose of the vehicle terminal at the key frame, which includes the position and the attitude.

[0107] Step 206, the other source data includes global satellite navigation data; determine the timestamp of the key frame, and find the global satellite navigation data under the timestamp.

[0108] Exemplarily, the other source data includes global satellite navigation data (i.e. gnss data) and the like, which is not limited. The electronic device can determine the timestamp of the key frame, find the global satellite navigation data under the same timestamp according to the timestamp of the key frame, and add the corresponding constraint after pose compensation. For example, when the nearest gnss data is found and the time difference between the gnss data and the time of the key frame is less than a certain value, the gnss data is determined as the global satellite navigation data under the same timestamp.

[0109] Step 207, fuse the initial pose and the global satellite navigation data to generate the current pose of the vehicle terminal in the current frame.

[0110] Exemplarily, the electronic device can compensate the gnss data according to the timestamp and the estimated speed, and appropriately expand the variance of the gnss data. Then the initial pose is further limited by the gnss data to generate the current pose of the vehicle terminal in the current frame. It should be noted that if the state between the key frames is considered to be small, the optimization is not meaningful, so the data with large time difference is not added.

[0111] Specifically, the optimization is for the state of all key frames in the window. The value provided by the positioning source corresponds to a calculation method of an error, and the state of the key frame is brought into the calculation to obtain the corresponding error value. Pose fusion needs to adjust the state of the key frame to minimize the sum of all errors. Since there are generally unreliable sources in multi-source fusion, the error corresponding to each constraint needs to be queried after the first optimization. If the error corresponding to the constraint is large, it is considered that the constraint is unreliable, and the error weight of the constraint needs to be reduced. Then secondary optimization is performed to reduce the influence of the error source as much as possible.

[0112] After optimization, the data carrying information with a timestamp less than the reference time is formed into an edge constraint by edge processing for the next optimization based on the time of the first frame of the sliding window. At the same time, the data with a timestamp less than the reference time will not be needed and can be deleted. In this way, the size of the calculation scale is ensured, and the problem of longer and longer calculation time due to larger and larger calculation scale will not occur. Therefore, the robustness is greatly improved, and even if the gnss data fails, high-precision positioning can be performed for a long time.

[0113] Step 208, splicing the point cloud of the historical key frame to obtain a spliced point cloud.

[0114] Exemplarily, the electronic device can perform splicing processing on the point cloud of the historical key frame to obtain a spliced point cloud.

[0115] In step 209, according to the current pose of the key frame, a local point cloud within a preset range of the current pose is loaded from a preset point cloud base map.

[0116] Exemplarily, the electronic device can load the local point cloud within the preset range from the preset point cloud base map according to the predicted current pose of the key frame.

[0117] In step 210, the spliced point cloud and the local point cloud are merged to obtain a target point cloud.

[0118] Exemplarily, the electronic device can merge the spliced point cloud and the local point cloud to obtain the target point cloud.

[0119] In step 211, the point cloud of the key frame and the target point cloud are matched to generate a matching score.

[0120] Exemplarily, the electronic device can perform point cloud matching on the point cloud of the key frame and the target point cloud to generate a matching score. Specifically, the point cloud matching on the point cloud of the key frame and the target point cloud can be matching on the surface features in the point cloud to generate the matching score.

[0121] In step 212, if it is determined that the matching score is greater than a preset score threshold, it is determined that the point cloud of the key frame is an absolute position constraint of the vehicle terminal, and a target pose of the vehicle terminal is determined according to the absolute position constraint of the vehicle terminal.

[0122] Exemplarily, the electronic device determines whether the matching score is greater than a preset score threshold. If it is determined that the matching score is greater than the preset score threshold, it is determined that the point cloud of the key frame is an absolute position constraint of the vehicle terminal, and a target pose of the vehicle terminal is determined according to the absolute position constraint of the vehicle terminal. Therefore, in the initialization process, on the one hand, the laser point cloud data of the key frame is matched with the point cloud base map, and on the other hand, the RTK global positioning information is directly obtained. By comparing the results of the two, a target pose with high reliability can be obtained.

[0123] In step 213, the target pose of the vehicle terminal is input to a preset pre-integration module, and data between the time when the current pose is generated and the current time when the pre-integration result is generated is re-integrated by the pre-integration module to determine a target pose of the vehicle terminal after re-integration. The data includes inertial measurement data, wheel speed data, laser point cloud data, and other source data.

[0124] Exemplarily, the pose fusion is to fuse information of all sources, which is time-lagged, and the target pose obtained in step 212 is the pose obtained according to the acquired data of each type, that is, the target pose obtained in step 212 is the pose at the time when the data of each type is acquired. There is a time difference between the time when the data is acquired and the time when the target pose is generated in step 212. In order to avoid time delay, the current data is used to perform track extrapolation by a preset pre-integration module, and the pose fusion is performed. When the pose fusion is completed and the target pose is obtained, the parameters of the track extrapolation are adjusted according to the result of the pose fusion, the data between the time when the current pose is fused and the current time is re-integrated, and the target pose at the current time is further obtained.

[0125] Specifically, the target pose of the vehicle terminal is input into a preset pre-integration module. In the pre-integration module, the data between the time when the current pose is generated and the current time when the pre-integration result is generated is re-integrated, and the target pose of the vehicle terminal after re-integration is determined. The target pose after re-integration is the real-time pose at the current time. The data includes inertial measurement data, wheel speed data, laser point cloud data and other source data.

[0126] Step 214: updating the point cloud of the key frame to the point cloud base map.

[0127] Exemplarily, whether the matching score is greater than the preset score threshold or not, the electronic device needs to update the key frame point cloud to the point cloud base map. When determining the current pose of the key frame, if the number of key frames is greater than a preset value, the key frame needs to be deleted on the edge, which indicates that the key frame has been optimized for multiple times and is the optimal estimation at present. In case of poor result, the part of the point cloud cannot be guaranteed to be consistent, and the point cloud matching cannot obtain a good result at this point, and naturally cannot provide absolute constraint to the pose fusion.

[0128] In addition, the perception features can also be acquired, and the vehicle terminal can be assisted in positioning according to the perception features. The perception features include visual lane lines, bridge crane detection, etc. The pictures can be collected by photographing through the camera, and the lane line detection is performed on the pictures. The position information of the bridge crane or the gantry crane can be obtained through the image or the laser radar point cloud information. Further, the visual lane line, the bridge crane detection and the like can provide additional information for the fusion positioning in some scenes, so that the vehicle can better perceive the surrounding environment, so as to improve the robustness of the fusion positioning, and to make the output result more stable and accurate.

[0129] Step 215: smoothing the target pose by a preset smoother based on the historical motion information of the vehicle terminal and the preset vehicle kinematics constraint, to obtain a smoothed target pose.

[0130] Exemplarily, due to the output result of graph optimization, i.e. the pose, small amplitude jumps that do not conform to the vehicle kinematics may occur due to the addition of new constraints, errors of sensor information, loop closing, etc. Directly passing through the pre-integration extrapolation output will cause the trajectory to be discontinuous, which will bring difficulties to the downstream control, and therefore the smoothing processing is needed.

[0131] Based on the historical motion information of the vehicle terminal and the preset vehicle kinematic constraint, the target pose of the pre-integration extrapolation is adjusted by the smoother, the difference with the original result is gradually reduced, and the occurrence of positioning jump is effectively avoided. At the same time, the smoother also monitors the difference between the target pose and the smoothed result. If the difference is too large or cannot converge for a long time, it is considered that the positioning result is abnormal, and a warning is sent to the monitor. Therefore, by smoothing the positioning result through the smoother, a smoother positioning trajectory can be obtained.

[0132] In the embodiments of the present application, during the driving process of the vehicle terminal, the inertial measurement data, wheel speed data, laser point cloud data and other source data of the vehicle terminal in the current frame are obtained. The inertial measurement data and wheel speed data are pre-integrated to generate a pre-integration result. The surface features in the laser point cloud data of the current frame are matched with the surface features in the laser point cloud data of the last frame of the current frame to generate matching result information. If it is determined that the confidence of the matching result information is greater than a preset confidence threshold, the current frame is determined as a key frame, and the point cloud of the key frame is determined. Wherein, there are multiple historical point clouds of key frames before the point cloud of the current frame. The pre-integrated result after smoothing and the point cloud of the key frame are fused to generate an initial pose of the vehicle terminal in the current frame. The other source data includes global satellite navigation data. The timestamp of the key frame is determined, and the global satellite navigation data at the timestamp is searched. The initial pose and the global satellite navigation data are fused to generate a current pose of the vehicle terminal in the current frame. The historical point clouds of the key frames are spliced to obtain a spliced point cloud. According to the current pose of the key frame, a local point cloud within a preset range of the current pose is loaded in a preset point cloud base map. The spliced point cloud and the local point cloud are merged to obtain a target point cloud. The point cloud of the key frame and the target point cloud are matched to generate a matching score. If it is determined that the matching score is greater than a preset score threshold, the point cloud of the key frame is determined as an absolute position constraint of the vehicle terminal. The target pose of the vehicle terminal is determined according to the absolute position constraint of the vehicle terminal. The target pose of the vehicle terminal is input into a preset pre-integration module, and the data between the time of generating the current pose and the current time of generating the pre-integration result is re-integrated through the pre-integration module to determine the target pose of the vehicle terminal after re-integration. Wherein, the data includes inertial measurement data, wheel speed data, laser point cloud data and other source data. The point cloud of the key frame is updated to the point cloud base map. Based on the historical motion information of the vehicle terminal and the preset vehicle kinematics constraint, the target pose is smoothed by the preset smoother to obtain a smoothed target pose. In the present scheme, through graph optimization, multi-sensor fusion positioning is performed on inertial measurement data, wheel speed data, laser point cloud data and other source data, and a target pose with high reliability is obtained, which has the advantages of high positioning accuracy, strong robustness and smooth trajectory, and solves the technical problems of low positioning accuracy and poor robustness of the vehicle terminal.

[0133] In one example, FIG. 3 is an architecture schematic diagram of a multi-source fusion positioning processing method of a vehicle terminal provided by the embodiments of the present application, as shown in FIG. 3, including: IMU data, wheel speed data, laser point cloud data, RTK, perception features, through a pre-integration module, an LIO module, a radar positioning module, a fusion positioning module, a smoother, a perception module, the above data is pre-integrated, LIO (i.e. determine pose), radar positioning, fusion positioning, smoothed output through the smoother. Wherein,

[0134] The pre-integration module is configured to pre-integrate the IMU data and the wheel speed data to generate a pre-integrated result, and further configured to re-integrate the target pose to obtain a re-integrated target pose.

[0135] The LIO module is configured to fuse the pre-integrated result and the laser point cloud data, so as to combine data of other sources in the pose fusion module subsequently.

[0136] The radar positioning module is configured to perform point cloud matching according to the point cloud of the key frame, the point cloud of the historical key frame and the point cloud map to obtain the target pose of the key frame.

[0137] The fusion positioning module is configured to combine the radar positioning result, other sources (RTK, perception information, pre-integrated result, etc.) to determine the target pose of the vehicle terminal. The other sources can be RTK, global satellite navigation data, etc.

[0138] The smoother module is configured to perform smoothing processing on the pre-integrated result.

[0139] The perception module is configured to obtain perception features, so as to further assist in positioning the vehicle terminal according to the perception features.

[0140] Therefore, the application fuses IMU data, wheel speed data, laser point cloud data, RTK and other information. The inertial measurement data and the wheel speed data are pre-integrated to obtain continuous relative pose information, i.e. the pre-integrated result. The pre-integrated result provides a pose initial value for the LIO and obtains a relative pose of the laser point cloud data after correction. The relative pose is the relative pose of the current key frame relative to the last historical key frame. The relative pose provides an initial value for the radar positioning, and on this basis, the point cloud map is matched to obtain an absolute pose based on the point cloud. The absolute pose is the current pose. When the absolute pose matches the pose under the RTK data, the point cloud map is updated online. Then, two kinds of relative poses and two kinds of absolute poses are added to the graph optimization framework for fusion to obtain a preliminary target pose. In order to further improve the accuracy and redundancy, perception feature information is added in some scenes. Finally, the fusion positioning module outputs feedback to adjust the real-time pose output of the pre-integration, i.e. the target pose after the pre-integration. After smoothing by the smoother, the final positioning output is obtained.

[0141] FIG. 4 is a structural schematic diagram of a multi-source fusion positioning processing device of a vehicle terminal according to an embodiment of the application. As shown in FIG. 4, the device comprises:

[0142] The acquisition module 31 is configured to acquire inertial measurement data, wheel speed data, laser point cloud data and other source data of the vehicle terminal in a current frame during driving of the vehicle terminal.

[0143] The first generation module 32 is used to pre-integrate the inertial measurement data and wheel speed data to generate pre-integration results.

[0144] The first determining module 33 is used to perform feature matching between the laser point cloud data of the current frame and the laser point cloud data of the previous frame to determine the key frame and the point cloud of the key frame; wherein, there are point clouds of multiple historical key frames before the point cloud of the current frame.

[0145] The second generation module 34 is used to fuse the pre-integration result, the point cloud of the key frame, and other source data to generate the current pose of the vehicle terminal in the key frame; wherein, the current pose includes the position and orientation of the vehicle terminal.

[0146] The second determining module 35 is used to determine the target pose of the vehicle terminal based on the current pose of the key frame, the point cloud of the key frame, the point cloud of the historical key frames, and the preset point cloud base map.

[0147] The apparatus in this embodiment can execute the technical solutions in the above method. Its specific implementation process and technical principles are the same, and will not be repeated here.

[0148] Figure 5 is a schematic diagram of another multi-source fusion positioning processing device for a vehicle terminal provided in an embodiment of this application. Based on the embodiment shown in Figure 4, as shown in Figure 5, the first determining module 33 includes:

[0149] The first matching unit 331 is used to perform feature matching between the surface features in the laser point cloud data of the current frame and the surface features in the laser point cloud data of the previous frame, and generate matching result information.

[0150] The first determining unit 332 is used to determine the current frame as a key frame and determine the point cloud of the key frame if the confidence level of the determined matching result information is greater than a preset confidence threshold.

[0151] In one example, other source data includes global satellite navigation data; the second generation module 34 includes:

[0152] The first generation unit 341 is used to fuse the pre-integration result and the point cloud of the key frame to generate the initial pose of the vehicle terminal in the current frame.

[0153] The lookup unit 342 is used to determine the timestamp of the key frame and to look up the global satellite navigation data under the timestamp.

[0154] The second generation unit 343 is used to fuse the initial pose and global satellite navigation data to generate the current pose of the vehicle terminal in the current frame.

[0155] In one example, the second determining module 35 includes:

[0156] The splicing unit 351 is configured to splice the point clouds of the historical key frames to obtain spliced point clouds.

[0157] The loading unit 352 is configured to load, according to the current pose of the key frame, a local point cloud in a preset range of the current pose in a preset point cloud base map.

[0158] The merging unit 353 is configured to merge the spliced point clouds and the local point cloud to obtain target point clouds.

[0159] The second matching unit 354 is configured to perform point cloud matching on the point clouds of the key frame and the target point clouds to generate a matching score.

[0160] The second determining unit 355 is configured to determine, if the matching score is greater than a preset score threshold, that the point clouds of the key frame are absolute position constraints of the vehicle terminal, and determine a target pose of the vehicle terminal according to the absolute position constraints of the vehicle terminal.

[0161] In one example, the apparatus is further configured to:

[0162] input the target pose of the vehicle terminal into a preset pre-integration module, re-integrate, through the pre-integration module, data between a time at which the current pose is generated and a current time at which a pre-integration result is generated to determine a re-integrated target pose of the vehicle terminal, wherein the data includes inertial measurement data, wheel speed data, laser point cloud data, and other source data.

[0163] In one example, the apparatus further includes:

[0164] The updating unit 356 is configured to update the point clouds of the key frame into the point cloud base map.

[0165] In one example, the apparatus further includes:

[0166] The smoothing module 41 is configured to perform smoothing processing on the target pose through a preset smoother based on historical motion information of the vehicle terminal and a preset vehicle kinematics constraint to obtain a smoothed target pose.

[0167] The apparatus of the embodiment can execute the technical solutions in the above method, and the specific implementation process and technical principles are the same, which will not be repeated here.

[0168] FIG. 6 is a structural schematic diagram of an electronic device provided by an embodiment of the present application. As shown in FIG. 6, the electronic device includes a memory 51 and a processor 52.

[0169] The memory 51 stores a computer program that can run on the processor 52.

[0170] The processor 52 is configured to execute the method provided by the above embodiment.

[0171] The electronic device further includes a receiver 53 and a transmitter 54. The receiver 53 is configured to receive instructions and data transmitted from external devices, and the transmitter 54 is configured to transmit instructions and data to external devices.

[0172] FIG. 7 is a block diagram of an electronic device provided by an embodiment of the present application. The electronic device can be a mobile phone, a computer, a digital broadcast terminal, a messaging device, a game console, a tablet device, a medical device, a fitness device, a personal digital assistant, etc.

[0173] The apparatus 600 can include one or more of the following components: a processing component 602, a memory 604, a power supply component 606, a multimedia component 608, an audio component 610, an input / output (I / O) interface 612, a sensor component 614, and a communication component 616.

[0174] The processing component 602 typically controls overall operations of the apparatus 600, such as operations associated with display, phone calls, data communications, camera operations, and recording operations. The processing component 602 can include one or more processors 620 to execute instructions to complete all or part of steps of the above-described methods. In addition, the processing component 602 can include one or more modules to facilitate interaction between the processing component 602 and other components. For example, the processing component 602 can include a multimedia module to facilitate the interaction between the multimedia component 608 and the processing component 602.

[0175] The memory 604 is configured to store various types of data to support operations of the apparatus 600. Examples of these data include instructions for any applications or methods operating on the apparatus 600, contact data, phonebook data, messages, pictures, videos, and so on. The memory 604 can be implemented by any type of volatile or non-volatile storage devices or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk, or optical disk.

[0176] The power supply component 606 supplies electrical power for the various components of the apparatus 600. The power supply component 606 can include a power supply management system, one or more power supplies, and other components associated with generating, managing, and distributing power for the apparatus 600.

[0177] The multimedia component 608 includes a screen providing an output interface between the device 600 and a user. In some embodiments, the screen includes a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen can be implemented as a touch screen to receive input signals from a user. The touch panel includes one or more touch sensors to sense touch, swiping, and gestures on the touch panel. The touch sensor can not only sense a boundary of a touching or swiping action, but also detect duration and pressure related to the touching or swiping action. In some embodiments, the multimedia component 608 includes a front camera and / or a rear camera. When the device 600 is in an operation mode, such as a shooting mode or a video mode, the front camera and / or the rear camera can receive external multimedia data. Each of the front and rear camera can be a fixed optical lens system or have a focal length and optical zooming capability.

[0178] The audio component 610 is configured to output and / or input audio signals. For example, the audio component 610 includes a microphone (MIC) configured to receive an external audio signal when the device 600 is in an operation mode, such as a call mode, a recording mode, and a voice recognition mode. The received audio signal can be further stored in the memory 604 or transmitted via the communication component 616. In some embodiments, the audio component 610 also includes a speaker for outputting audio signals.

[0179] The I / O interface 612 provides an interface between the processing component 602 and peripheral interface modules, which can be a keypad, a click wheel, buttons, and the like. The buttons can include, but are not limited to, a home button, a volume button, a start button, and a lock button.

[0180] The sensor component 614 includes one or more sensors to provide various state assessments for the device 600. For example, the sensor component 614 can detect an open / closed state of the device 600, relative positioning of components, such as a display and a keypad of the device 600, a change in position of the device 600 or a component of the device 600, presence or absence of user contact with the device 600, a change in orientation of the device 600 or acceleration / deceleration of the device 600, and a temperature change of the device 600. The sensor component 614 can include a proximity sensor configured to detect presence of a nearby object without any physical touch. The sensor component 614 can further include a light sensor, such as a CMOS or CCD image sensor, for use in an imaging application. In some embodiments, the sensor component 614 can further include an acceleration sensor, a gyroscope sensor, a magnetic sensor, a pressure sensor, or a temperature sensor.

[0181] The communication component 616 is configured to facilitate wired or wireless communication between the device 600 and other devices. The device 600 can access a wireless network based on a communication standard, such as WiFi, 2G, or 3G, or a combination thereof. In an exemplary embodiment, the communication component 616 receives a broadcast signal or broadcast related information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communication component 616 further includes a Near Field Communication (NFC) module to facilitate close proximity communication. For example, the NFC module can be implemented based on Radio Frequency Identification (RFID) technology, infrared data association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology and other technologies.

[0182] In an exemplary embodiment, the device 600 can be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, micro-controllers, microprocessors or other electronic components, for performing the above-described methods.

[0183] In an exemplary embodiment, a non-transitory computer readable storage medium including instructions, such as the memory 604 including instructions, is also provided, which can be executed by the processor 620 of the device 600 to complete the above-described methods. For example, the non-transitory computer readable storage medium can be a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disc, and an optical data storage device, etc.

[0184] The embodiments of the present application also provide a non-transitory computer readable storage medium, when instructions in the storage medium are executed by a processor of an electronic device, the electronic device can execute the method provided by the above-described embodiments.

[0185] The embodiments of the present application also provide a computer program product, the computer program product includes: a computer program, the computer program is stored in a readable storage medium, at least one processor of an electronic device can read the computer program from the readable storage medium, and the at least one processor executes the computer program so that the electronic device executes the scheme provided by any of the above-described embodiments.

[0186] Other embodiments of the present disclosure will be apparent to those skilled in the art from consideration of the specification and practice of the application disclosed herein. It is intended that the present application cover any and all variations of the present disclosure including combinations of features falling within the general scope of the application. The specification and examples are to be considered exemplary only, with the true scope and spirit of the application indicated by the following claims.

[0187] It should be understood that the present disclosure is not limited to the precise construction that has been described above and shown in the accompanying drawings, and that various modifications and changes can be made by those skilled in the art without departing from the scope of the present disclosure. The scope of the present disclosure is limited only by the appended claims.

Claims

1. A multi-source fusion positioning processing method of a vehicle terminal, characterized in that, The method comprises the following steps: During the driving process of the vehicle terminal, inertial measurement data, wheel speed data, laser point cloud data and other source data of the vehicle terminal in a current frame are acquired; The inertial measurement data and the wheel speed data are pre-integrated to generate a pre-integration result; The laser point cloud data of the current frame is matched with the laser point cloud data of a previous frame of the current frame, and if it is determined that the current frame is a key frame, the point cloud of the key frame is determined; wherein, there are a plurality of historical key frame point clouds before the point cloud of the current frame; The pre-integration result, the point cloud of the key frame and other source data are fused to generate a current pose of the vehicle terminal at the key frame; wherein, the current pose comprises the position and attitude of the vehicle terminal; According to the current pose of the key frame, the point cloud of the key frame, the historical key frame point cloud and a preset point cloud base map, a target pose of the vehicle terminal is determined.

2. The method of claim 1, wherein, The laser point cloud data of the current frame is matched with the laser point cloud data of a previous frame of the current frame, and if it is determined that the current frame is a key frame, the point cloud of the key frame is determined, comprising: The surface features in the laser point cloud data of the current frame are matched with the surface features in the laser point cloud data of a previous frame of the current frame to generate matching result information; If it is determined that the confidence of the matching result information is greater than a preset confidence threshold, it is determined that the current frame is a key frame, and the point cloud of the key frame is determined.

3. The method of claim 2, wherein, The other source data comprises global satellite navigation data; The pre-integration result, the point cloud of the key frame and other source data are fused to generate a current pose of the vehicle terminal at the key frame, comprising: The pre-integration result and the point cloud of the key frame are fused to generate an initial pose of the vehicle terminal at the current frame; The timestamp of the key frame is determined, and the global satellite navigation data at the timestamp is searched; The initial pose and the global satellite navigation data are fused to generate a current pose of the vehicle terminal at the current frame.

4. The method of claim 3, wherein, The target pose of the vehicle terminal is determined according to the current pose of the key frame, the point cloud of the key frame, the historical key frame point cloud and a preset point cloud base map, comprising: The historical key frame point cloud is spliced to obtain a spliced point cloud; According to the current pose of the key frame, a local point cloud within a preset range of the current pose is loaded in a preset point cloud base map; The spliced point cloud and the local point cloud are merged to obtain a target point cloud; The point cloud matching of the point cloud of the key frame and the target point cloud is performed to generate a matching score; If it is determined that the matching score is greater than a preset score threshold, it is determined that the point cloud of the key frame is an absolute position constraint of the vehicle terminal; and the target pose of the vehicle terminal is determined according to the absolute position constraint of the vehicle terminal.

5. The method of claim 4, wherein, The method further comprises: input the target pose of the vehicle terminal into a preset pre-integration module, re-integrate data between a time when the current pose is generated and a current time when a pre-integration result is generated through the pre-integration module, and determine a re-integrated target pose of the vehicle terminal; wherein the data includes the inertial measurement data, the wheel speed data, the laser point cloud data, and the other source data.

6. The method of claim 4, wherein, The method further includes: updating the point cloud of the key frame into the point cloud base map.

7. The method according to any one of claims 1 to 6, characterized in that, The method further includes: performing smoothing processing on the target pose through a preset smoother based on historical motion information of the vehicle terminal and a preset vehicle kinematics constraint, and obtaining a smoothed target pose.

8. A multi-source fusion positioning processing device of a vehicle terminal, characterized in that, comprise: an acquisition module, configured to acquire inertial measurement data, wheel speed data, laser point cloud data, and other source data of a vehicle terminal in a current frame during driving of the vehicle terminal; a first generation module, configured to pre-integrate the inertial measurement data and the wheel speed data to generate a pre-integration result; a first determination module, configured to perform feature matching on the laser point cloud data of the current frame and laser point cloud data of a previous frame of the current frame, determine a key frame, and determine a point cloud of the key frame; wherein there are a plurality of historical point clouds of key frames before a point cloud of the current frame; a second generation module, configured to fuse the pre-integration result, the point cloud of the key frame, and the other source data to generate a current pose of the vehicle terminal in the key frame; wherein the current pose includes a position and an attitude of the vehicle terminal; a second determination module, configured to determine a target pose of the vehicle terminal according to the current pose of the key frame, the point cloud of the key frame, historical point clouds of the key frames, and a preset point cloud base map.

9. An electronic device, comprising: comprise a memory and a processor, the memory stores a computer program that can run on the processor, and the processor implements the method of any one of claims 1-7 when executing the computer program.

10. A computer-readable storage medium, characterized in that, The computer readable storage medium stores computer execution instructions, and the computer execution instructions are executed by the processor to implement the method of any one of claims 1-7.

Images