Multi-source fusion positioning processing method, device and equipment for vehicle terminal

Abstract

The invention provides a multi-source fusion positioning processing method, device and equipment for a vehicle terminal, and relates to an automatic driving technology, and the method comprises the steps: obtaining inertia measurement data, wheel speed data, laser point cloud data and other source data of the vehicle terminal in a current frame in a driving process of the vehicle terminal; performing pre-integration on the inertia measurement data and the wheel speed data to generate a pre-integration result; and performing feature matching on the laser point cloud data of the current frame and the laser point cloud data of the previous frame of the current frame, and if the current frame is determined as the key frame, determining the point cloud of the key frame. Fusing the pre-integration result, the point cloud of the key frame and other source data to generate a current pose of the vehicle terminal in the key frame; and determining a target pose of the vehicle terminal according to the current pose of the key frame, the point cloud of the key frame, the historical point cloud of the key frame and a preset point cloud base map. According to the method provided by the invention, the technical problems of low positioning precision and poor robustness of the vehicle terminal are solved.

Description

Technical Field

[0001] The present application relates to autonomous driving technology, and in particular to a multi-source fusion positioning processing method, device and equipment for a vehicle terminal. Background Art

[0002] Currently, in autonomous driving technology, positioning refers to determining the current position and posture of the vehicle. Positioning is a crucial link in the autonomous driving system. Its main purpose is to determine the accurate position and posture of the vehicle in the global or local map in order to make key decisions such as navigation, path planning, and obstacle avoidance.

[0003] In the prior art, positioning is usually performed using a variety of sensor data, including lidar, inertial measurement unit and wheel odometer. This method first calibrates the relative pose of the lidar, IMU and wheel odometer to the robot positioning origin. Then set the weight of each sensor data in the fusion process. Next, use the lidar data for global search or other methods to obtain the initial pose estimate. Then read the IMU data and predict the position through the error state extended Kalman filter. Then use the wheel odometer data to update the position through the error state extended Kalman filter. Finally, combine the previous pose estimation results with the lidar data, and use iterative closest point matching or other matching methods to update the position.

[0004] However, in the prior art, since filtering schemes are used for positioning, the positioning accuracy is relatively low, and if the data from one of the sources fails, the positioning result will have a large error, that is, the robustness is low. Summary of the invention

[0005] The present application provides a multi-source fusion positioning processing method, device and equipment for a vehicle terminal, which are used to solve the technical problems of low positioning accuracy and poor robustness of the vehicle terminal.

[0006] In a first aspect, the present application provides a multi-source fusion positioning processing method for a vehicle terminal, comprising:

[0007] During the driving process of the vehicle terminal, inertial measurement data, wheel speed data, laser point cloud data and other source data of the vehicle terminal in the current frame are obtained;

[0008] Pre-integrating the inertial measurement data and the wheel speed data to generate a pre-integration result;

[0009] Perform feature matching on the laser point cloud data of the current frame and the laser point cloud data of the previous frame of the current frame, and if the current frame is determined to be a key frame, determine the point cloud of the key frame; wherein there are multiple point clouds of historical key frames before the point cloud of the current frame;

[0010] Fusing the pre-integration result, the point cloud of the key frame and other source data to generate the current posture of the vehicle terminal in the key frame; wherein the current posture includes the position and posture of the vehicle terminal;

[0011] The target posture of the vehicle terminal is determined according to the current posture of the key frame, the point cloud of the key frame, the point cloud of the historical key frame and the preset point cloud base map.

[0012] Further, feature matching is performed on the laser point cloud data of the current frame and the laser point cloud data of the previous frame of the current frame. If the current frame is determined to be a key frame, the point cloud of the key frame is determined, including:

[0013] Perform feature matching on the surface features in the laser point cloud data of the current frame and the surface features in the laser point cloud data of the previous frame of the current frame to generate matching result information;

[0014] If it is determined that the confidence of the matching result information is greater than a preset confidence threshold, the current frame is determined to be a key frame, and a point cloud of the key frame is determined.

[0015] Further, the other source data includes global satellite navigation data;

[0016] The pre-integration result, the point cloud of the key frame and other source data are fused to generate the current posture of the vehicle terminal in the key frame, including:

[0017] Fusing the pre-integration result with the point cloud of the key frame to generate an initial posture of the vehicle terminal in the current frame;

[0018] Determine the timestamp of the key frame, and search for the global satellite navigation data at the timestamp;

[0019] Based on the initial posture and the global satellite navigation data, the current posture of the vehicle terminal in the current frame is generated.

[0020] Further, determining the target posture of the vehicle terminal according to the current posture of the key frame, the point cloud of the key frame, the point cloud of the historical key frame and the preset point cloud base map includes:

[0021] Performing stitching processing on the point clouds of the historical key frames to obtain a stitched point cloud;

[0022] According to the current posture of the key frame, loading a local point cloud within a preset range of the current posture into a preset point cloud base map;

[0023] Merging the spliced ​​point cloud and the local point cloud to obtain a target point cloud;

[0024] Performing point cloud matching on the key frame point cloud and the target point cloud to generate a matching score;

[0025] If it is determined that the matching score is greater than a preset score threshold, the point cloud of the key frame is determined as the absolute position constraint of the vehicle terminal; and based on the absolute position constraint of the vehicle terminal, the target posture of the vehicle terminal is determined.

[0026] Furthermore, the method further comprises:

[0027] The target posture of the vehicle terminal is input into a preset pre-integration module, and the pre-integration module re-integrates the data between the moment when the current posture is generated and the current moment when the pre-integration result is generated, so as to determine the re-integrated target posture of the vehicle terminal; wherein the data includes inertial measurement data, wheel speed data, laser point cloud data and other source data.

[0028] Furthermore, the method further comprises:

[0029] The point cloud of the key frame is updated to the point cloud base map shown.

[0030] Furthermore, the method further comprises:

[0031] Based on the historical motion information of the vehicle terminal and the preset vehicle kinematic constraints, the target posture is smoothed by a preset smoother to obtain a smoothed target posture.

[0032] In a second aspect, the present application provides a multi-source fusion positioning processing device for a vehicle terminal, comprising:

[0033] An acquisition module, used to acquire inertial measurement data, wheel speed data, laser point cloud data and other source data of the vehicle terminal in the current frame during the driving process of the vehicle terminal;

[0034] A first generating module, configured to pre-integrate the inertial measurement data and the wheel speed data to generate a pre-integration result;

[0035] A first determination module is used to perform feature matching on the laser point cloud data of the current frame and the laser point cloud data of the previous frame of the current frame, determine a key frame, and determine the point cloud of the key frame; wherein there are multiple historical point clouds of key frames before the point cloud of the current frame;

[0036] A second generation module is used to fuse the pre-integration result, the point cloud of the key frame and other source data to generate the current posture of the vehicle terminal in the key frame; wherein the current posture includes the position and posture of the vehicle terminal;

[0037] The second determination module is used to determine the target posture of the vehicle terminal according to the current posture of the key frame, the point cloud of the key frame, the point cloud of the historical key frame and the preset point cloud base map.

[0038] Furthermore, the first determining module includes:

[0039] A first matching unit is used to perform feature matching on the surface features in the laser point cloud data of the current frame and the surface features in the laser point cloud data of the previous frame of the current frame to generate matching result information;

[0040] The first determination unit is configured to determine that the current frame is a key frame and determine a point cloud of the key frame if the confidence level of the matching result information is greater than a preset confidence threshold.

[0041] Furthermore, the other source data includes global satellite navigation data; and the second generating module includes:

[0042] A first generating unit, configured to fuse the pre-integration result with the point cloud of the key frame to generate an initial posture of the vehicle terminal in a current frame;

[0043] A search unit, used to determine the timestamp of the key frame and search for the global satellite navigation data at the timestamp;

[0044] The second generating unit is used to fuse the initial posture and the global satellite navigation data to generate the current posture of the vehicle terminal in the current frame.

[0045] Furthermore, the second determining module includes:

[0046] A splicing unit, used for splicing the point clouds of the historical key frames to obtain a spliced ​​point cloud;

[0047] A loading unit, configured to load a local point cloud within a preset range of the current posture into a preset point cloud base map according to the current posture of the key frame;

[0048] A merging unit, used for merging the spliced ​​point cloud and the local point cloud to obtain a target point cloud;

[0049] A second matching unit, configured to perform point cloud matching between the point cloud of the key frame and the target point cloud to generate a matching score;

[0050] The second determination unit is used to determine the point cloud of the key frame as the absolute position constraint of the vehicle terminal if it is determined that the matching score is greater than a preset score threshold; and determine the target posture of the vehicle terminal based on the absolute position constraint of the vehicle terminal.

[0051] Furthermore, the device is also specifically used for:

[0052] The target posture of the vehicle terminal is input into a preset pre-integration module, and the pre-integration module is used to re-integrate the data between the moment when the current posture is generated and the current moment when the pre-integration result is generated, so as to determine the re-integrated target posture of the vehicle terminal; wherein the data includes the inertial measurement data, the wheel speed data, the laser point cloud data and the other source data.

[0053] Furthermore, the device also includes:

[0054] An updating unit is used to update the point cloud of the key frame to the point cloud base map shown.

[0055] Furthermore, the device also includes:

[0056] The smoothing module is used to smooth the target posture through a preset smoother based on the historical motion information of the vehicle terminal and the preset vehicle kinematic constraints to obtain a smoothed target posture.

[0057] In a third aspect, the present application provides an electronic device, comprising a memory and a processor, wherein the memory stores a computer program executable on the processor, and when the processor executes the computer program, the method described in the first aspect is implemented.

[0058] In a fourth aspect, the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores computer-executable instructions, and when the computer-executable instructions are executed by a processor, they are used to implement the method described in the first aspect.

[0059] In a fifth aspect, the present application provides a computer program product, including a computer program, which implements the method described in the first aspect when executed by a processor.

[0060] The present application provides a multi-source fusion positioning processing method, device and equipment for a vehicle terminal. During the driving process of the vehicle terminal, the inertial measurement data, wheel speed data, laser point cloud data and other source data of the vehicle terminal in the current frame are obtained. The inertial measurement data and wheel speed data are pre-integrated to generate a pre-integration result. The laser point cloud data of the current frame is feature matched with the laser point cloud data of the previous frame of the current frame. If the current frame is determined to be a key frame, the point cloud of the key frame is determined; wherein, there are multiple historical point clouds of key frames before the point cloud of the current frame. The pre-integration result, the point cloud of the key frame and other source data are fused to generate the current posture of the vehicle terminal in the key frame; wherein, the current posture includes the position and posture of the vehicle terminal. According to the current posture of the key frame, the point cloud of the key frame, the point cloud of the historical key frame and the preset point cloud base map, the target posture of the vehicle terminal is determined. In this solution, through graph optimization, multi-sensor fusion positioning is performed on various information such as inertial measurement data, wheel speed data, laser point cloud data and other source data to obtain a highly reliable target posture. It has the advantages of high positioning accuracy, strong robustness, and smooth trajectory, and solves the technical problems of low positioning accuracy and poor robustness of vehicle terminals. BRIEF DESCRIPTION OF THE DRAWINGS

[0061] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present disclosure and, together with the description, serve to explain the principles of the present disclosure.

[0062] Figure 1 A schematic diagram of a flow chart of a multi-source fusion positioning processing method for a vehicle terminal provided in an embodiment of the present application;

[0063] Figure 2 A flowchart of another multi-source fusion positioning processing method for a vehicle terminal provided in an embodiment of the present application;

[0064] Figure 3 A schematic diagram of the architecture of a multi-source fusion positioning processing method for a vehicle terminal provided in an embodiment of the present application;

[0065] Figure 4 A schematic diagram of the structure of a multi-source fusion positioning processing device for a vehicle terminal provided in an embodiment of the present application;

[0066] Figure 5 A schematic diagram of the structure of another multi-source fusion positioning processing device for a vehicle terminal provided in an embodiment of the present application;

[0067] Figure 6 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application;

[0068] Figure 7 A block diagram of an electronic device provided in an embodiment of the present application.

[0069] The above drawings show clear embodiments of the present disclosure, which will be described in more detail below. These drawings and text descriptions are not intended to limit the scope of the present disclosure in any way, but to illustrate the concepts of the present disclosure to those skilled in the art by referring to specific embodiments. DETAILED DESCRIPTION

[0070] Here, exemplary embodiments will be described in detail, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The implementations described in the following exemplary embodiments do not represent all implementations consistent with the present disclosure.

[0071] Currently, in autonomous driving technology, positioning refers to determining the current position and posture of the vehicle. Positioning is a crucial link in the autonomous driving system. Its main purpose is to determine the accurate position and posture of the vehicle in the global or local map in order to make key decisions such as navigation, path planning, and obstacle avoidance. Multi-source fusion positioning refers to the fusion of multiple different types of sensor data to provide more accurate and reliable positioning results. Different types of sensors have their own advantages and limitations. By fusing the data of multiple sensors, they can make up for each other's shortcomings, thereby improving the precision and accuracy of positioning. In addition, a single sensor may be interfered with or fail in a specific environment, resulting in inaccurate or unobtainable positioning results. Through multi-source fusion positioning, the robustness and reliability of the system can be improved by comparing and integrating data from different sensors.

[0072] In one example, positioning is usually performed using multiple sensor data, including lidar, inertial measurement unit, and wheel odometer. The method first calibrates the relative pose of the lidar, IMU, and wheel odometer to the robot positioning origin. Then, the weight of each sensor data in the fusion process is set. Next, the lidar data is used for global search or other methods to obtain the initial pose estimate. Then, the IMU data is read and the position is predicted through the error state extended Kalman filter. Then, the wheel odometer data is used to update the position through the error state extended Kalman filter. Finally, the previous pose estimation results are combined with the lidar data, and the position is updated by iterative nearest point matching or other matching methods. However, in the prior art, due to the use of filtering schemes, the positioning accuracy is relatively low, and the global positioning information cannot be directly obtained through the lidar, inertial measurement unit, and wheel odometer. The global positioning information can only be obtained by matching with the global map. Once an error occurs in the point cloud matching or the prior map is updated, it may lead to completely wrong initialization positioning results.

[0073] In one example, when multi-source positioning is performed on a vehicle terminal, the multidimensional state variables, system initial state prior information and prior factors corresponding to the train to be estimated are first obtained to generate a factor graph model. When the inertial sensor IMU data is detected, the IMU pre-integration factor is added to the factor graph model. When the global satellite navigation system GNSS data is detected, the corresponding GNSS factor is added to the factor graph model. And the earliest state variables in the factor graph model outside the sliding window length are marginalized and deleted, and a sliding window factor graph model is constructed. Then, the graph optimization solution is performed based on the sliding window factor graph model to calculate the positioning state information of the train to be estimated. The problem with this method is that when the GNSS data fails or is erroneous, simply using IMU data for pose estimation will result in a large cumulative error in the positioning result for a period of time. In addition, this method also requires prior information on the initial state of the system. Moreover, the trajectory obtained by only fusion positioning with GNSS and IMU is not smooth.

[0074] Terminology explanation:

[0075] RTK: refers to real-time kinematic differential positioning, which is a high-precision GNSS (Global Navigation Satellite System) positioning technology. It achieves real-time, centimeter-level positioning accuracy by transmitting differential data between a reference base station and a mobile station with known coordinates. RTK is used to obtain GNSS data.

[0076] LIO: refers to laser odometry, which refers to the fusion of laser point cloud and inertial navigation data for positioning.

[0077] The present application provides a multi-source fusion positioning processing method, device and equipment for a vehicle terminal, aiming to solve the above technical problems in the prior art.

[0078] The technical solution of the present application and how the technical solution of the present application solves the above-mentioned technical problems are described in detail below with specific embodiments. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments. The embodiments of the present application will be described below in conjunction with the accompanying drawings.

[0079] Figure 1 A flowchart of a multi-source fusion positioning processing method for a vehicle terminal provided in an embodiment of the present application is shown in FIG. Figure 1 As shown, the method includes:

[0080] Step 101: During the driving process of the vehicle terminal, inertial measurement data, wheel speed data, laser point cloud data and other source data of the vehicle terminal in the current frame are obtained.

[0081] Exemplarily, the execution subject of this embodiment may be an electronic device, or a terminal device, or a multi-source fusion positioning processing device or device of a vehicle terminal, or other devices or equipment that can execute this embodiment, without limitation. In this embodiment, the execution subject is introduced as an electronic device, and the application scenario of the electronic device may be positioning processing of a vehicle terminal at a port, or positioning processing of a vehicle terminal at other locations, without limitation.

[0082] First, during the driving process of the vehicle terminal, the inertial measurement data, wheel speed data, laser point cloud data and other source data of the vehicle terminal in the current frame are obtained. Among them, the inertial measurement unit (IMU) is a device that uses sensors such as accelerometers and gyroscopes to measure the acceleration and rotational angular velocity of an object. The inertial measurement unit (IMU) can be used to obtain the inertial measurement data of the vehicle terminal in the current frame, that is, IMU data. The wheel speed data is obtained through the speedometer; the laser point cloud data is obtained through the laser radar, and the laser point cloud data refers to the point cloud data of the surrounding environment of the vehicle terminal; other source data include global satellite navigation data, etc., which are not limited to this, and the global satellite navigation data can be obtained through the preset global navigation satellite system (GNSS).

[0083] Step 102: pre-integrate the inertial measurement data and the wheel speed data to generate a pre-integration result.

[0084] Exemplarily, the inertial measurement data and wheel speed data are first pre-integrated. Pre-integration provides real-time posture output as the value of track deduction on the one hand, and provides the relative relationship between the estimated states on the other hand for posture fusion. The angle reliability of IMU is relatively high, while the translation reliability of the posture calculated by the wheel speed meter combined with the kinematic model is higher than the rotation reliability, so pre-integration needs to combine the data of IMU and wheel speed meter.

[0085] Step 103, feature matching is performed on the laser point cloud data of the current frame and the laser point cloud data of the previous frame of the current frame. If the current frame is determined to be a key frame, the point cloud of the key frame is determined; wherein there are multiple historical point clouds of key frames before the point cloud of the current frame.

[0086] For example, the inter-frame point cloud matching of the laser can provide the relative pose between the two, while the point cloud matching requires a predicted pose as the initial value, that is, the initial position, and the initial value has a greater impact. Using IMU data and calculating the initial value based on the pose of the previous frame will be more robust than estimating the initial value based on uniform motion, so the laser odometer is combined with IMU data.

[0087] Specifically, feature matching is performed on the laser point cloud data of the current frame and the laser point cloud data of the previous frame. At the same time, a maximum value is set for the number of feature points in the local area of ​​each point cloud to prevent the problem of too many feature points causing long calculation time. Furthermore, because the number of feature surfaces is much larger than the number of corner points in actual use, the role of corner points in point cloud matching is too small, and point cloud matching in scenes with many corner points and few feature surfaces is difficult to stabilize. Using IMU smoothing will produce better results, so feature points only include surface features, not corner points. Considering the comprehensive computing power, corner points are not used in point cloud matching.

[0088] Specifically, feature matching is performed on the surface features of the laser point cloud data of the current frame and the surface features of the laser point cloud data of the previous frame of the current frame to generate matching result information. After matching, it is determined whether the current frame is a key frame based on the matching result information. If the current frame is determined to be a key frame, the point cloud of the key frame is determined, and there are multiple historical point clouds of key frames before the point cloud of the current frame. It should be noted that each frame of laser point cloud data will perform point cloud matching to obtain relative pose, but only key frames will be provided as sources for pose fusion and subsequent laser positioning. The relative pose relationship between key frames is the result of the superposition of multiple frames of relative poses.

[0089] Step 104: fuse the pre-integration result, the point cloud of the key frame and other source data to generate the current posture of the vehicle terminal in the key frame; wherein the current posture includes the position and posture of the vehicle terminal.

[0090] Exemplarily, the pre-integration result is fused with the point cloud of the key frame to generate the initial pose of the vehicle terminal. If other source data include global satellite navigation data, the initial pose is fused with the global satellite navigation data to generate the current pose of the vehicle terminal in the key frame, wherein the current pose includes the position and attitude of the vehicle terminal.

[0091] Step 105: Determine the target posture of the vehicle terminal according to the current posture of the key frame, the point cloud of the key frame, the point cloud of the historical key frame and the preset point cloud base map.

[0092] For example, for the high-dynamic scenes in ports, laser positioning needs to update the point cloud base map in real time, and offline maps cannot be used directly. Therefore, a point cloud base map needs to be pre-established first, which contains both static objects and dynamic objects. The static part is the containers, walls, and ground in the scene, while the dynamic part is the moving objects that need to be updated in real time.

[0093] In this step, since the keyframes have been filtered, this part will only receive the point cloud of the keyframes. After receiving the point cloud of the keyframe, the point cloud of the historical keyframes is first spliced ​​according to the current pose of the keyframe to obtain the point cloud of the real-time part, that is, the spliced ​​point cloud after splicing. Then, according to the current pose of the predicted keyframe, the local point cloud within the preset range is loaded from the preset point cloud base map, and the spliced ​​point cloud and the local point cloud are merged to obtain the target point cloud, which is used as the target of positioning matching. The point cloud of the keyframe is used as the matching source, and the point cloud of the current keyframe and the target point cloud are matched to obtain the matching score. The matching score is used to determine whether the result of this matching is available. If it is available, the point cloud of the keyframe is used as an absolute pose constraint, and the target pose of the vehicle terminal is determined based on the absolute pose constraint. For example, the pre-built base map contains absolute pose information. If the point cloud of the current frame can match the offline point cloud, the target pose of the current frame in the offline map can be known.

[0094] In an embodiment of the present application, during the driving process of the vehicle terminal, the inertial measurement data, wheel speed data, laser point cloud data and other source data of the vehicle terminal in the current frame are obtained. The inertial measurement data and wheel speed data are pre-integrated to generate a pre-integration result. The laser point cloud data of the current frame is feature matched with the laser point cloud data of the previous frame of the current frame. If the current frame is determined to be a key frame, the point cloud of the key frame is determined; wherein, there are multiple historical point clouds of key frames before the point cloud of the current frame. The pre-integration result, the point cloud of the key frame and other source data are fused to generate the current posture of the vehicle terminal in the key frame; wherein, the current posture includes the position and posture of the vehicle terminal. According to the current posture of the key frame, the point cloud of the key frame, the point cloud of the historical key frame and the preset point cloud base map, the target posture of the vehicle terminal is determined. In this solution, through graph optimization, multi-sensor fusion positioning is performed on various information such as inertial measurement data, wheel speed data, laser point cloud data and other source data to obtain a highly reliable target posture. It has the advantages of high positioning accuracy, strong robustness, and smooth trajectory, and solves the technical problems of low positioning accuracy and poor robustness of vehicle terminals.

[0095] Figure 2 A flowchart of another multi-source fusion positioning processing method for a vehicle terminal provided in an embodiment of the present application is shown as follows: Figure 2 As shown, the method includes:

[0096] Step 201: During the driving process of the vehicle terminal, inertial measurement data, wheel speed data, laser point cloud data and other source data of the vehicle terminal in the current frame are obtained.

[0097] For example, this step can be referred to Figure 1 Step 101 in the above is not described in detail.

[0098] Step 202: pre-integrate the inertial measurement data and the wheel speed data to generate a pre-integration result.

[0099] For example, this step can be referred to Figure 1 Step 102 in the above is not described in detail.

[0100] Step 203: perform feature matching on the surface features in the laser point cloud data of the current frame and the surface features in the laser point cloud data of the previous frame of the current frame to generate matching result information.

[0101] Exemplarily, during the driving process of the vehicle terminal, the electronic device can collect the current frame in real time, and perform feature matching on the surface features in the laser point cloud data of the current frame with the surface features in the laser point cloud data of the previous frame of the current frame, and generate matching result information. For example, if the current frame is the second frame, the surface features in the laser point cloud data of the second frame are feature matched with the surface features in the laser point cloud data of the first frame, and matching result information is generated. Similarly, the surface features in the laser point cloud data of the third frame are feature matched with the surface features in the laser point cloud data of the second frame, and matching result information is generated until the collection of laser point cloud data is stopped. Therefore, there are multiple historical key frame point clouds before the point cloud of the current frame.

[0102] Step 204: If it is determined that the confidence of the matching result information is greater than a preset confidence threshold, the current frame is determined to be a key frame, and the point cloud of the key frame is determined; wherein, there are multiple historical point clouds of key frames before the point cloud of the current frame.

[0103] For example, the electronic device can determine whether the current frame is a key frame based on the matching result information. If the confidence of the matching result information is determined to be greater than a preset confidence threshold, the current frame is determined to be a key frame, and the point cloud of the key frame is determined. Alternatively, if the time difference and / or movement distance between the next adjacent key frame and the previous key frame is determined to be greater than a preset threshold, the current frame is determined to be a key frame, and the point cloud of the key frame is determined. The judgment method is not limited.

[0104] Step 205: Fuse the smoothed pre-integration result with the point cloud of the key frame to generate the initial posture of the vehicle terminal in the current frame.

[0105] Exemplarily, the posture fusion of the electronic device uses a graph optimization framework with a sliding window, and processes the sliding window data based on the key frame of the laser radar. When a key frame of the laser radar is added, the point cloud of the added key frame and the smoothed pre-integration result are fused to create the state that the vehicle terminal needs to be estimated, that is, the initial posture of the vehicle terminal in the key frame, which includes position and posture.

[0106] Step 206: Other source data include global satellite navigation data; determine the timestamp of the key frame, and search for the global satellite navigation data under the timestamp.

[0107] Exemplarily, other source data include global satellite navigation data (i.e., GNSS data), etc., which are not limited. The electronic device can determine the timestamp of the key frame, and according to the timestamp of the key frame, search for the global satellite navigation data at the same timestamp, and add corresponding constraints after performing posture compensation. For example, when the nearest GNSS data is found, and the time difference between the GNSS data and the key frame is less than a certain value, the GNSS data is determined to be the global satellite navigation data at the same timestamp.

[0108] Step 207: Fuse the initial position and posture with the global satellite navigation data to generate the current position and posture of the vehicle terminal in the current frame.

[0109] For example, the electronic device can compensate the GNSS data according to the timestamp combined with the estimated speed, and appropriately expand the variance of the GNSS data. Then the initial posture is further limited by the GNSS data to generate the current posture of the vehicle terminal in the current frame. It should be noted that if the state between key frames is considered to have a small posture change, it is not meaningful to add optimization, so the data with a large time gap is not added.

[0110] Specifically, the optimization targets the states of all keyframes in the window. The value provided by the positioning source corresponds to a method of calculating the error. The state of the keyframe is brought into the calculation to obtain the corresponding error value. Posture fusion requires adjusting the state of the keyframe to minimize the sum of all errors. When multiple sources are fused, there are generally unreliable sources among the multiple sources. Therefore, after the initial optimization, it is necessary to query the error corresponding to each constraint. If the error corresponding to the constraint is large, the constraint is considered unreliable and the error weight of the constraint needs to be reduced. Then, a secondary optimization is performed to minimize the impact of the error source.

[0111] After the optimization is completed, the time of the first frame of the sliding window is used as the benchmark, and the information carried by the data with a timestamp less than the benchmark time is marginalized to form a marginalization constraint for the next optimization. At the same time, the data with a timestamp less than the benchmark time will no longer be needed and can be deleted. In this way, the size of the calculation scale is guaranteed, and the problem of increasing calculation time as the calculation scale increases will not occur. Therefore, the robustness is greatly improved, and even if the GNSS data fails, high-precision positioning can be performed for a long time.

[0112] Step 208: stitching the point clouds of the historical key frames to obtain a stitched point cloud.

[0113] Exemplarily, the electronic device may perform stitching processing on the point clouds of historical key frames to obtain a stitched point cloud.

[0114] Step 209: According to the current posture of the key frame, a local point cloud within a preset range of the current posture is loaded into a preset point cloud base map.

[0115] Exemplarily, the electronic device may load a local point cloud within a preset range from a preset point cloud base map according to the current pose of the predicted key frame.

[0116] Step 210: merge the spliced ​​point cloud and the local point cloud to obtain a target point cloud.

[0117] Exemplarily, the electronic device may merge the stitched point cloud and the local point cloud to obtain a target point cloud.

[0118] Step 211: perform point cloud matching on the key frame point cloud and the target point cloud to generate a matching score.

[0119] Exemplarily, the electronic device may perform point cloud matching on the key frame point cloud and the target point cloud to generate a matching score. Specifically, performing point cloud matching on the key frame point cloud and the target point cloud may be to match the surface features in the electronic cloud to generate a matching score.

[0120] Step 212: If it is determined that the matching score is greater than a preset score threshold, the point cloud of the key frame is determined as the absolute position constraint of the vehicle terminal; and based on the absolute position constraint of the vehicle terminal, the target posture of the vehicle terminal is determined.

[0121] Exemplarily, the electronic device determines whether the matching score is greater than a preset score threshold. If it is determined that the matching score is greater than the preset score threshold, the point cloud of the key frame is determined as the absolute position constraint of the vehicle terminal, and the target posture of the vehicle terminal is further determined based on the absolute position constraint of the vehicle terminal. Therefore, during the initialization process, on the one hand, the laser point cloud data of the key frame is matched with the point cloud base map, and on the other hand, the RTK global positioning information is directly obtained. By comparing the results of the two, a highly reliable target posture can be comprehensively obtained.

[0122] Step 213: input the target posture of the vehicle terminal into a preset pre-integration module, and use the pre-integration module to re-integrate the data between the time when the current posture is generated and the current time when the pre-integration result is generated, so as to determine the re-integrated target posture of the vehicle terminal; wherein the data includes inertial measurement data, wheel speed data, laser point cloud data and other source data.

[0123] Exemplarily, posture fusion is the fusion of information from all sources, which has a time lag. The target posture obtained in step 212 is the posture obtained based on the acquired data of various types, that is, the target posture obtained in step 212 is the posture at the time of acquiring various types of data. There is a time difference between the time of acquiring the data and the time of generating the target posture in step 212. In order to avoid time delay, it is necessary to use the current data for track deduction and posture fusion through a preset pre-integration module. When the posture fusion is completed and the target posture is obtained, the parameters of the track deduction are adjusted according to the result of the posture fusion, and the data between the time when the current posture is fused to the current time is re-integrated to further obtain the target posture at the current time.

[0124] Specifically, the target posture of the vehicle terminal is input into a preset pre-integration module, in which the data between the moment when the current posture is generated and the current moment when the pre-integration result is generated are re-integrated to determine the re-integrated target posture of the vehicle terminal, which is the real-time posture at the current moment. The data includes inertial measurement data, wheel speed data, laser point cloud data and other source data.

[0125] Step 214: Update the point cloud of the key frame to the shown point cloud base map.

[0126] Exemplarily, regardless of whether the matching score is greater than a preset score threshold, the electronic device needs to update the keyframe point cloud to the point cloud base map. In addition, when determining the current pose of the keyframe, if the number of keyframes is greater than a preset value, it is necessary to marginalize and delete the keyframe, indicating that the keyframe has been optimized multiple times and is the current optimal estimate. In the event that the result is poor, the consistency of this part of the point cloud cannot be guaranteed, and point cloud matching cannot produce a good result at this point, and naturally will not provide absolute constraints for pose fusion.

[0127] In addition, perception features can also be obtained to assist in positioning the vehicle terminal based on the perception features. Perception features include visual lane lines, bridge dragon detection, etc. You can take pictures with a camera and perform lane line detection on the pictures. Bridge dragon detection can obtain the location information of the bridge crane or gantry crane through images or lidar point cloud information. Then, based on visual lane lines, bridge dragon detection, etc., additional information can be provided for fusion positioning in some scenarios, so that the vehicle can better perceive the surrounding environment, so as to improve the robustness of fusion positioning and make its output more stable and accurate.

[0128] Step 215: Based on the historical motion information of the vehicle terminal and the preset vehicle kinematic constraints, the target posture is smoothed by a preset smoother to obtain a smoothed target posture.

[0129] For example, since the output result of graph optimization, i.e., the posture, may have small jumps that are inconsistent with the vehicle kinematics due to the addition of new constraints, errors in sensor information, loops, etc., directly outputting through pre-integration and extrapolation will result in discontinuous trajectories and bring difficulties to downstream control, so it needs to be smoothed.

[0130] Based on the historical motion information of the vehicle terminal and the preset vehicle kinematic constraints, the pre-integrated extrapolated target pose is adjusted through the smoother, gradually reducing the difference with the original result, effectively avoiding the occurrence of positioning jumps. At the same time, the smoother will also monitor the difference between the target pose and the smoothed result. If the difference is too large or cannot converge for a long time, the positioning result is considered abnormal and a warning is issued to the monitor. Therefore, by smoothing the positioning result through the smoother, a smoother positioning trajectory can be obtained.

[0131] In an embodiment of the present application, during the driving process of the vehicle terminal, the inertial measurement data, wheel speed data, laser point cloud data and other source data of the vehicle terminal in the current frame are obtained. The inertial measurement data and wheel speed data are pre-integrated to generate a pre-integration result. The surface features in the laser point cloud data of the current frame are feature matched with the surface features in the laser point cloud data of the previous frame of the current frame to generate matching result information. If the confidence of the matching result information is determined to be greater than a preset confidence threshold, the current frame is determined to be a key frame, and the point cloud of the key frame is determined; wherein, there are multiple historical key frame point clouds before the point cloud of the current frame. The smoothed pre-integration result and the point cloud of the key frame are fused to generate the initial posture of the vehicle terminal in the current frame. Other source data include global satellite navigation data; the timestamp of the key frame is determined, and the global satellite navigation data under the timestamp is searched. The initial posture and the global satellite navigation data are fused to generate the current posture of the vehicle terminal in the current frame. The point clouds of the historical key frames are spliced ​​to obtain a spliced ​​point cloud. According to the current posture of the key frame, load the local point cloud within the preset range of the current posture in the preset point cloud base map. Merge the spliced ​​point cloud and the local point cloud to obtain the target point cloud. Perform point cloud matching on the point cloud of the key frame and the target point cloud to generate a matching score. If it is determined that the matching score is greater than the preset score threshold, the point cloud of the key frame is determined to be the absolute position constraint of the vehicle terminal; and according to the absolute position constraint of the vehicle terminal, the target posture of the vehicle terminal is determined. The target posture of the vehicle terminal is input into the preset pre-integration module, and the data between the time when the current posture is generated and the current time when the pre-integration result is generated are re-integrated by the pre-integration module to determine the re-integrated target posture of the vehicle terminal; wherein the data includes inertial measurement data, wheel speed data, laser point cloud data and other source data. Update the point cloud of the key frame to the point cloud base map shown. Based on the historical motion information of the vehicle terminal and the preset vehicle kinematic constraints, the target posture is smoothed by a preset smoother to obtain the smoothed target posture. In this solution, through graph optimization, multi-sensor fusion positioning is performed on various information such as inertial measurement data, wheel speed data, laser point cloud data and other source data to obtain a highly reliable target posture. It has the advantages of high positioning accuracy, strong robustness, and smooth trajectory, and solves the technical problems of low positioning accuracy and poor robustness of vehicle terminals.

[0132] In one example, Figure 3 A schematic diagram of the architecture of a multi-source fusion positioning processing method for a vehicle terminal provided in an embodiment of the present application, such as Figure 3As shown, it includes: IMU data, wheel speed data, laser point cloud data, RTK, perception features, through the pre-integration module, LIO module, radar positioning module, fusion positioning module, smoother, perception module, the above data is pre-integrated, LIO (i.e., determining the posture), radar positioning, fusion positioning, and smoothed output through the smoother. Among them,

[0133] The pre-integration module is used to pre-integrate the IMU data and wheel speed data to generate a pre-integration result; further, it can also perform a re-integration process on the target posture to obtain the re-integrated target posture.

[0134] The LIO module is used to fuse the pre-integration results and laser point cloud data so that they can be combined with data from other sources in the subsequent pose fusion module.

[0135] The radar positioning module is used to perform point cloud matching based on the point cloud of the key frame, the point cloud of the historical key frame and the point cloud base map to obtain the target pose of the key frame.

[0136] The fusion positioning module is used to combine radar positioning results and other sources (RTK, perception information, pre-integration results, etc.) to determine the target posture of the vehicle terminal. Other sources can be RTK, global satellite navigation data, etc.

[0137] The smoother module is used to smooth the pre-integration results.

[0138] The perception module is used to obtain perception features to facilitate further auxiliary positioning of the vehicle terminal based on the perception features.

[0139] Therefore, this application integrates IMU data, wheel speed data, laser point cloud data, RTK and other information. Pre-integration of inertial measurement data and wheel speed data can obtain continuous relative pose information, that is, pre-integration results; the pre-integration results provide LIO with initial pose values ​​and obtain the relative pose after correction of laser point cloud data. The relative pose is the relative pose of the current key frame relative to the key frame of the previous history; the relative pose provides initial values ​​for radar positioning, and on this basis, it is matched with the point cloud base map to obtain the absolute pose based on the point cloud. The absolute pose is the current pose. When the absolute pose matches the pose under RTK data, the point cloud base map is updated online; then the two relative poses and the two absolute poses are added to the graph optimization framework for fusion to obtain the preliminary target pose. In order to further improve accuracy and redundancy, perception feature information will be added in some scenes; finally, the fusion positioning module output feedback adjusts the real-time pose output of the pre-integration, that is, the target pose after pre-integration, and the output is smoothed by the smoother to obtain the final positioning output.

[0140] Figure 4 A schematic diagram of the structure of a multi-source fusion positioning processing device for a vehicle terminal provided in an embodiment of the present application is shown in FIG. Figure 4 As shown, the device comprises:

[0141] The acquisition module 31 is used to acquire inertial measurement data, wheel speed data, laser point cloud data and other source data of the vehicle terminal in the current frame during the driving process of the vehicle terminal.

[0142] The first generating module 32 is used to pre-integrate the inertial measurement data and the wheel speed data to generate a pre-integration result.

[0143] The first determination module 33 is used to perform feature matching on the laser point cloud data of the current frame and the laser point cloud data of the previous frame of the current frame, determine the key frame, and determine the point cloud of the key frame; wherein there are multiple historical point clouds of key frames before the point cloud of the current frame.

[0144] The second generation module 34 is used to fuse the pre-integration results, the point cloud of the key frame and other source data to generate the current posture of the vehicle terminal in the key frame; wherein the current posture includes the position and posture of the vehicle terminal.

[0145] The second determination module 35 is used to determine the target posture of the vehicle terminal according to the current posture of the key frame, the point cloud of the key frame, the point cloud of the historical key frame and the preset point cloud base map.

[0146] The device of this embodiment can execute the technical solution in the above method. Its specific implementation process and technical principles are the same and will not be repeated here.

[0147] Figure 5 A schematic diagram of the structure of another multi-source fusion positioning processing device for a vehicle terminal provided in an embodiment of the present application, Figure 4 Based on the embodiment shown, Figure 5 As shown, the first determination module 33 includes:

[0148] The first matching unit 331 is used to perform feature matching on the surface features in the laser point cloud data of the current frame with the surface features in the laser point cloud data of the previous frame of the current frame, and generate matching result information.

[0149] The first determination unit 332 is configured to determine that the current frame is a key frame and determine a point cloud of the key frame if the confidence level of the matching result information is greater than a preset confidence threshold.

[0150] In one example, the other source data includes global satellite navigation data; the second generation module 34 includes:

[0151] The first generating unit 341 is used to fuse the pre-integration result and the point cloud of the key frame to generate the initial posture of the vehicle terminal in the current frame.

[0152] The search unit 342 is used to determine the timestamp of the key frame and search for the global satellite navigation data under the timestamp.

[0153] The second generating unit 343 is used to fuse the initial posture and the global satellite navigation data to generate the current posture of the vehicle terminal in the current frame.

[0154] In one example, the second determining module 35 includes:

[0155] The stitching unit 351 is used to stitch the point clouds of the historical key frames to obtain a stitched point cloud.

[0156] The loading unit 352 is used to load the local point cloud within a preset range of the current posture in the preset point cloud base map according to the current posture of the key frame.

[0157] The merging unit 353 is used to merge the spliced ​​point cloud and the local point cloud to obtain the target point cloud.

[0158] A second matching unit 354 is used to perform point cloud matching between the point cloud of the key frame and the target point cloud to generate a matching score;

[0159] The second determination unit 355 is used to determine the point cloud of the key frame as the absolute position constraint of the vehicle terminal if the matching score is greater than a preset score threshold; and determine the target posture of the vehicle terminal based on the absolute position constraint of the vehicle terminal.

[0160] In one example, the device is also specifically used for:

[0161] The target posture of the vehicle terminal is input into a preset pre-integration module, and the data between the moment when the current posture is generated and the current moment when the pre-integration result is generated are re-integrated through the pre-integration module to determine the re-integrated target posture of the vehicle terminal; wherein the data includes inertial measurement data, wheel speed data, laser point cloud data and other source data.

[0162] In one example, the device further includes:

[0163] The updating unit 356 is used to update the point cloud of the key frame to the point cloud base map shown.

[0164] In one example, the device further includes:

[0165] The smoothing module 41 is used to smooth the target posture through a preset smoother based on the historical motion information of the vehicle terminal and the preset vehicle kinematic constraints to obtain a smoothed target posture.

[0166] The device of this embodiment can execute the technical solution in the above method. Its specific implementation process and technical principles are the same and will not be repeated here.

[0167] Figure 6 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application is shown in FIG. Figure 6 As shown, the electronic device includes: a memory 51 and a processor 52.

[0168] The memory 51 stores a computer program that can be executed on the processor 52 .

[0169] The processor 52 is configured to execute the method provided in the above embodiment.

[0170] The electronic device further includes a receiver 53 and a transmitter 54. The receiver 53 is used to receive instructions and data sent by an external device, and the transmitter 54 is used to send instructions and data to the external device.

[0171] Figure 7 This is a block diagram of an electronic device provided in an embodiment of the present application. The electronic device may be a mobile phone, a computer, a digital broadcast terminal, a message transceiver, a game console, a tablet device, a medical device, a fitness device, a personal digital assistant, etc.

[0172] The device 600 may include one or more of the following components: a processing component 602 , a memory 604 , a power component 606 , a multimedia component 608 , an audio component 610 , an input / output (I / O) interface 612 , a sensor component 614 , and a communication component 616 .

[0173] The processing component 602 generally controls the overall operation of the device 600, such as operations associated with display, phone calls, data communications, camera operations, and recording operations. The processing component 602 may include one or more processors 620 to execute instructions to complete all or part of the steps of the above-mentioned method. In addition, the processing component 602 may include one or more modules to facilitate the interaction between the processing component 602 and other components. For example, the processing component 602 may include a multimedia module to facilitate the interaction between the multimedia component 608 and the processing component 602.

[0174] The memory 604 is configured to store various types of data to support operations on the device 600. Examples of such data include instructions for any application or method operating on the device 600, contact data, phone book data, messages, pictures, videos, etc. The memory 604 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk.

[0175] The power supply component 606 provides power to the various components of the device 600. The power supply component 606 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power for the device 600.

[0176] The multimedia component 608 includes a screen that provides an output interface between the device 600 and the user. In some embodiments, the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen may be implemented as a touch screen to receive input signals from the user. The touch panel includes one or more touch sensors to sense touch, slide, and gestures on the touch panel. The touch sensor may not only sense the boundaries of the touch or slide action, but also detect the duration and pressure associated with the touch or slide operation. In some embodiments, the multimedia component 608 includes a front camera and / or a rear camera. When the device 600 is in an operating mode, such as a shooting mode or a video mode, the front camera and / or the rear camera may receive external multimedia data. Each front camera and rear camera may be a fixed optical lens system or have a focal length and optical zoom capability.

[0177] The audio component 610 is configured to output and / or input audio signals. For example, the audio component 610 includes a microphone (MIC), and when the device 600 is in an operating mode, such as a call mode, a recording mode, and a speech recognition mode, the microphone is configured to receive an external audio signal. The received audio signal can be further stored in the memory 604 or sent via the communication component 616. In some embodiments, the audio component 610 also includes a speaker for outputting audio signals.

[0178] I / O interface 612 provides an interface between processing component 602 and peripheral interface modules, such as keyboards, click wheels, buttons, etc. These buttons may include but are not limited to: a home button, a volume button, a start button, and a lock button.

[0179] The sensor assembly 614 includes one or more sensors for providing various aspects of status assessment for the device 600. For example, the sensor assembly 614 can detect the open / closed state of the device 600, the relative positioning of components, such as the display and keypad of the device 600, the sensor assembly 614 can also detect the position change of the device 600 or a component of the device 600, the presence or absence of user contact with the device 600, the orientation or acceleration / deceleration of the device 600 and the temperature change of the device 600. The sensor assembly 614 can include a proximity sensor configured to detect the presence of nearby objects without any physical contact. The sensor assembly 614 can also include an optical sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, the sensor assembly 614 can also include an accelerometer, a gyroscope sensor, a magnetic sensor, a pressure sensor or a temperature sensor.

[0180] The communication component 616 is configured to facilitate wired or wireless communication between the device 600 and other devices. The device 600 can access a wireless network based on a communication standard, such as WiFi, 2G or 3G, or a combination thereof. In an exemplary embodiment, the communication component 616 receives a broadcast signal or broadcast-related information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communication component 616 also includes a near field communication (NFC) module to facilitate short-range communication. For example, the NFC module can be implemented based on radio frequency identification (RFID) technology, infrared data association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology and other technologies.

[0181] In an exemplary embodiment, the apparatus 600 may be implemented by one or more application specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors or other electronic components to perform the above method.

[0182] In an exemplary embodiment, a non-transitory computer-readable storage medium including instructions is also provided, such as a memory 604 including instructions, and the instructions can be executed by the processor 620 of the device 600 to perform the above method. For example, the non-transitory computer-readable storage medium can be a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, etc.

[0183] An embodiment of the present application also provides a non-temporary computer-readable storage medium. When the instructions in the storage medium are executed by a processor of an electronic device, the electronic device can execute the method provided by the above embodiment.

[0184] An embodiment of the present application also provides a computer program product, which includes: a computer program, which is stored in a readable storage medium, and at least one processor of an electronic device can read the computer program from the readable storage medium, and at least one processor executes the computer program so that the electronic device executes the solution provided by any of the above embodiments.

[0185] Those skilled in the art will readily appreciate other embodiments of the present disclosure after considering the specification and practicing the invention disclosed herein. This application is intended to cover any variations, uses or adaptations of the present disclosure that follow the general principles of the present disclosure and include common knowledge or customary techniques in the art that are not disclosed in the present disclosure. The specification and examples are intended to be exemplary only, and the true scope and spirit of the present disclosure are indicated by the following claims.

[0186] It should be understood that the present disclosure is not limited to the exact structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.

Claims

1. A multi-source fusion positioning processing method for a vehicle terminal, characterized in that: include: During the driving process of the vehicle terminal, inertial measurement data, wheel speed data, laser point cloud data and other source data of the vehicle terminal in the current frame are obtained; Pre-integrating the inertial measurement data and the wheel speed data to generate a pre-integration result; Perform feature matching on the laser point cloud data of the current frame and the laser point cloud data of the previous frame of the current frame, and if the current frame is determined to be a key frame, determine the point cloud of the key frame; wherein there are multiple point clouds of historical key frames before the point cloud of the current frame; Fusing the pre-integration result, the point cloud of the key frame and other source data to generate the current posture of the vehicle terminal in the key frame; wherein the current posture includes the position and posture of the vehicle terminal; The target posture of the vehicle terminal is determined according to the current posture of the key frame, the point cloud of the key frame, the point cloud of the historical key frame and the preset point cloud base map.

2. The method according to claim 1, characterized in that Performing feature matching on the laser point cloud data of the current frame and the laser point cloud data of the previous frame of the current frame, and if it is determined that the current frame is a key frame, determining the point cloud of the key frame, including: Perform feature matching on the surface features in the laser point cloud data of the current frame and the surface features in the laser point cloud data of the previous frame of the current frame to generate matching result information; If it is determined that the confidence of the matching result information is greater than a preset confidence threshold, the current frame is determined to be a key frame, and a point cloud of the key frame is determined.

3. The method according to claim 2, characterized in that The other source data include global satellite navigation data; The pre-integration result, the point cloud of the key frame and other source data are fused to generate the current posture of the vehicle terminal in the key frame, including: Fusing the pre-integration result with the point cloud of the key frame to generate an initial posture of the vehicle terminal in the current frame; Determine the timestamp of the key frame, and search for the global satellite navigation data at the timestamp; The initial posture and the global satellite navigation data are fused to generate the current posture of the vehicle terminal in the current frame.

4. The method according to claim 3, characterized in that Determining the target posture of the vehicle terminal according to the current posture of the key frame, the point cloud of the key frame, the point cloud of the historical key frame and the preset point cloud base map includes: Performing stitching processing on the point clouds of the historical key frames to obtain a stitched point cloud; According to the current posture of the key frame, loading a local point cloud within a preset range of the current posture into a preset point cloud base map; Merging the spliced ​​point cloud and the local point cloud to obtain a target point cloud; Performing point cloud matching on the key frame point cloud and the target point cloud to generate a matching score; If it is determined that the matching score is greater than a preset score threshold, the point cloud of the key frame is determined as the absolute position constraint of the vehicle terminal; and based on the absolute position constraint of the vehicle terminal, the target posture of the vehicle terminal is determined.

5. The method according to claim 4, characterized in that The method further comprises: The target posture of the vehicle terminal is input into a preset pre-integration module, and the pre-integration module is used to re-integrate the data between the moment when the current posture is generated and the current moment when the pre-integration result is generated, so as to determine the re-integrated target posture of the vehicle terminal; wherein the data includes the inertial measurement data, the wheel speed data, the laser point cloud data and the other source data.

6. The method according to claim 4, characterized in that The method further comprises: The point cloud of the key frame is updated to the point cloud base map shown.

7. The method according to any one of claims 1 to 6, characterized in that: The method further comprises: Based on the historical motion information of the vehicle terminal and the preset vehicle kinematic constraints, the target posture is smoothed by a preset smoother to obtain a smoothed target posture.

8. A multi-source fusion positioning processing device for a vehicle terminal, characterized in that: include: An acquisition module, used to acquire inertial measurement data, wheel speed data, laser point cloud data and other source data of the vehicle terminal in the current frame during the driving process of the vehicle terminal; A first generating module, configured to pre-integrate the inertial measurement data and the wheel speed data to generate a pre-integration result; A first determination module is used to perform feature matching on the laser point cloud data of the current frame and the laser point cloud data of the previous frame of the current frame, determine a key frame, and determine the point cloud of the key frame; wherein there are multiple historical point clouds of key frames before the point cloud of the current frame; A second generation module is used to fuse the pre-integration result, the point cloud of the key frame and other source data to generate the current posture of the vehicle terminal in the key frame; wherein the current posture includes the position and posture of the vehicle terminal; The second determination module is used to determine the target posture of the vehicle terminal according to the current posture of the key frame, the point cloud of the key frame, the point cloud of the historical key frame and the preset point cloud base map.

9. An electronic device, characterized in that: The method comprises a memory and a processor, wherein the memory stores a computer program executable on the processor, and the processor implements the method according to any one of claims 1 to 7 when executing the computer program.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-executable instructions, which are used to implement the method according to any one of claims 1 to 7 when executed by a processor.

Images