Multi-sensing data fusion positioning method and device and vehicle

Abstract

The invention provides a multi-sensor data fusion positioning method and device and a vehicle, and the method comprises the steps: determining the current confidence coefficient of a first pose of a target object based on the current driving information of the target object when the current positioning scene of the target object is a laser degradation scene; the first pose is a pose determined based on laser point cloud data collected by a laser detection device in the target object, and performing fusion processing on the first pose and the second pose of the target object at the current moment based on the current confidence of the first pose to obtain a target positioning result of the target object, the second pose is determined based on the sensing data except the laser point cloud data, so that the confidence coefficient of the first pose in the multi-sensing data fusion positioning process can be dynamically adjusted according to the driving information of the target object in a laser degradation scene such as a tunnel, and the positioning accuracy is improved. And the positioning precision of multi-sensing data fusion positioning can be effectively improved.

Description

Technical Field

[0001] The present application relates to the field of positioning technology, and in particular to a multi-sensor data fusion positioning method, device and vehicle. Background Art

[0002] Multi-sensor data fusion positioning systems such as SLAM (Simultaneous Localization and Mapping) complete positioning by fusing the laser point cloud collected by LiDAR with the data collected by IMU (Inertial Measurement Unit) and the data collected by wheel speed sensor (Odometry). It can significantly improve the accuracy and robustness of positioning and is therefore widely used.

[0003] Among them, lidar-based positioning and mapping are the core components of the multi-sensor data fusion positioning system. However, in laser-degraded scenarios such as tunnels, the positioning accuracy of lidar is greatly reduced, and thus the positioning accuracy cannot be guaranteed during the multi-sensor data fusion positioning process. Summary of the invention

[0004] In order to solve the above technical problems, the present application provides a multi-sensor data fusion positioning method, device and vehicle to solve the problem in the prior art that positioning accuracy cannot be guaranteed in laser degradation scenarios.

[0005] To achieve the above technical objectives, the present application provides the following technical solutions:

[0006] In a first aspect, an embodiment of the present specification provides a multi-sensor data fusion positioning method, including:

[0007] When the current positioning scene of the target object is a laser degradation scene, determining a current confidence of a first pose of the target object based on current driving information of the target object; the first pose is a pose determined based on laser point cloud data collected by a laser detection device in the target object;

[0008] Based on the current confidence of the first posture, the first posture and the second posture of the target object at the current moment are fused to obtain a target positioning result of the target object, wherein the second posture is determined based on the sensor data other than the laser point cloud data.

[0009] In one implementation, determining the current confidence of the first pose of the target object based on the current driving information of the target object includes:

[0010] Determining an initial confidence level of the first posture at a current moment based on current driving information of the target object;

[0011] The initial confidence of the first posture at the current moment is corrected based on the historical confidence of the first posture at the previous moment to obtain the current confidence of the first posture.

[0012] In one implementation, the current driving information includes the current driving angular velocity and the current driving speed of the target object;

[0013] The determining the initial confidence of the first posture at the current moment based on the current driving information of the target object includes:

[0014] determining a current driving mode of the target object based on the current driving angular velocity, the driving mode comprising straight driving and turning;

[0015] When the current driving mode is straight driving, determining an initial confidence of the first posture at the current moment based on a current driving speed of the target object and a first predetermined corresponding relationship between the driving speed of the target object and the confidence of the first posture;

[0016] When the current driving mode is turning, the initial confidence of the first posture at the current moment is determined based on the current driving angular velocity of the target object and a second predetermined corresponding relationship between the driving angular velocity of the target object and the confidence of the first posture.

[0017] In one embodiment, the first pose of the target object at the current moment is determined by the following method:

[0018] Performing data expansion processing on the laser point cloud data collected by the laser detection device at the current moment to obtain laser point cloud data after data expansion processing;

[0019] Based on the laser point cloud data after the data expansion processing, the first position of the target object at the current moment is determined.

[0020] In one embodiment, the step of performing data expansion processing on the laser point cloud data collected by the laser detection device at the current moment to obtain the laser point cloud data after data expansion processing includes:

[0021] Acquire target laser point cloud data, wherein the target laser point cloud data includes incremental data in the laser point cloud data collected by the laser detection device at a previous moment;

[0022] The target laser point cloud data is combined with the laser point cloud data collected by the laser detection device at the current moment to obtain the laser point cloud data after the data expansion processing.

[0023] In one implementation, determining the first position of the target object at the current moment based on the laser point cloud data after the data expansion processing includes:

[0024] Based on a target downsampling step length, downsampling the laser point cloud data after the data expansion process is performed to obtain current laser point cloud data; the target downsampling step length is smaller than a predetermined downsampling step length in a non-laser degradation scenario;

[0025] Based on the current laser point cloud data, determine the first position of the target object at the current moment.

[0026] In one embodiment, it further includes:

[0027] Acquire current point cloud distribution information; the current point cloud distribution information includes distribution information of laser point cloud data collected by the laser detection device at the current moment in each target direction;

[0028] Based on the current point cloud distribution information, it is determined whether the current positioning scene of the target object is a laser degraded scene.

[0029] In one implementation, determining whether the current positioning scene of the target object is a laser-degraded scene based on the current point cloud distribution information includes:

[0030] Matching the current point cloud distribution information with a plurality of predetermined scene models; the plurality of predetermined scene models correspond one-to-one to a plurality of laser degradation scenes; the predetermined scene models include thresholds of distribution information of the laser point cloud data in each of the target directions under the corresponding laser degradation scenes;

[0031] If there is a successfully matched predetermined scene model, determining the current positioning scene of the target object as a laser degradation scene;

[0032] If there is no successfully matched predetermined scene model, the current positioning scene of the target object is determined as a non-laser degraded scene.

[0033] In a second aspect, the embodiments of this specification provide a multi-sensor data fusion positioning device, including:

[0034] A first processing module, used to determine a current confidence of a first pose of the target object based on current driving information of the target object when the current positioning scene of the target object is a laser degradation scene; the first pose is a pose determined based on laser point cloud data collected by a laser detection device in the target object;

[0035] The second processing module is used to fuse the first pose and the second pose of the target object at the current moment based on the current confidence of the first pose to obtain the target positioning result of the target object, wherein the second pose is determined based on the sensor data other than the laser point cloud data.

[0036] In a third aspect, an embodiment of the present specification provides a vehicle, including a laser detection device and a multi-sensor data fusion positioning device as described above.

[0037] In a fourth aspect, an embodiment of the present specification provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the multi-sensor data fusion positioning method as described in any one of the above items is implemented.

[0038] In a fifth aspect, an embodiment of this specification provides a computer program product or a computer program, wherein the computer program product includes a computer program, and the computer program is stored in a computer-readable storage medium; the processor of the computer device reads the computer program from the computer-readable storage medium, and when the processor executes the computer program, it implements the multi-sensor data fusion positioning method as described in any one of the above items.

[0039] It can be seen from the above technical scheme that the embodiments of the present application provide a multi-sensor data fusion positioning method, device and vehicle. When the current positioning scene of the target object is a laser degradation scene, the method determines the current confidence of the first pose of the target object based on the current driving information of the target object. The first pose is a pose determined based on the laser point cloud data collected by the laser detection device in the target object. Based on the current confidence of the first pose, the first pose and the second pose of the target object at the current moment are fused to obtain the target positioning result of the target object. The second pose is determined based on sensor data other than laser point cloud data. Therefore, in laser degradation scenes such as tunnels, the confidence of the first pose in the multi-sensor data fusion positioning process can be dynamically adjusted according to the driving information of the target object, thereby effectively improving the positioning accuracy of the multi-sensor data fusion positioning. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying any creative work.

[0041] Figure 1A flowchart of a multi-sensor data fusion positioning method provided for an implementation manner of this specification.

[0042] Figure 2 A schematic diagram of the change trend of the R value corresponding to the first posture with the driving angular velocity of the target object provided in an implementation manner of this specification.

[0043] Figure 3 A schematic diagram of the structure of a multi-sensor data fusion positioning device provided for an implementation manner of this specification. DETAILED DESCRIPTION

[0044] Unless otherwise defined, the technical terms or scientific terms used in the embodiments of this specification shall have the common meanings understood by persons with ordinary skills in the field to which this specification belongs. The words "first", "second" and similar words used in the embodiments of this specification do not indicate any order, quantity or importance, but are only used to avoid confusion of constituent elements.

[0045] Unless the context requires otherwise, throughout the specification, "plurality" means "at least two", and "including" is interpreted as an open, inclusive meaning, that is, "including, but not limited to". In the description of the specification, the terms "one embodiment", "some embodiments", "exemplary embodiments", "examples", "specific examples" or "some examples" are intended to indicate that a particular feature, structure, material or characteristic associated with the embodiment or example is included in at least one embodiment or example of the specification. The schematic representation of the above terms does not necessarily refer to the same embodiment or example.

[0046] The following will be combined with the drawings in the embodiments of this specification to clearly and completely describe the technical solutions in the embodiments of this specification. Obviously, the described embodiments are only part of the embodiments of this specification, not all of the embodiments. Based on the embodiments in this specification, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this specification.

[0047] Overview

[0048] As described in the background technology, multi-sensor data fusion positioning systems such as SLAM complete positioning by fusing the laser point cloud collected by the lidar with the data collected by the IMU and the data collected by the wheel speed sensor (Odometry). It can significantly improve the accuracy and robustness of positioning and is therefore widely used.

[0049] Among them, laser radar-based positioning cannot directly obtain the absolute coordinates of the target object on the earth's surface like GPS (Global Positioning System), nor can it obtain the target object's posture data through integration like IMU and odometer. Instead, it is necessary to build a high-precision map of a certain area in advance to use the matching relationship between the point cloud data of the current frame and the prior features in the high-precision map to perform posture estimation.

[0050] However, in scenarios such as long tunnels, single-side walls, bridges, and special-shaped buildings, the laser point cloud data may fail unexpectedly, i.e., a laser degradation scenario is formed. In laser degradation scenarios, due to the limitations of the scene structure, the movement of the target object in some degrees of freedom cannot be estimated, which greatly reduces the positioning accuracy of the lidar. For example, in open areas such as squares and airports, even multi-line lasers can only collect point clouds on the ground. When only ground point clouds are used for matching and positioning, only the motion information of the target object in the three degrees of freedom of Z axis, roll (rotation angle) and pitch (pitch angle) can be obtained, but the motion information of the target object in the three degrees of freedom of X axis, Y axis and yaw (yaw angle) cannot be obtained, resulting in positioning failure. In addition, in the process of the target object moving along the tunnel, the laser point clouds collected at each time are the same, so the movement of the target object in the forward direction cannot be accurately estimated.

[0051] Therefore, when the positioning accuracy of the lidar is poor, the final positioning accuracy cannot be guaranteed during the multi-sensor data fusion positioning process.

[0052] In order to solve the problem that traditional methods cannot guarantee positioning accuracy in laser-degraded scenarios, in the technical solution of the present application, when the current positioning scenario of the target object is a laser-degraded scenario, the multi-sensor data fusion positioning method determines the current confidence of the first pose of the target object based on the current driving information of the target object. The first pose is a pose determined based on the laser point cloud data collected by the laser detection device in the target object. Based on the current confidence of the first pose, the first pose and the second pose of the target object at the current moment are fused to obtain the target positioning result of the target object. The second pose is determined based on sensor data other than laser point cloud data. Therefore, in laser-degraded scenarios such as tunnels, the confidence of the first pose in the multi-sensor data fusion positioning process can be dynamically adjusted according to the driving information of the target object, thereby effectively improving the positioning accuracy of the multi-sensor data fusion positioning.

[0053] Based on the above-mentioned inventive concept, the multi-sensor data fusion positioning method provided in the implementation manner of this specification is exemplarily described below.

[0054] Exemplary Methods

[0055] The embodiments of this specification provide a multi-sensor data fusion positioning method, such as Figure 1 As shown, including:

[0056] S101. When the current positioning scene of the target object is a laser degradation scene, determine the current confidence of the first posture of the target object based on the current driving information of the target object; the first posture is a posture determined based on the laser point cloud data collected by the laser detection device in the target object.

[0057] Specifically, the target object is an object to be positioned, such as a vehicle, a robot, etc. The current positioning scene of the target object is the scene where the target object is located at the current moment during the real-time positioning of the target object.

[0058] Laser degradation scenarios can include tunnels, single-side walls, bridges, special-shaped buildings, and more open scenes such as squares and airports.

[0059] During implementation, it is possible to determine whether the current positioning scene is a laser degraded scene in response to a user's input operation on a laser degraded scene. It is also possible to determine whether the current positioning scene is a laser degraded scene by receiving scene data sent by an external device. It is also possible to determine whether the current positioning scene is a laser degraded scene by performing image recognition on image data of the area where the target object is located. It is also possible to identify whether the current positioning scene is a laser degraded scene based on laser point cloud data collected by a laser detection device. The specific settings can be made according to actual needs.

[0060] The first pose of the target object may be a pose determined based on laser point cloud data collected by a laser detection device in the target object, and the laser detection device may be a laser radar. In the process of determining the first pose of the target object based on the laser point cloud data collected by the laser detection device, the laser point cloud data collected by the laser detection device at the current moment may be matched with the prior features in the map of the area to obtain the first pose of the target object at the current moment. Among them, before determining the first pose of the target object based on the laser point cloud data collected by the laser detection device, the laser point cloud data collected by the laser detection device may also be processed, such as data expansion processing, downsampling processing, filtering processing, etc., which may be specifically set according to actual needs.

[0061] The current confidence of the first pose can represent the credibility of the first pose in the process of fusion positioning of the target object through multi-sensor data. It can be understood that the higher the credibility of the first pose, the higher the weight of the first pose in the process of fusion positioning of the target object through multi-sensor data. In implementation, when the current positioning scene is a laser degradation scene, the confidence of the first pose can be reduced.

[0062] The current driving information of the target object may include the current driving speed and / or the current driving angular velocity of the target object, which may be specifically set according to actual needs.

[0063] When the current positioning scenario of the target object is a laser degradation scenario, the current confidence of the first pose of the target object can be determined based on the current driving information of the target object. For example, the confidence corresponding to the current driving information can be determined based on the current driving information and a predetermined correspondence between the driving information of the target object and the confidence of the first pose, and the confidence corresponding to the current driving information can be used as the current confidence of the first pose. In addition, the confidence corresponding to the current driving information can also be corrected to obtain the current confidence of the first pose.

[0064] It can be understood that the confidence of the first pose can be negatively correlated with the target object's driving speed, that is, the faster the target object's driving speed, the lower the confidence of the first pose, and the slower the target object's driving speed, the higher the confidence of the first pose. At the same time, the confidence of the first pose can be negatively correlated with the target object's driving angular velocity, that is, the greater the target object's driving angular velocity, the lower the confidence of the first pose, and the smaller the target object's driving angular velocity, the higher the confidence of the first pose.

[0065] Therefore, during the driving process of the target object, the confidence of the first pose of the target object can be updated in real time according to the driving information of the target object.

[0066] S102. Based on the current confidence of the first posture, the first posture and the second posture of the target object at the current moment are fused to obtain a target positioning result of the target object, wherein the second posture is determined based on sensor data other than the laser point cloud data.

[0067] Specifically, the second posture of the target object can be determined based on sensor data other than laser point cloud data, such as IMU data, wheel speed data, etc. The number of the second postures can be one or more. For example, the second posture corresponding to the IMU data can be obtained by integrating the IMU data, and the second posture corresponding to the wheel speed data can be obtained by integrating the wheel speed data.

[0068] In implementation, the first pose and each second pose of the target object at the current moment can be fused based on the current confidence of the first pose to obtain the target positioning result of the target object. For example, the first pose and each second pose of the target object at the current moment can be fused by ESKF (Error State Kalman Filter). As a result, in laser-degraded scenarios such as tunnels, the positioning accuracy of multi-sensor data fusion positioning can be effectively improved.

[0069] It can be understood that in the process of constructing a map using laser point cloud data collected by a laser detection device, the map can also be constructed based on the current confidence of the first pose, the laser point cloud data, and sensor data other than the laser point cloud data. At this time, the current confidence of the first pose can be used to determine the weight of the laser point cloud data in the map construction process, thereby effectively improving the accuracy of the constructed map in the laser degradation scenario, and then in the process of locating the target object according to the constructed map, the positioning accuracy of the target object can be further improved.

[0070] In a feasible implementation manner, determining the current confidence of the first pose of the target object based on the current driving information of the target object includes:

[0071] Determining an initial confidence level of the first posture at a current moment based on current driving information of the target object;

[0072] The initial confidence of the first posture at the current moment is corrected based on the historical confidence of the first posture at the previous moment to obtain the current confidence of the first posture.

[0073] Specifically, the initial confidence of the first pose at the current moment can be determined based on the current driving information of the target object. For example, the confidence corresponding to the current driving information can be determined based on the current driving information and the predetermined correspondence between the driving information of the target object and the confidence of the first pose, and the confidence corresponding to the current driving information is used as the initial confidence of the first pose at the current moment. It can be understood that the predetermined correspondence can be a mapping table, a function expression, or a machine learning model, which can be set according to actual needs.

[0074] After determining the initial confidence of the first pose at the current moment, the initial confidence of the first pose at the current moment can be corrected based on the historical confidence of the first pose at the previous moment to obtain the current confidence of the first pose. For example, the target correction value can be determined based on the historical confidence of the first pose at the previous moment, the initial confidence of the first pose at the current moment, and the target correspondence, and the sum of the initial confidence of the first pose at the current moment and the target correction value is determined as the current confidence of the first pose. The target correspondence can include a predetermined correspondence between the historical confidence of the first pose at the previous moment, the initial confidence of the first pose at the current moment, and the correction value to ensure the validity of the determination result of the current confidence of the first pose. In addition, the median or mean of the historical confidence of the first pose at the previous moment and the initial confidence of the first pose at the current moment can be used as the current confidence of the first pose to ensure the validity of the determination result of the current confidence of the first pose, and to avoid the confidence of the first pose changing too quickly, resulting in a large difference between the positioning result at the current moment and the positioning result at the previous moment, thereby avoiding the influence of the large difference in positioning results at adjacent moments on the driving control stability of the target object.

[0075] It can be understood that the difference between the initial confidence of the first pose at the current moment and the historical confidence of the first pose at the previous moment can also be obtained. When the difference is greater than a predetermined value, the initial confidence of the first pose at the current moment is corrected based on the historical confidence of the first pose at the previous moment. When the difference is less than or equal to the predetermined value, the initial confidence of the first pose at the current moment is determined as the current confidence of the first pose.

[0076] In addition, when the current moment is the first moment, the historical confidence of the first pose at the previous moment can be the same as the predetermined confidence.

[0077] As a preferred implementation, the historical confidence of the first pose at the previous moment and the initial confidence of the first pose at the current moment can be weighted and summed based on a predetermined weight coefficient to obtain the current confidence of the first pose, thereby effectively ensuring the linear and gentle change of the confidence of the first pose, and further ensuring the smoothness of the positioning trajectory of the target object. For example, the current confidence R_t of the first pose can be shown as formula (1):

[0078] R_t=wR_t'+(1-w)R_(t-1) (1)

[0079] Where R_t' is the initial confidence of the first pose at the current moment, R_(t-1) is the historical confidence of the first pose at the previous moment, and w is the predetermined weight coefficient, 0<w<1.

[0080] In a feasible implementation manner, the current driving information includes a current driving angular velocity and a current driving speed of the target object;

[0081] The determining the initial confidence of the first posture at the current moment based on the current driving information of the target object includes:

[0082] determining a current driving mode of the target object based on the current driving angular velocity, the driving mode comprising straight driving and turning;

[0083] When the current driving mode is straight driving, determining an initial confidence of the first posture at the current moment based on a current driving speed of the target object and a first predetermined corresponding relationship between the driving speed of the target object and the confidence of the first posture;

[0084] When the current driving mode is turning, the initial confidence of the first posture at the current moment is determined based on the current driving angular velocity of the target object and a second predetermined corresponding relationship between the driving angular velocity of the target object and the confidence of the first posture.

[0085] Specifically, the current driving angular velocity of the target object may be the driving angular velocity of the target object at the current moment, which may be detected by an IMU, etc. The current driving speed of the target object may be the driving speed of the target object at the current moment, which may be detected by an IMU, a wheel speed sensor, etc.

[0086] The current driving mode of the target object is the driving mode of the target object at the current moment, and the driving mode of the target object may include straight driving and turning. In implementation, the current driving mode of the target object may be determined based on the current driving angular velocity of the target object. For example, when the current driving angular velocity of the target object is greater than a predetermined angular velocity, the current driving mode of the target object may be determined to be turning, and when the current driving angular velocity of the target object is less than or equal to the predetermined angular velocity, the current driving mode of the target object may be determined to be straight driving.

[0087] When the current driving mode of the target object is straight driving, the initial confidence of the first posture at the current moment can be determined based on the current driving speed of the target object and the first predetermined correspondence between the driving speed of the target object and the confidence of the first posture. When the current driving mode of the target object is turning, the driving speed has little effect on the accuracy of the first posture. At this time, the initial confidence of the first posture at the current moment can be determined based on the current driving angular velocity of the target object and the second predetermined correspondence between the driving angular velocity of the target object and the confidence of the first posture.

[0088] In implementation, the first predetermined correspondence and the second predetermined correspondence may be the correspondence between the confidence of the first posture, the confidence threshold of the first posture, and the target parameter, wherein the confidence threshold of the first posture may include: the upper limit value of the confidence of the first posture, the lower limit value of the confidence of the first posture, the rate of change of the confidence of the first posture, the median of the upper limit value and the lower limit value of the confidence of the first posture; the target parameter under the first correspondence is the driving speed of the target object, and the target parameter under the second correspondence is the driving angular velocity of the target object.

[0089] Among them, the first predetermined correspondence relationship and the second predetermined correspondence relationship can both be function expressions. For example, when the first pose and each second pose of the target object at the current moment are fused by ESKF, the confidence of the first pose can be represented by the value of the observation noise covariance matrix R corresponding to the first pose. The larger the value of R, the lower the confidence, and the smaller the value of R, the higher the confidence. The first predetermined correspondence relationship and the second predetermined correspondence relationship can be shown as formula (2):

[0090] R_t'=(R_max-R_min) / (1+e^k(ax))+R_min (2)

[0091] Where R_max is the upper limit of the confidence of the first pose, R_min is the lower limit of the confidence of the first pose, k is the rate of change of the confidence of the first pose, a is the median of the upper and lower limits of the confidence of the first pose, R_max, R_min, k and a are all constants; x is the target parameter.

[0092] Taking the second correspondence as an example, the variation trend of the R value corresponding to the first posture with the driving angular velocity of the target object can be shown as follows: Figure 2 As shown by Figure 2 It can be seen that as the target object's driving angular velocity increases, the R value corresponding to the first pose gradually increases, that is, as the target object's driving angular velocity increases, the confidence of the first pose gradually decreases.

[0093] In a feasible implementation manner, the first pose of the target object at the current moment is determined by the following method:

[0094] Performing data expansion processing on the laser point cloud data collected by the laser detection device at the current moment to obtain laser point cloud data after data expansion processing;

[0095] Based on the laser point cloud data after the data expansion processing, the first position of the target object at the current moment is determined.

[0096] Specifically, data expansion processing is to increase the number of laser point clouds. Therefore, by performing data expansion processing on the laser point cloud data collected by the laser detection device at the current moment, the number of laser point clouds used to determine the first pose of the target object can be effectively increased, thereby effectively improving the accuracy of the determination result of the first pose of the target object, and then in the process of fusing the first pose and the second pose of the target object to obtain the target positioning result of the target object, the accuracy of the target positioning result can be further improved.

[0097] It can be understood that in the process of constructing a map using laser point cloud data collected by a laser detection device, the map can also be constructed based on the current confidence of the first pose, the laser point cloud data after data expansion processing, and the sensor data other than the laser point cloud data. Thus, the amount of laser point cloud data used to construct the map can be effectively increased, thereby effectively improving the accuracy of the constructed map in laser degradation scenarios, and further improving the positioning accuracy of the target object in the process of positioning the target object according to the constructed map.

[0098] In a feasible implementation manner, the laser point cloud data collected by the laser detection device at the current moment is subjected to data expansion processing to obtain the laser point cloud data after data expansion processing, including:

[0099] Acquire target laser point cloud data, wherein the target laser point cloud data includes incremental data in the laser point cloud data collected by the laser detection device at a previous moment;

[0100] The target laser point cloud data is combined with the laser point cloud data collected by the laser detection device at the current moment to obtain the laser point cloud data after the data expansion processing.

[0101] Specifically, for any moment, the incremental data in the laser point cloud data collected by the laser detection device at that moment can be the newly added laser point cloud data compared with the laser point cloud data collected by the laser detection device at the previous moment. In implementation, the incremental data in the laser point cloud data collected by the laser detection device at the previous moment can be used as the target laser point cloud data.

[0102] It is understandable that if the current moment is the first moment or the second moment, the incremental data corresponding to the moment before the current moment does not exist. At this time, the target laser point cloud data can be set to an empty set.

[0103] During implementation, in the process of data expansion processing of the laser point cloud data collected by the laser detection device at the current moment, the target laser point cloud data can be merged with the laser point cloud data collected by the laser detection device at the current moment to obtain the laser point cloud data after data expansion processing, thereby effectively increasing the amount of laser point cloud data in the direction where the actual collected laser point cloud data is less due to the limitation of the scene structure, thereby further improving the accuracy of the determination result of the first pose of the target object.

[0104] In a feasible implementation manner, determining the first position of the target object at the current moment based on the laser point cloud data after the data expansion processing includes:

[0105] Based on a target downsampling step length, downsampling the laser point cloud data after the data expansion process is performed to obtain current laser point cloud data; the target downsampling step length is smaller than a predetermined downsampling step length in a non-laser degradation scenario;

[0106] Based on the current laser point cloud data, determine the first position of the target object at the current moment.

[0107] Specifically, in the process of determining the first position pose of the target object at the current moment based on the laser point cloud data after data expansion processing, the laser point cloud data after data expansion processing can be downsampled based on the target downsampling step size to obtain the current laser point cloud data, wherein the target downsampling step size is less than the predetermined downsampling step size in the non-laser degradation scenario, that is, when the current positioning scenario of the target object is a laser degradation scenario, the downsampling step size in the downsampling process of the laser point cloud data is reduced, and the first position pose of the target object at the current moment is determined based on the downsampled laser point cloud data. As a result, the number of laser point clouds used to determine the first position pose of the target object can be effectively increased while improving the efficiency of determining the first position pose, reducing the consumption of computing resources, and maintaining the main features of the laser point cloud, thereby effectively improving the accuracy of the determination result of the first position pose of the target object, and further improving the accuracy of the target positioning result in the process of fusing the first position pose and the second position pose of the target object to obtain the target positioning result of the target object.

[0108] In a feasible implementation, it also includes:

[0109] Acquire current point cloud distribution information; the current point cloud distribution information includes distribution information of laser point cloud data collected by the laser detection device at the current moment in each target direction;

[0110] Based on the current point cloud distribution information, it is determined whether the current positioning scene of the target object is a laser degraded scene.

[0111] Specifically, the current point cloud distribution information may include the distribution information of the laser point cloud data collected by the laser detection device at the current moment in each target direction, and the target direction may include the X-axis direction, the Y-axis direction and the Z-axis direction.

[0112] For any direction in each target direction, the distribution information of the laser point cloud data in that direction can be used to characterize the concentration of the laser point cloud data in that direction. For example, it can include the variance or standard deviation of the laser point cloud data in that direction. The larger the variance or standard deviation, the more dispersed the distribution of the laser point cloud data in that direction. The smaller the variance or standard deviation, the more concentrated the distribution of the laser point cloud data in that direction. 2 For example, the calculation process can be shown as formula (3):

[0113]

[0114] Where N is the total number of laser point cloud data collected by the laser detection device at the current moment, X i is the value of the i-th point on the X-axis in the laser point cloud data collected by the laser detection device at the current moment, and μ is the mean value of the laser point cloud data collected by the laser detection device at the current moment on the X-axis.

[0115] In implementation, it is possible to determine whether the current positioning scene of the target object is a laser-degraded scene based on the current point cloud distribution information. For example, in a spiral bend scene, the laser point cloud data is concentrated in the X-axis, Y-axis, and Z-axis directions. In a tunnel scene, the laser point cloud data is concentrated in a certain range of space in the X-axis and Z-axis directions; in an open scene, the laser point cloud data is dispersed in the Z-axis direction.

[0116] As a preferred implementation manner, determining whether the current positioning scene of the target object is a laser degraded scene based on the current point cloud distribution information includes:

[0117] Matching the current point cloud distribution information with a plurality of predetermined scene models; the plurality of predetermined scene models correspond one-to-one to a plurality of laser degradation scenes; the predetermined scene models include thresholds of distribution information of the laser point cloud data in each of the target directions under the corresponding laser degradation scenes;

[0118] If there is a successfully matched predetermined scene model, determining the current positioning scene of the target object as a laser degradation scene;

[0119] If there is no successfully matched predetermined scene model, the current positioning scene of the target object is determined as a non-laser degraded scene.

[0120] Specifically, multiple scene models can be pre-constructed and used as predetermined scene models, wherein the multiple predetermined scene models can correspond one-to-one to multiple laser degradation scenes. For any predetermined scene model, the predetermined scene model can include a threshold value of the distribution information of the laser point cloud data in each target direction under the corresponding laser degradation scene, for example, it can include a threshold range of the variance of the laser point cloud data in each target direction. In implementation, the distribution information of multiple frames of historical point cloud data in each target direction under the laser degradation scene can be statistically analyzed to construct the corresponding predetermined scene model. It is understandable that the predetermined scene model can also be iteratively updated to ensure the accuracy of the recognition result of the laser degradation scene.

[0121] In the process of positioning the target object, the current point cloud distribution information can be matched with multiple predetermined scene models respectively. For example, for any predetermined scene model, if the distribution information in each target direction in the current point cloud distribution information meets the threshold of the distribution information in the corresponding target direction in the predetermined scene model, then the current point cloud distribution information is successfully matched with the predetermined scene model; if there is at least one target direction in the current point cloud distribution information that does not meet the threshold of the distribution information in the corresponding target direction in the predetermined scene model, then the current point cloud distribution information fails to match the predetermined scene model.

[0122] Among them, if there is a predetermined scene model that successfully matches the current point cloud distribution information, the current positioning scene of the target object is determined to be a laser degraded scene; if there is no predetermined scene model that successfully matches the current point cloud distribution information, the current positioning scene of the target object is determined to be a non-laser degraded scene, so that it is possible to quickly and accurately identify whether the current positioning scene of the target object is a laser degraded scene.

[0123] It is understandable that the distribution information of the laser point cloud data collected by the laser detection device at the current moment in each target direction can also be compared with the corresponding predetermined threshold interval. If the distribution information of the laser point cloud data collected by the laser detection device at the current moment in each target direction is within the corresponding predetermined threshold interval, the current positioning scene of the target object is determined as a non-laser degraded scene. If the distribution information of the laser point cloud data collected by the laser detection device at the current moment in at least one target direction is not within the corresponding predetermined threshold interval, the current positioning scene of the target object is determined as a laser degraded scene.

[0124] Exemplary Devices

[0125] In an exemplary embodiment of the present specification, a multi-sensor data fusion positioning device is also provided, such as Figure 3 As shown, including:

[0126] A first processing module 301 is used to determine a current confidence of a first pose of the target object based on current driving information of the target object when the current positioning scene of the target object is a laser degradation scene; the first pose is a pose determined based on laser point cloud data collected by a laser detection device in the target object;

[0127] The second processing module 302 is used to fuse the first pose and the second pose of the target object at the current moment based on the current confidence of the first pose to obtain the target positioning result of the target object, wherein the second pose is determined based on the sensor data other than the laser point cloud data.

[0128] In a feasible implementation manner, the first processing module 301 is specifically configured to:

[0129] Determining an initial confidence level of the first posture at a current moment based on current driving information of the target object;

[0130] The initial confidence of the first posture at the current moment is corrected based on the historical confidence of the first posture at the previous moment to obtain the current confidence of the first posture.

[0131] In a feasible implementation manner, the current driving information includes a current driving angular velocity and a current driving speed of the target object;

[0132] The first processing module 301 is specifically used for:

[0133] determining a current driving mode of the target object based on the current driving angular velocity, the driving mode comprising straight driving and turning;

[0134] When the current driving mode is straight driving, determining an initial confidence of the first posture at the current moment based on a current driving speed of the target object and a first predetermined corresponding relationship between the driving speed of the target object and the confidence of the first posture;

[0135] When the current driving mode is turning, the initial confidence of the first posture at the current moment is determined based on the current driving angular velocity of the target object and a second predetermined corresponding relationship between the driving angular velocity of the target object and the confidence of the first posture.

[0136] In a feasible implementation manner, the first processing module 301 is further configured to:

[0137] Performing data expansion processing on the laser point cloud data collected by the laser detection device at the current moment to obtain laser point cloud data after data expansion processing;

[0138] Based on the laser point cloud data after the data expansion processing, the first position of the target object at the current moment is determined.

[0139] In a feasible implementation manner, the first processing module 301 is specifically configured to:

[0140] Acquire target laser point cloud data, wherein the target laser point cloud data includes incremental data in the laser point cloud data collected by the laser detection device at a previous moment;

[0141] The target laser point cloud data is combined with the laser point cloud data collected by the laser detection device at the current moment to obtain the laser point cloud data after the data expansion processing.

[0142] In a feasible implementation manner, the first processing module 301 is specifically configured to:

[0143] Based on a target downsampling step length, downsampling the laser point cloud data after the data expansion process is performed to obtain current laser point cloud data; the target downsampling step length is smaller than a predetermined downsampling step length in a non-laser degradation scenario;

[0144] Based on the current laser point cloud data, determine the first position of the target object at the current moment.

[0145] In a feasible implementation manner, the first processing module 301 is further configured to:

[0146] Acquire current point cloud distribution information; the current point cloud distribution information includes distribution information of laser point cloud data collected by the laser detection device at the current moment in each target direction;

[0147] Based on the current point cloud distribution information, it is determined whether the current positioning scene of the target object is a laser degraded scene.

[0148] In a feasible implementation manner, the first processing module 301 is specifically configured to:

[0149] Matching the current point cloud distribution information with a plurality of predetermined scene models; the plurality of predetermined scene models correspond one-to-one to a plurality of laser degradation scenes; the predetermined scene models include thresholds of distribution information of the laser point cloud data in each of the target directions under the corresponding laser degradation scenes;

[0150] If there is a successfully matched predetermined scene model, determining the current positioning scene of the target object as a laser degradation scene;

[0151] If there is no successfully matched predetermined scene model, the current positioning scene of the target object is determined as a non-laser degraded scene.

[0152] The multi-sensor data fusion positioning device provided in this embodiment belongs to the same application concept as the multi-sensor data fusion positioning method provided in the above embodiments of this application, and can execute the multi-sensor data fusion positioning method provided in any of the above embodiments of this application, and has the corresponding functional modules and beneficial effects of executing the multi-sensor data fusion positioning method. For technical details not fully described in this embodiment, please refer to the specific processing content of the multi-sensor data fusion positioning method provided in the above embodiments of this application, which will not be repeated here.

[0153] Exemplary Devices

[0154] In an exemplary embodiment of the present specification, an electronic device is also provided, which includes at least one processor and at least one memory, wherein a computer program is stored in the memory, and when the computer program is executed by the processor, the multi-sensor data fusion positioning method as described in any of the above embodiments is implemented.

[0155] Example Vehicles

[0156] In an exemplary embodiment of the present specification, a vehicle is also provided, comprising a laser detection device and a multi-sensor data fusion positioning device as described in any one of the above embodiments.

[0157] Exemplary computer program products and storage media

[0158] In addition to the above-mentioned methods and devices, the multi-sensor data fusion positioning method provided in the embodiments of this specification may also be a computer program product, which includes computer program instructions, which, when executed by a processor, enable the processor to execute the steps of the multi-sensor data fusion positioning method according to various embodiments of this specification described in the above "Exemplary Method" section of this specification.

[0159] The computer program product can be written in any combination of one or more programming languages ​​to write program codes for executing the operations of the embodiments of this specification, and the programming languages ​​include object-oriented programming languages, such as Java, C++, etc., and also include conventional procedural programming languages, such as "C" language or similar programming languages.

[0160] In addition, an embodiment of the present specification also provides a computer-readable storage medium on which a computer program is stored, and the computer program is executed by a processor to execute the steps of the multi-sensor data fusion positioning method according to various embodiments of the present specification described in the above "Exemplary Method" section of the present specification.

[0161] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiments can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this specification can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

[0162] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0163] The above-mentioned embodiments only express several implementation methods of this specification, and the descriptions are relatively specific and detailed, but they cannot be understood as limiting the scope of the solutions provided by the embodiments of this specification. It should be pointed out that for ordinary technicians in this field, several modifications and improvements can be made without departing from the concept of this specification, which all belong to the protection scope of this specification. Therefore, the protection scope of the patent of this specification shall be based on the attached claims.

Claims

1. A multi-sensor data fusion positioning method, characterized in that: include: When the current positioning scene of the target object is a laser degradation scene, determining a current confidence of a first pose of the target object based on current driving information of the target object; the first pose is a pose determined based on laser point cloud data collected by a laser detection device in the target object; Based on the current confidence of the first posture, the first posture and the second posture of the target object at the current moment are fused to obtain a target positioning result of the target object, wherein the second posture is determined based on the sensor data other than the laser point cloud data.

2. The method according to claim 1, characterized in that The determining the current confidence of the first pose of the target object based on the current driving information of the target object comprises: Determining an initial confidence level of the first posture at a current moment based on current driving information of the target object; The initial confidence of the first posture at the current moment is corrected based on the historical confidence of the first posture at the previous moment to obtain the current confidence of the first posture.

3. The method according to claim 2, characterized in that The current driving information includes the current driving angular velocity and the current driving speed of the target object; The determining the initial confidence of the first posture at the current moment based on the current driving information of the target object includes: determining a current driving mode of the target object based on the current driving angular velocity, the driving mode comprising straight driving and turning; When the current driving mode is straight driving, determining an initial confidence of the first posture at the current moment based on a current driving speed of the target object and a first predetermined corresponding relationship between the driving speed of the target object and the confidence of the first posture; When the current driving mode is turning, the initial confidence of the first posture at the current moment is determined based on the current driving angular velocity of the target object and a second predetermined corresponding relationship between the driving angular velocity of the target object and the confidence of the first posture.

4. The method according to claim 1, characterized in that: The first pose of the target object at the current moment is determined by the following method: Performing data expansion processing on the laser point cloud data collected by the laser detection device at the current moment to obtain laser point cloud data after data expansion processing; Based on the laser point cloud data after the data expansion processing, the first position of the target object at the current moment is determined.

5. The method according to claim 4, characterized in that The step of performing data expansion processing on the laser point cloud data collected by the laser detection device at the current moment to obtain the laser point cloud data after data expansion processing includes: Acquire target laser point cloud data, wherein the target laser point cloud data includes incremental data in the laser point cloud data collected by the laser detection device at a previous moment; The target laser point cloud data is combined with the laser point cloud data collected by the laser detection device at the current moment to obtain the laser point cloud data after the data expansion processing.

6. The method according to claim 4, characterized in that Determining the first position of the target object at the current moment based on the laser point cloud data after the data expansion processing includes: Based on a target downsampling step length, downsampling the laser point cloud data after the data expansion process is performed to obtain current laser point cloud data; the target downsampling step length is smaller than a predetermined downsampling step length in a non-laser degradation scenario; Based on the current laser point cloud data, determine the first position of the target object at the current moment.

7. The method according to any one of claims 1 to 6, characterized in that: Also includes: Acquire current point cloud distribution information; the current point cloud distribution information includes distribution information of laser point cloud data collected by the laser detection device at the current moment in each target direction; Based on the current point cloud distribution information, it is determined whether the current positioning scene of the target object is a laser degraded scene.

8. The method according to claim 7, characterized in that The determining, based on the current point cloud distribution information, whether the current positioning scene of the target object is a laser degraded scene includes: Matching the current point cloud distribution information with a plurality of predetermined scene models; the plurality of predetermined scene models correspond one-to-one to a plurality of laser degradation scenes; the predetermined scene models include thresholds of distribution information of the laser point cloud data in each of the target directions under the corresponding laser degradation scenes; If there is a successfully matched predetermined scene model, determining the current positioning scene of the target object as a laser degradation scene; If there is no successfully matched predetermined scene model, the current positioning scene of the target object is determined as a non-laser degraded scene.

9. A multi-sensor data fusion positioning device, characterized in that: include: A first processing module, used to determine a current confidence of a first pose of the target object based on current driving information of the target object when the current positioning scene of the target object is a laser degradation scene; the first pose is a pose determined based on laser point cloud data collected by a laser detection device in the target object; The second processing module is used to fuse the first pose and the second pose of the target object at the current moment based on the current confidence of the first pose to obtain the target positioning result of the target object, wherein the second pose is determined based on the sensor data other than the laser point cloud data.

10. A vehicle, characterized in that: It comprises a laser detection device and the multi-sensor data fusion positioning device as claimed in claim 9.

Images