Positioning system for automatic driving and multi-sensor fusion positioning method

Abstract

The invention relates to the technical field of automatic driving, in particular to a positioning system for automatic driving and a multi-sensor fusion positioning method. The positioning system comprises a first laser radar positioning module, a second laser radar positioning module, a visual positioning module, a GNSS navigation positioning module and a positioning fusion module. According to the invention, the positioning principles of the positioning modules are different, and different application scenes are emphasized. In the real-time positioning process of the vehicle, each positioning module obtains one piece of corresponding position information, then the positioning fusion module carries out information fusion on the collected multiple pieces of position information, and final positioning information of the fused vehicle is obtained. When one of the positioning modules fails or the positioning precision of one positioning module in the current scene is low, the other positioning modules can work normally, and the output final position information has high positioning precision by performing information fusion on multiple pieces of position information.

Description

Technical Field

[0001] The present application relates to the field of autonomous driving technology, and in particular to a positioning system for autonomous driving and a multi-sensor fusion positioning method. Background Art

[0002] With the rapid development of computer science, hardware manufacturing, artificial intelligence and other industries, the research on autonomous driving is also advancing. As an important technology in the future development of society, autonomous driving technology is widely used in transportation logistics, transportation and travel.

[0003] In the field of autonomous driving, location information provides key information such as road obstacles, lane lines, traffic signals, etc. for autonomous vehicles, helping them to identify and take corresponding obstacle avoidance measures, maintain the correct driving route, and comply with traffic rules. This information is not only related to the safe driving of autonomous vehicles, but also the basis for the realization of autonomous driving technology. Therefore, real-time acquisition of high-precision location information of vehicles is particularly important for the realization of autonomous driving technology.

[0004] In the related art, when the vehicle is positioned in real time with high precision, it is found through actual application scenario testing that the positioning method based on a single sensor is easily affected by environmental factors when positioning the vehicle in real time, resulting in reduced positioning accuracy. For example, when the vehicle enters a tunnel or the sensor is blocked, the signal is weak or lost, which will cause positioning failure or reduced positioning accuracy. Therefore, the vehicle positioning system in the related art is difficult to meet the needs of autonomous driving scenarios. Summary of the invention

[0005] In order to solve the technical problem in the related art that a single sensor has low positioning accuracy for a vehicle or is prone to positioning failure, the present application provides a positioning system for autonomous driving and a multi-sensor fusion positioning method.

[0006] In order to achieve the above objectives, this application adopts the following technical solutions:

[0007] On the one hand, the present application provides a positioning system for autonomous driving, comprising a first laser radar positioning module, a second laser radar positioning module, a visual positioning module, a GNSS navigation positioning module, and a positioning fusion module;

[0008] The first laser radar positioning module is used to obtain first point cloud information of the vehicle in the current environment, and determine first position information of the vehicle in the current environment according to the first point cloud information and the prior map;

[0009] The second laser radar positioning module is used to obtain second point cloud information of the vehicle in the current environment, and determine second position information of the vehicle in the current environment according to the online constructed map model and the second point cloud information;

[0010] The visual positioning module is used to obtain image information of the vehicle in the current environment, and determine the third position information of the vehicle in the current environment according to the feature points in the image information;

[0011] The GNSS navigation and positioning module is used to obtain fourth position information of the vehicle in the current environment;

[0012] The positioning fusion module is used to fuse at least two of the first position information, the second position information, the third position information and the fourth position information to obtain the final positioning information of the vehicle after fusion.

[0013] In one possible design, the positioning system further includes an IMU auxiliary positioning module and a wheel speed meter auxiliary positioning module;

[0014] The IMU-assisted positioning module is used to obtain angular velocity and acceleration information of the vehicle, and determine fifth position information of the vehicle in the current environment according to the angular velocity and acceleration information;

[0015] The wheel speed meter auxiliary positioning module is used to obtain wheel speed information of the vehicle and determine the sixth position information of the vehicle in the current environment according to the wheel speed information;

[0016] The positioning fusion module is also used to fuse at least four of the first position information, the second position information, the third position information, the fourth position information, the fifth position information and the sixth position information to obtain the final positioning information of the vehicle after fusion.

[0017] In one possible design, the positioning system further includes a positioning status monitoring module;

[0018] The positioning status monitoring module is used to monitor the status information of the first laser radar positioning module and the GNSS navigation positioning module respectively, and when it is monitored that the first laser radar positioning module and the GNSS navigation positioning module are both operating normally, generate a first monitoring signal, and send the first monitoring signal to the positioning fusion module;

[0019] The positioning fusion module is used to fuse the first position information, the fourth position information, the fifth position information and the sixth position information when receiving the first monitoring signal, so as to obtain the final positioning information of the vehicle after fusion.

[0020] In a possible design, the positioning state monitoring module is further used to generate a second monitoring signal when detecting that the first laser radar positioning module fails, and send the second monitoring signal to the positioning fusion module;

[0021] The positioning fusion module is used to fuse the second position information, the fourth position information, the fifth position information and the sixth position information when the second monitoring signal is received, so as to obtain the final positioning information of the vehicle after fusion; or the positioning fusion module is used to fuse the second position information, the third position information, the fourth position signal, the fifth position information and the sixth position information when the second monitoring signal is received, so as to obtain the final positioning information of the vehicle after fusion.

[0022] In a possible design, the positioning status monitoring module is further used to generate a third monitoring signal when it is detected that the GNSS navigation and positioning module fails, and send the third monitoring signal to the positioning fusion module;

[0023] The positioning fusion module is used to fuse the first position information, the third position information, the fifth position information and the sixth position information when receiving the third monitoring signal to obtain the final positioning information of the vehicle after fusion; or the positioning fusion module is used to fuse the first position information, the second position information, the third position information, the fifth position information and the sixth position information when receiving the third monitoring signal to obtain the final positioning information of the vehicle after fusion.

[0024] In a possible design, the positioning status monitoring module is further used to generate a fourth monitoring signal when it is detected that both the first laser radar positioning module and the GNSS navigation positioning module fail, and send the fourth monitoring signal to the positioning fusion module;

[0025] The positioning fusion module is used to fuse the second position information, the third position information, the fifth position information and the sixth position information when receiving the fourth monitoring signal, so as to obtain the final positioning information of the vehicle after fusion.

[0026] In a possible design, fusing at least four of the first position information, the second position information, the third position information, the fourth position information, the fifth position information, and the sixth position information to obtain the fused final positioning information of the vehicle includes:

[0027] Obtain a weight value corresponding to each of the first location information, the second location information, the third location information, the fourth location information, the fifth location information, and the sixth location information;

[0028] According to the weight values ​​corresponding to the respective position information, at least four of the first position information, the second position information, the third position information, the fourth position information, the fifth position information and the sixth position information are fused to obtain the final fused positioning information of the vehicle.

[0029] On the other hand, the present application also provides a multi-sensor fusion positioning method for processing the position information collected by each positioning module in the positioning system on the vehicle to obtain the final position information of the vehicle, wherein the positioning system is the positioning system as described above;

[0030] The multi-sensor fusion positioning method comprises:

[0031] When it is monitored that the first laser radar positioning module and the GNSS navigation positioning module are both operating normally, obtaining the first position information collected by the first laser radar positioning module, the fourth position information collected by the GNSS navigation positioning module, the fifth position information collected by the IMU auxiliary positioning module, and the sixth position information collected by the wheel speed meter auxiliary positioning module;

[0032] The first position information, the fourth position information, the fifth position information and the sixth position information are fused to obtain final fused positioning information of the vehicle.

[0033] In a possible design, the first position information, the fourth position information, the fifth position information, and the sixth position information are fused to obtain the fused final positioning information of the vehicle, including:

[0034] Obtain a weight value corresponding to each of the first location information, the fourth location information, the fifth location information, and the sixth location information;

[0035] According to the weight values ​​corresponding to the respective position information, the first position information, the fourth position information, the fifth position information and the sixth position information are fused to obtain the final fused positioning information of the vehicle.

[0036] On the other hand, the present application also provides an autonomous driving vehicle, which includes the positioning system as described above.

[0037] According to the positioning system for automatic driving provided by the present application, it includes a first laser radar positioning module, a second laser radar positioning module, a visual positioning module, a GNSS navigation positioning module and a positioning fusion module. Among them, the positioning principles of each positioning module in the present application are different, and different positioning modules focus on different application scenarios. In the process of real-time positioning of the vehicle, each positioning module obtains a corresponding position information, and then the positioning fusion module fuses the collected multiple position information to obtain the final positioning information of the fused vehicle. In this way, when one of the positioning modules fails, or the positioning accuracy of a certain positioning module in the current scene is low, the remaining positioning modules can work normally. By fusing multiple position information, the technical defects can be overcome, and the final position information obtained after fusion has a higher positioning accuracy.

[0038] Based on the implementations provided in the above aspects, this application can also be further combined to provide more implementations. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] Figure 1 A schematic diagram of the structure of a positioning system provided in an embodiment of the present application;

[0040] Figure 2 A schematic diagram of the positioning process when the positioning system provided in the embodiment of the present application is working;

[0041] Figure 3 A schematic diagram of the distribution of sensors in the positioning system provided in an embodiment of the present application;

[0042] Figure 4 A schematic diagram of the flow of the multi-sensor fusion positioning method provided in an embodiment of the present application. DETAILED DESCRIPTION

[0043] The technical solution in this application will be described below in conjunction with the accompanying drawings.

[0044] In the description of the embodiments of the present application, unless otherwise specified, “ / ” means or. For example, A / B can mean A or B. The “and / or” in this article is merely a way to describe the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone.

[0045] In the following, the terms "first" and "second" are used for descriptive purposes only and are not to be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of the features. In the description of this embodiment, unless otherwise specified, "plurality" means two or more.

[0046] In related technologies, the positioning system on an autonomous driving vehicle usually uses a positioning method based on a single sensor. For example, most vehicles use a GNSS (Global Navigation Satellite System) positioning system to perform real-time positioning of the vehicle. However, in actual application scenarios, the positioning method based on a single sensor is easily affected by environmental factors. For example, when the vehicle enters a long-distance tunnel or the sensor is blocked by a large vehicle, the positioning accuracy is prone to decrease or positioning failure, and it is unable to meet the real-time positioning requirements under multiple traffic flows.

[0047] In order to overcome the above-mentioned technical defects, the present application provides a positioning system for autonomous driving, which includes a plurality of different positioning modules and a positioning fusion module. Among them, the positioning principles of each positioning module in the present application are different, and different positioning modules focus on different application scenarios. In the process of real-time positioning of the vehicle, each positioning module obtains a corresponding position information, and then the positioning fusion module fuses the collected multiple position information to obtain the final positioning information of the fused vehicle. In this way, when one of the positioning modules fails, or the positioning accuracy of a certain positioning module in the current scene is low, the remaining positioning modules can work normally. By fusing multiple position information, the technical defects can be overcome, and the final position information obtained after fusion has a higher positioning accuracy.

[0048] A positioning system for autonomous driving provided in an embodiment of the present application is described in detail below in conjunction with the accompanying drawings. Figure 1 For a schematic diagram of the structure of the positioning system provided in the embodiment of the present application, see Figure 1 As shown, the positioning system includes a first laser radar positioning module 11, a second laser radar positioning module 12, a visual positioning module 13, a GNSS navigation positioning module 14 and a positioning fusion module 15.

[0049] Among them, the first laser radar positioning module 11 is used to obtain the first point cloud information of the vehicle in the current environment, and determine the first position information of the vehicle in the current environment based on the first point cloud information and the prior map. The second laser radar positioning module 12 is used to obtain the second point cloud information of the vehicle in the current environment, and determine the second position information of the vehicle in the current environment based on the online constructed map model and the second point cloud information. The visual positioning module 13 is used to obtain the image information of the vehicle in the current environment, and determine the third position information of the vehicle in the current environment based on the feature points in the image information; the GNSS navigation positioning module 14 is used to obtain the fourth position information of the vehicle in the current environment.

[0050] The positioning fusion module 15 is used to fuse at least two of the first position information, the second position information, the third position information and the fourth position information to obtain the final positioning information of the vehicle after fusion.

[0051] Please continue to see Figure 1 As shown, in some embodiments, the positioning system further includes an IMU-assisted positioning module 16 and a speedometer-assisted positioning module 17 .

[0052] Among them, the IMU auxiliary positioning module 16 is used to obtain the angular velocity and acceleration information of the vehicle, and determine the fifth position information of the vehicle in the current environment according to the angular velocity and acceleration information; the wheel speed meter auxiliary positioning module 17 is used to obtain the wheel speed information of the vehicle, and determine the sixth position information of the vehicle in the current environment according to the wheel speed information. The positioning fusion module 15 is also used to fuse at least four of the first position information, the second position information, the third position information, the fourth position information, the fifth position information and the sixth position information to obtain the final positioning information of the fused vehicle.

[0053] Among them, the first laser radar positioning module 11, the second laser radar positioning module 12, the visual positioning module 13 and the GNSS navigation positioning module 14 are main positioning modules, and the IMU auxiliary positioning module 16 and the speedometer auxiliary positioning module 17 are auxiliary positioning modules. Since the IMU auxiliary positioning module 16 and the speedometer auxiliary positioning module 17 are generally arranged on the vehicle and have higher stability, the IMU auxiliary positioning module 16 and the speedometer auxiliary positioning module 17 are used for auxiliary positioning, which can improve the positioning accuracy of the vehicle.

[0054] In some embodiments, the first laser radar positioning module 11 relies on the high-precision environmental scanning data provided by the laser radar and the prior environmental map data to achieve precise matching positioning; the GNSS navigation positioning module 14 relies on RTK (Real-time kinematic) differential positioning and IMU (Inertial Measuring Unit) high-frequency motion and attitude data to provide continuous high-precision positioning, attitude and heading information, etc.; the second laser radar positioning module 12 relies on the second point cloud information collected in real time, and in the absence of a priori map, by matching the currently scanned second point cloud information with the pre-built map model, the transformation of the current second point cloud information relative to the map (such as position and attitude) is calculated, thereby determining the current position of the vehicle.

[0055] In some embodiments, the visual positioning module 13 can rely on the camera to identify key frame feature points, match feature points between consecutive frames, and calculate the motion trajectory of the feature points to estimate the current position of the vehicle; the IMU-assisted positioning module 16 and the speedometer-assisted positioning module 17 respectively provide high-frequency motion data, posture information and vehicle wheel speed to enhance the robustness and accuracy of several main positioning modules.

[0056] In some embodiments, the positioning system of the autonomous driving further includes a positioning state monitoring module 18, which is used to monitor the state information of the first laser radar positioning module 11 and the GNSS navigation positioning module 14 respectively, and the state and performance of the positioning module can be determined according to the state information obtained by monitoring. For example, according to the state information obtained by monitoring, it can be determined whether the positioning module is in a normal working state, and the accuracy of the position information output by the positioning module can be determined.

[0057] Exemplarily, based on the status information obtained through monitoring, it can be determined whether the first laser radar positioning module 11 and the GNSS navigation positioning module 14 are currently in a normal working state or a fault occurs. The fault in this embodiment can be understood as: hardware failure, positioning method failure, or accuracy cannot meet the preset requirements. If the first laser radar positioning module 11 and the GNSS navigation positioning module 14 are currently in a normal working state, it means that the position information output by them is valid. If the first laser radar positioning module 11 and the GNSS navigation positioning module 14 are currently in a fault state, it means that the position information output by them is invalid.

[0058] In some embodiments, the positioning state monitoring module 18 is also used to monitor the working states of the first laser radar positioning module 11, the second laser radar positioning module 12, the visual positioning module 13, the GNSS navigation positioning module 14, the IMU auxiliary positioning module 16 and the wheel speed meter auxiliary positioning module 17, respectively, to determine whether each positioning module is working normally and whether the positioning accuracy meets the requirements. The positioning fusion module 15 is also used to select a positioning module that processes a normal working state and is adapted to the current scene according to the current scene information, and then fuse the position information output by the selected appropriate positioning module to obtain the final position information.

[0059] In some embodiments, when the positioning status monitoring module 18 detects that the first laser radar positioning module 11 and the GNSS navigation positioning module 14 are operating normally, it generates a first monitoring signal and sends the first monitoring signal to the positioning fusion module 15; the positioning fusion module 15 is used to fuse the first position information, the fourth position information, the fifth position information and the sixth position information when receiving the first monitoring signal to obtain the final positioning information of the vehicle after fusion.

[0060] It can be understood that in most scenarios, when the first laser radar positioning module 11 and the GNSS navigation positioning module 14 are working normally, the position signals output by the first laser radar positioning module 11 and the GNSS navigation positioning module 14 are used as the main positioning information, and combined with the auxiliary positioning information output by the IMU auxiliary positioning module 16 and the speedometer auxiliary positioning module 17, the current position of the vehicle can be determined and the positioning accuracy requirements of most scenarios can be met.

[0061] In some embodiments, the positioning status monitoring module 18 is also used to generate a second monitoring signal when a failure of the first laser radar positioning module 11 is detected, and send the second monitoring signal to the positioning fusion module 15. The positioning fusion module 15 is used to fuse the second position information, the fourth position information, the fifth position information and the sixth position information when the second monitoring signal is received to obtain the final positioning information of the fused vehicle. Alternatively, in another embodiment, the positioning fusion module 15 is used to fuse the second position information, the third position information, the fourth position signal, the fifth position information and the sixth position information when the second monitoring signal is received to obtain the final positioning information of the fused vehicle.

[0062] In some embodiments, the positioning state monitoring module 18 is further used to generate a third monitoring signal when a failure of the GNSS navigation positioning module 14 is detected, and send the third monitoring signal to the positioning fusion module 15. The positioning fusion module 15 is used to fuse the first position information, the third position information, the fifth position information and the sixth position information when receiving the third monitoring signal, and obtain the final positioning information of the fused vehicle; or the positioning fusion module 15 is used to fuse the first position information, the second position information, the third position information, the fifth position information and the sixth position information when receiving the third monitoring signal, and obtain the final positioning information of the fused vehicle.

[0063] In some embodiments, the positioning status monitoring module 18 is also used to generate a fourth monitoring signal and send the fourth monitoring signal to the positioning fusion module 15 when it is detected that both the first laser radar positioning module 11 and the GNSS navigation positioning module 14 have failed; the positioning fusion module 15 is used to fuse the second position information, the third position information, the fifth position information and the sixth position information when receiving the fourth monitoring signal to obtain the final positioning information of the vehicle after fusion.

[0064] In some embodiments, at least four of the first position information, the second position information, the third position information, the fourth position information, the fifth position information, and the sixth position information are fused to obtain the final positioning information of the fused vehicle, specifically including:

[0065] First, the weight value corresponding to each of the first position information, the second position information, the third position information, the fourth position information, the fifth position information and the sixth position information is obtained; then, according to the weight value corresponding to each of the position information, at least four of the first position information, the second position information, the third position information, the fourth position information, the fifth position information and the sixth position information are fused to obtain the final positioning information of the fused vehicle.

[0066] According to the above description, in different scenarios, based on the status information of each positioning module monitored by the positioning status monitoring module 18, the positioning module in normal working state can be determined first, and then based on the characteristics of the current scene, the location information output by multiple positioning modules that are adapted to the current scene can be selected, and the multiple location information can be integrated to obtain the final positioning information of the vehicle.

[0067] It can be seen that the present application monitors and evaluates the status and performance of each positioning module in real time through the positioning status monitoring module 18. When faced with various complex scenarios, the positioning system can effectively select an adaptive positioning method to meet the positioning requirements of all scenarios. The positioning fusion module 15 can fuse the location information output from different positioning modules, and can automatically select the location information output by the appropriate positioning module for fusion according to different scenarios, and assign corresponding weights to different location information according to the positioning accuracy to optimize the positioning results.

[0068] Figure 2 For a schematic diagram of the positioning process of the positioning system provided in the embodiment of the present application, please refer to Figure 2 As shown, the first lidar positioning module 11 uses RTK real-time dynamic differential technology to achieve point cloud matching positioning of the initial posture according to the NDT (Normal Distribution Transform) positioning algorithm and the environmental prior map. When the point cloud matching drifts or the positioning quality is poor, the point cloud matching can be achieved through the GNSS navigation positioning module 14 to achieve repositioning of the vehicle.

[0069] In this application, the longitude, latitude and altitude provided by the spherical coordinate system can be converted into the UTM coordinate system (adding the Z axis for altitude), thereby completing the unification of the a priori map coordinate system and the GNSS coordinate system.

[0070] Among them, after the GNSS coordinate system is converted into the UTM coordinate system, the conversion between the UTM and the local coordinate system of the prior map is completed through the Bursa seven-parameter model. The coordinate system includes three translation parameters (ΔX0, ΔY0, ΔZ0), three rotation parameters (ε x ,ε y ,ε z) and a scaling factor (m). The translation parameter defines the spatial position movement relationship between the two coordinate systems, the rotation parameter defines the axis rotation relationship between the two coordinate systems, and the scaling factor defines the scale relationship between the coordinate systems. The seven-parameter transformation model of different spatial rectangular coordinate systems is:

[0071]

[0072] Among them, (X A ,Y A ,Z A ) represents the three-dimensional space coordinates in the original coordinate system, (X B ,Y B ,Z B ) represents the three-dimensional spatial position coordinates in the target coordinate system relative to the original coordinate system, where the subscript A represents the original coordinate system and the subscript B represents the target coordinate system. The calculation method of the Bursa seven-parameter model requires knowing the same position in the original coordinate system and the target coordinate system. Only when the same position represented by two different coordinate systems is known can the conversion parameters be calculated. In this way, the coordinates in the original coordinate system can be directly obtained relative to the position in the target coordinate system through the seven parameters. When the Bursa seven parameters are known, the corresponding coordinates of the UTM coordinate system relative to the prior map coordinate system can be calculated as follows:

[0073] X B =(m+1)X A +ε Z Y A -ε Y Z A +ΔX0 (2)

[0074] Y B =(m+1)Y A +ε X Z A -ε z X A +ΔY0 (3)

[0075] Z B =(m+1)Z A +ε Y X A -ε X Y A +ΔZ0 (4)

[0076] Among them,

[0077] The optimal conversion parameters solved by the least square method can be expressed as L = BX, where the seven-parameter matrix obtained by solving is X = (B T B)-1 B T L, the coordinate transformation parameters can be calculated through the above expression.

[0078] When both the first laser radar positioning module 11 and the GNSS navigation positioning module 14 are not invalid, the covariance matrix of the two positioning methods is used to evaluate the positioning accuracy of each, and the two positioning methods are given corresponding confidences. The auxiliary positioning information provided by the IMU and the wheel speed meter is combined to complete the fusion positioning of the extended Kalman filter. Among them, the positioning fusion module 15 uses the EKF (Extended Kalman Filter) algorithm to simplify the four-wheel vehicle model. According to the method of track calculation, the position of the vehicle at time k is known to be (x k ,y k ,v k ,θ k ) T And the current speed of the vehicle is v k and acceleration a k , we can infer that the vehicle’s position relative to the high-precision map coordinate system at time (k+1) is (x k+1 ,y k+1 ,v k+1 ,θ k+1 ) T , the system state space equation can be specifically expressed as:

[0079]

[0080] Where Δt represents the time difference between time k and (k+1), θ is the heading angle of the vehicle, v is the speed of the vehicle, and θ k is the heading angle of the vehicle at time k, V k is the speed of the vehicle at time k, θ k and V k It is usually calculated from the coordinate position at the current moment and the previous moment, where θ can also be provided by the position information output by the GNSS navigation and positioning module 1 and the directional antenna. The position coordinates, speed and heading angle based on the high-precision map coordinate system directly provided by the first laser radar positioning module 11 and the GNSS navigation and positioning module 14 constitute the observation model of the vehicle posture, where x k represents the vehicle’s posture information at time k (x k ,y k ,v k ,θ k ) T , which specifically includes coordinate position, speed, and heading angle.

[0081] z k =Hx k +V k (6)

[0082] The state equation and observation equation of the fusion system can be expressed as:

[0083]

[0084] Among them, x k Represents the vehicle's posture information (x k ,y k ,v k ,θ k ) T , w k Satisfies the distribution w~N(0,Q), The prior estimate of the vehicle’s current state is obtained based on the best estimate at the previous moment, representing the vehicle’s current predicted posture information (x k ,y k ,v k ,θ k ) T , including the vehicle's position, speed and heading angle. Since there is no input from external information in the vehicle positioning system, B k is a zero matrix, A k The Jacobian matrix formed by the partial derivatives of each state quantity, A k It can be expressed as:

[0085]

[0086] When the first laser radar positioning module 11 fails to locate or there is no prior environmental map of the current environment, the online mapping (Online NDT) function of the second laser radar positioning module 12 can be used to generate a map of the current environment, and the updated map information can be used to estimate the current position. Positioning compensation is performed when the first laser radar positioning module 11 fails to locate. Specifically, the location information and confidence level of the evaluation provided by the GNSS navigation positioning module 14 can be combined with the IMU and wheel speed meter to complete the fusion positioning of the extended Kalman filter.

[0087] Among them, the core idea of ​​the Online NDT positioning technology is to divide the target point cloud (Target) into several grids (the projection of three-dimensional space on two-dimensional space), calculate the multidimensional normal distribution and probability density function of each grid, and when the source point cloud (Source) in the same coordinate system enters, the probability of each grid of the source point cloud in the corresponding grid is calculated according to the normal distribution parameters, and the probability P in all source point cloud grids is accumulated. When the probability P reaches the maximum, the optimal matching relationship is obtained, that is, the relative posture relationship between the source point cloud and the target point cloud is obtained. The first step of the Online NDT algorithm is to grid the target point cloud and calculate the probability density function of each point in the grid. For example, the following formula (9) is used to calculate the mean of the number of point clouds in each grid, where x i Represents the i-th point in the grid.

[0088]

[0089] The covariance matrix of each grid is calculated using the following formula (10), that is, the distribution discreteness in each grid:

[0090]

[0091] The probability density function of each grid is calculated using the following formula (11):

[0092]

[0093] The second step of the Online NDT algorithm is to solve the matching relationship between the original point cloud and the target point cloud, initialize the transformation matrix parameters, and have a transformation matrix in two-dimensional space. The three-dimensional space corresponds to P = [R0, t0]. For the point cloud (Source) that needs to be registered, it is converted into the grid space of the target point cloud (Target) through transformation. Confirm that the normal distribution corresponding to each mapping point in the point cloud corresponds to the grid in the Target. According to the solved transformation matrix P, the position and posture of the real-time point cloud of the LiDAR relative to the point cloud of the previous frame can be obtained.

[0094] When the GNSS navigation and positioning module 14 is affected by the multipath effect or environmental occlusion, the visual positioning module 13 obtains the surrounding image data and performs visual SLAM (Simultaneous Localization and Mapping), which, combined with the lane lines, IMU and wheel speed meter, can provide acceleration, angular velocity data and vehicle speed information, and complete the extended Kalman filter fusion positioning based on the evaluated confidence and NDT positioning.

[0095] The visual positioning module 13 includes five important components: camera data preprocessing module, front-end processing module, back-end processing module, loop detection module and mapping module. It mainly estimates the movement of the camera by quantitatively analyzing adjacent frame images, and splices the motion trajectories of these adjacent frames to form the overall motion trajectory of the autonomous driving vehicle, and finally generates a map to provide positioning services. The core principle of the visual positioning module 13 is to identify key feature points from multiple frame images, and parse the rotation transformation matrix and translation vector between each pair of adjacent frames to determine the geometric relationship between each pair of adjacent frames. Let any point in space be P, and its coordinates are expressed as [X, Y, Z] T , and then according to the pinhole camera mode, the spatial position of the corresponding pixel coordinates p1 and p2 in the two frames of images in the camera coordinate system can be inferred, which can be expressed as:

[0096] s1p1=KP (12)

[0097] s2p2=K(RP+t) (13)

[0098] Among them, s1 and s2 represent the z coordinates of the object in the camera coordinate system, p represents the coordinates of the image in the pixel coordinate system, K represents the intrinsic parameter matrix of the camera, and R represents the rotation relationship between the camera coordinate systems. The projection relationship can be expressed as:

[0099] p1≈KP (14)

[0100] p2≈K(RP+t) (15)

[0101] After transformation, the following polar geometric constraint relationship is obtained:

[0102]

[0103] By further simplifying the epipolar constraint, the matching relationship between two pixels can be derived, thereby calculating the change in the camera's posture and achieving positioning.

[0104] Therefore, according to the positioning system provided in this embodiment, when the signal of the GNSS navigation positioning module 14 is weak or missing, and the positioning of the first laser radar positioning module 11 fails, the second laser radar positioning module 12 and the visual positioning module 13 can be relied upon, combined with lane lines, IMU and wheel speed meter, to achieve dynamic environment map construction and real-time positioning, complete the fusion positioning of the EKF extended Kalman filter, and adjust the weight value in real time according to the positioning effect of each positioning module to obtain stable positioning accuracy and effect.

[0105] It can be understood that the first laser radar positioning module 11, the second laser radar positioning module 12, the visual positioning module 13, the GNSS navigation positioning module 14, the IMU auxiliary positioning module 16 and the speedometer auxiliary positioning module 17 in the present application all include corresponding sensors and data processing chips, and the sensors can be set in a three-dimensional spatial coordinate system.

[0106] Figure 3 For a schematic diagram of the distribution of sensors in the positioning system provided in the embodiment of the present application, please refer to Figure 3 As shown in the figure, the positioning system includes a first positioning antenna, a second positioning antenna, a laser radar and an inertial navigation combined positioning system, and an IMU and a wheel speed sensor (not shown in the figure) are also arranged on the vehicle body. Among them, the inertial navigation combined positioning system may include GPS (Global Positioning System) and IMU (Inertial Measuring Unit) and the like.

[0107] From the above description, it can be seen that the positioning system provided by the present application has the following technical advantages compared with the positioning system of a single sensor in the related art:

[0108] 1. The positioning system provided in this application can automatically select the corresponding positioning module according to the characteristics of the current application scenario, fuse the position information output by each selected positioning module, obtain the final positioning information, and improve the positioning accuracy of the vehicle.

[0109] 2. The positioning system of the present application fuses the position information output by multiple positioning modules, and sets different weight values ​​for each sensor during the fusion process. It utilizes the complementary advantages of each positioning module in different environments and scenarios to make up for the limitations of a single sensor. It avoids the situation where a single sensor may fail or provide erroneous information in certain environments, so that the positioning system can better adapt to various environmental changes, thereby improving the authenticity and robustness of the overall system.

[0110] Figure 4 This is a flow chart of a multi-sensor fusion positioning method provided in an embodiment of the present application, which is used to process the position information collected by each positioning module in the positioning system on the vehicle to obtain the final position information of the vehicle. The multi-sensor fusion positioning method provided in this embodiment can refer to the workflow of the positioning system provided in the above embodiment.

[0111] See also Figure 4 As shown, the multi-sensor fusion positioning method includes:

[0112] S401, when it is monitored that the first laser radar positioning module and the GNSS navigation positioning module are both operating normally, obtaining first position information collected by the first laser radar positioning module, fourth position information collected by the GNSS navigation positioning module, fifth position information collected by the IMU auxiliary positioning module, and sixth position information collected by the wheel speed meter auxiliary positioning module;

[0113] S402: Fusing the first position information, the fourth position information, the fifth position information, and the sixth position information to obtain final positioning information of the vehicle after fusion.

[0114] It is understandable that in other embodiments, the position information output by multiple positioning modules that are adapted to the current scene can be selected according to the characteristics of the current scene, and the multiple position information can be integrated to obtain the final positioning information of the vehicle.

[0115] Among them, the first position information, the fourth position information, the fifth position information and the sixth position information are fused to obtain the final positioning information of the fused vehicle, including: obtaining the weight value corresponding to each position information in the first position information, the fourth position information, the fifth position information and the sixth position information; according to the weight value corresponding to each position information, the first position information, the fourth position information, the fifth position information and the sixth position information are fused to obtain the final positioning information of the fused vehicle.

[0116] An embodiment of the present application also provides an autonomous driving vehicle, which includes the positioning system as described above. The positioning system can provide accurate positioning information in real time to assist the vehicle in completing autonomous driving.

[0117] The above embodiments can be implemented in whole or in part by software, hardware, firmware or any other combination. When implemented using software, the above embodiments can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded or executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from a website site, computer, server or data center to another website site, computer, server or data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) mode. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that contains one or more available media sets. The available medium can be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium. The semiconductor medium can be a solid state drive (SSD).

[0118] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the embodiments of the present application.

[0119] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0120] In the several embodiments provided in the embodiments of the present application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0121] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, and may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0122] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0123] If the function is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the embodiment of the present application is essentially or the part that contributes to the prior art or the part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions to enable a memory (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (Read-Only Memory, ROM), random access memory (RandomAccess Memory, RAM), disk or optical disk and other media that can store program code.

[0124] The above is only a specific implementation of the embodiment of the present application, but the protection scope of the embodiment of the present application is not limited thereto. Any technician familiar with the technical field can easily think of changes or replacements within the technical scope disclosed in the embodiment of the present application, which should be included in the protection scope of the embodiment of the present application. Therefore, the protection scope of the embodiment of the present application should be based on the protection scope of the claims.

Claims

1. A positioning system for autonomous driving, characterized in that: It includes a first laser radar positioning module, a second laser radar positioning module, a visual positioning module, a GNSS navigation positioning module and a positioning fusion module; The first laser radar positioning module is used to obtain first point cloud information of the vehicle in the current environment, and determine first position information of the vehicle in the current environment according to the first point cloud information and the prior map; The second laser radar positioning module is used to obtain second point cloud information of the vehicle in the current environment, and determine second position information of the vehicle in the current environment according to the online constructed map model and the second point cloud information; The visual positioning module is used to obtain image information of the vehicle in the current environment, and determine the third position information of the vehicle in the current environment according to the feature points in the image information; The GNSS navigation and positioning module is used to obtain fourth position information of the vehicle in the current environment; The positioning fusion module is used to fuse at least two of the first position information, the second position information, the third position information and the fourth position information to obtain the final positioning information of the vehicle after fusion.

2. The automatic driving positioning system according to claim 1, characterized in that: The positioning system also includes an IMU auxiliary positioning module and a wheel speed meter auxiliary positioning module; The IMU-assisted positioning module is used to obtain angular velocity and acceleration information of the vehicle, and determine fifth position information of the vehicle in the current environment according to the angular velocity and acceleration information; The wheel speed meter auxiliary positioning module is used to obtain wheel speed information of the vehicle and determine the sixth position information of the vehicle in the current environment according to the wheel speed information; The positioning fusion module is also used to fuse at least four of the first position information, the second position information, the third position information, the fourth position information, the fifth position information and the sixth position information to obtain the final positioning information of the vehicle after fusion.

3. The automatic driving positioning system according to claim 2, characterized in that: The positioning system also includes a positioning status monitoring module; The positioning status monitoring module is used to monitor the status information of the first laser radar positioning module and the GNSS navigation positioning module respectively, and when it is monitored that the first laser radar positioning module and the GNSS navigation positioning module are both operating normally, generate a first monitoring signal, and send the first monitoring signal to the positioning fusion module; The positioning fusion module is used to fuse the first position information, the fourth position information, the fifth position information and the sixth position information when receiving the first monitoring signal, so as to obtain the final positioning information of the vehicle after fusion.

4. The automatic driving positioning system according to claim 3, characterized in that: The positioning state monitoring module is further used to generate a second monitoring signal when detecting that the first laser radar positioning module fails, and send the second monitoring signal to the positioning fusion module; The positioning fusion module is used to fuse the second position information, the fourth position information, the fifth position information and the sixth position information when receiving the second monitoring signal, so as to obtain the final positioning information of the vehicle after fusion; Or the positioning fusion module is used to fuse the second position information, the third position information, the fourth position signal, the fifth position information and the sixth position information when receiving the second monitoring signal to obtain the final positioning information of the vehicle after fusion.

5. The automatic driving positioning system according to claim 3, characterized in that: The positioning status monitoring module is further configured to generate a third monitoring signal when detecting that the GNSS navigation and positioning module fails, and send the third monitoring signal to the positioning fusion module; The positioning fusion module is used to fuse the first position information, the third position information, the fifth position information and the sixth position information when receiving the third monitoring signal, so as to obtain the final positioning information of the vehicle after fusion; Or the positioning fusion module is used to fuse the first position information, the second position information, the third position information, the fifth position information and the sixth position information when receiving the third monitoring signal to obtain the final positioning information of the vehicle after fusion.

6. The automatic driving positioning system according to claim 3, characterized in that: The positioning status monitoring module is further configured to generate a fourth monitoring signal when it is detected that both the first laser radar positioning module and the GNSS navigation positioning module fail, and send the fourth monitoring signal to the positioning fusion module; The positioning fusion module is used to fuse the second position information, the third position information, the fifth position information and the sixth position information when receiving the fourth monitoring signal, so as to obtain the final positioning information of the vehicle after fusion.

7. The automatic driving positioning system according to claim 3, characterized in that: The fusing at least four of the first position information, the second position information, the third position information, the fourth position information, the fifth position information and the sixth position information to obtain the final positioning information of the vehicle after fusion, comprises: Obtain a weight value corresponding to each of the first location information, the second location information, the third location information, the fourth location information, the fifth location information, and the sixth location information; According to the weight values ​​corresponding to the respective position information, at least four of the first position information, the second position information, the third position information, the fourth position information, the fifth position information and the sixth position information are fused to obtain the final fused positioning information of the vehicle.

8. A multi-sensor fusion positioning method, used to process the position information collected by each positioning module in the positioning system on the vehicle to obtain the final position information of the vehicle, characterized in that: The positioning system is a positioning system as claimed in any one of claims 2 to 7; The multi-sensor fusion positioning method comprises: When it is monitored that the first laser radar positioning module and the GNSS navigation positioning module are both operating normally, obtaining the first position information collected by the first laser radar positioning module, the fourth position information collected by the GNSS navigation positioning module, the fifth position information collected by the IMU auxiliary positioning module, and the sixth position information collected by the wheel speed meter auxiliary positioning module; The first position information, the fourth position information, the fifth position information and the sixth position information are fused to obtain final fused positioning information of the vehicle.

9. The multi-sensor fusion positioning method according to claim 8, characterized in that: The first position information, the fourth position information, the fifth position information and the sixth position information are fused to obtain the final positioning information of the vehicle after fusion, including: Obtain a weight value corresponding to each of the first location information, the fourth location information, the fifth location information, and the sixth location information; According to the weight values ​​corresponding to the respective position information, the first position information, the fourth position information, the fifth position information and the sixth position information are fused to obtain the final fused positioning information of the vehicle.

10. An autonomous driving vehicle, characterized in that: Comprising a positioning system as described in any one of claims 1-7.

Images