A method and device for processing vehicle pose distortion

Abstract

The present application provides a method and device for processing vehicle pose distortion. The method includes: obtaining vehicle environment information of a target vehicle; when it is determined according to IMU detection information that the target vehicle undergoes a rapid change in the vertical direction, calculating first pose information of the target vehicle according to laser point cloud information; calculating second pose information of the target vehicle according to image information; calculating third pose information of the target vehicle according to IMU detection information; performing fusion processing on the first pose information and the second pose information to obtain fused pose information; when it is determined according to the fused pose information that acceleration correction needs to be performed on the IMU module of the target vehicle, performing acceleration correction processing on the IMU module according to the third pose information. The method and device can effectively detect the rapid change of the vehicle in the vertical direction in real time, correct the acceleration of the IMU when there is a large drift in the IMU, reduce the vehicle pose error, and thus ensure the normal operation and safety of the vehicle.

Description

Technical Field

[0001] The present application relates to the technical field of vehicle detection, and more particularly, to a method and device for processing vehicle pose distortion. Background Art

[0002] In recent years, with the rapid development of driverless technology, driverless vehicle technology has shown great application potential in many fields such as terminal automation operations, the service industry, and warehouse management. However, during the process of a driverless vehicle performing tasks, it often needs to face complex and ever-changing environmental and road condition challenges, which pose more stringent requirements on various key technologies of the driverless vehicle, especially positioning and mapping technologies. In existing methods, drift detection is performed based on the predicted pose of the current frame to preliminarily determine whether the driverless vehicle has experienced pose drift; at the same time, normal state detection is performed on the current frame to further confirm whether the driverless vehicle is in a normal operating state. Based on the above detection results, existing methods will further perform corresponding state judgments on the current frame and adopt means such as factor graph optimization processing to obtain the pose optimization result of the current frame. Subsequently, pose prediction of the current radar point cloud frame and NDT point cloud registration processing are performed according to the optimized pose result, so as to achieve precise control of the vehicle pose. However, it is found in practice that existing methods lack a special detection mechanism for changes in the vertical direction of the IMU sensor, and at the same time lack a correction and feedback mechanism for the IMU sensor, which will cause the problem of continuously increasing drift, resulting in a large pose error. Summary of the Invention

[0003] The purpose of the embodiments of the present application is to provide a method and device for processing vehicle pose distortion, which can effectively detect in real time the problem of vehicle pose distortion generated when the vehicle changes rapidly in the vertical direction, and at the same time correct the acceleration of the IMU when a large drift of the IMU is detected, reduce the vehicle pose error, and thus ensure the normal operation and safety of the vehicle.

[0004] The first aspect of the present application provides a method for processing vehicle pose distortion, including:

[0005] Obtain the vehicle environment information of the target vehicle; wherein, the vehicle environment information at least includes laser point cloud information, image information, and IMU detection information;

[0006] When it is determined according to the IMU detection information that the target vehicle changes rapidly in the vertical direction, calculate the first pose information of the target vehicle according to the laser point cloud information;

[0007] Calculate the second pose information of the target vehicle according to the image information;

[0008] Calculate the third pose information of the target vehicle according to the IMU detection information;

[0009] Fuse the first pose information and the second pose information to obtain fused pose information;

[0010] When it is determined according to the fused pose information that the acceleration of the IMU module of the target vehicle needs to be corrected, correct the acceleration of the IMU module according to the third pose information.

[0011] In the above implementation process, the method can first obtain the vehicle environment information of the target vehicle; when it is determined according to the IMU detection information that the target vehicle changes rapidly in the vertical direction, calculate the first pose information of the target vehicle according to the lidar point cloud information; then, calculate the second pose information of the target vehicle according to the image information; then, calculate the third pose information of the target vehicle according to the IMU detection information; finally, fuse the first pose information and the second pose information to obtain fused pose information; when it is determined according to the fused pose information that the acceleration of the IMU module of the target vehicle needs to be corrected, correct the acceleration of the IMU module according to the third pose information. It can be seen that the method can effectively detect the pose distortion problem generated when the vehicle changes rapidly in the vertical direction in real time, and at the same time correct the acceleration of the IMU when it is detected that the IMU has a large drift, reduce the vehicle pose error, so as to ensure the normal operation and safety of the vehicle.

[0012] Further, the obtaining of the vehicle environment information of the target vehicle includes:

[0013] Detect lidar point cloud information through a lidar on the target vehicle;

[0014] Obtain image information through a monocular camera on the target vehicle;

[0015] Collect IMU detection information through an IMU module on the target vehicle; wherein, the IMU detection information at least includes acceleration information and angular velocity information;

[0016] Summarize the lidar point cloud information, the image information and the IMU detection information to obtain environment information.

[0017] Further, the judging whether the target vehicle changes rapidly in the vertical direction according to the IMU detection information includes:

[0018] Obtain acceleration information according to the IMU detection information;

[0019] Calculate the change extreme value according to the acceleration information;

[0020] Determine the value of the change degree flag bit according to the change extreme value;

[0021] Determine the number of laser point cloud frames to be clipped according to the value of the change degree flag bit;

[0022] Judge whether the number of laser point cloud frames to be clipped is greater than a preset number threshold;

[0023] If so, determine that the target vehicle has changed rapidly in the vertical direction;

[0024] If not, determine that the target vehicle has not changed rapidly in the vertical direction.

[0025] Further, the calculating the first pose information of the target vehicle according to the laser point cloud information includes:

[0026] Perform two-dimensional projection on the laser point cloud information to obtain a point cloud depth map;

[0027] Preprocess the point cloud depth map to obtain a preprocessed depth map;

[0028] Generate an optimized descriptor according to the preprocessed depth map;

[0029] Calculate the centroid of all pixel points in the optimized descriptor;

[0030] Perform centroid removal processing on the optimized descriptor according to the centroid of all pixel points to obtain the position information of all pixel points after centroid removal;

[0031] Construct a Hessian matrix according to the position information;

[0032] Perform SVD decomposition on the Hessian matrix to obtain the first pose information of the target vehicle.

[0033] Further, the calculating the third pose information of the target vehicle according to the IMU detection information includes:

[0034] Obtain acceleration information according to the IMU detection information;

[0035] Calculate the third pose information of the target vehicle according to the acceleration information.

[0036] Further, the method further includes:

[0037] Determine the pose information of the target vehicle at the current moment according to the fused pose information;

[0038] Calculate a drift value according to the pose information at the current moment and the third pose information;

[0039] Judge whether the drift value is greater than a preset drift threshold;

[0040] If so, determine that acceleration correction needs to be performed on the IMU module of the target vehicle, and execute the acceleration correction process for the IMU module according to the third pose information.

[0041] If not, determine that the IMU module of the target vehicle is operating normally and acceleration correction is not required.

[0042] Further, the acceleration correction process for the IMU module according to the third pose information includes:

[0043] Calculate the correction amount in the vertical direction according to the current moment pose information and the third pose information.

[0044] Obtain the acceleration information according to the IMU detection information.

[0045] Obtain the original acceleration of the IMU module in the vertical direction according to the acceleration information.

[0046] Correct the original acceleration according to the correction amount to obtain the corrected acceleration of the IMU in the vertical direction.

[0047] The second aspect of the present application provides a vehicle pose distortion processing device, and the vehicle pose distortion processing device includes:

[0048] An acquisition unit, configured to acquire the vehicle environment information of the target vehicle; wherein, the vehicle environment information at least includes laser point cloud information, image information, and IMU detection information;

[0049] A first calculation unit, configured to calculate the first pose information of the target vehicle according to the laser point cloud information when it is determined according to the IMU detection information that the target vehicle changes rapidly in the vertical direction;

[0050] A second calculation unit, configured to calculate the second pose information of the target vehicle according to the image information;

[0051] A third calculation unit, configured to calculate the third pose information of the target vehicle according to the IMU detection information;

[0052] A fusion unit, configured to perform a fusion process on the first pose information and the second pose information to obtain fusion pose information;

[0053] An acceleration correction unit, configured to perform an acceleration correction process on the IMU module according to the third pose information when it is determined according to the fusion pose information that acceleration correction needs to be performed on the IMU module of the target vehicle.

[0054] Further, the acquisition unit includes:

[0055] A detection subunit, configured to detect lidar point cloud information through a lidar on a target vehicle;

[0056] A first acquisition subunit, configured to acquire image information through a monocular camera on the target vehicle;

[0057] An acquisition subunit, configured to acquire IMU detection information through an IMU module on the target vehicle; wherein, the IMU detection information at least includes acceleration information and angular velocity information;

[0058] A summarization subunit, configured to summarize the lidar point cloud information, the image information, and the IMU detection information to obtain vehicle environment information.

[0059] Further, the vehicle pose distortion processing device further includes:

[0060] A first judgment unit, configured to judge whether the target vehicle changes rapidly in the vertical direction according to the IMU detection information.

[0061] Further, the first judgment unit includes:

[0062] A second acquisition subunit, configured to acquire acceleration information according to the IMU detection information;

[0063] A first calculation subunit, configured to calculate a change extreme value according to the acceleration information;

[0064] A determination subunit, configured to determine the value of a change degree flag bit according to the change extreme value;

[0065] The determination subunit is further configured to determine the number of lidar point cloud frames to be clipped according to the value of the change degree flag bit;

[0066] A judgment subunit, configured to judge whether the number of lidar point cloud frames to be clipped is greater than a preset number threshold; if so, determine that the target vehicle changes rapidly in the vertical direction; if not, determine that the target vehicle does not change rapidly in the vertical direction.

[0067] Further, the first calculation unit includes:

[0068] A projection subunit, configured to perform two-dimensional projection according to the lidar point cloud information to obtain a point cloud depth map;

[0069] A preprocessing subunit, configured to preprocess the point cloud depth map to obtain a preprocessed depth map;

[0070] A generation subunit, configured to generate an optimized descriptor according to the preprocessed depth map;

[0071] A second calculation subunit, configured to calculate the centroid of all pixel points in the optimized descriptor;

[0072] A centroid removal subunit, configured to perform centroid removal processing on the optimized descriptor according to the centroid of all pixel points, to obtain the position information of all pixel points after centroid removal;

[0073] A construction subunit, configured to construct a Hessian matrix according to the position information;

[0074] A decomposition subunit, configured to perform SVD decomposition on the Hessian matrix, to obtain the first pose information of the target vehicle.

[0075] Further, the third calculation unit includes:

[0076] A third acquisition subunit, configured to acquire acceleration information according to the IMU detection information;

[0077] A third calculation subunit, configured to calculate the third pose information of the target vehicle according to the acceleration information.

[0078] Further, the vehicle pose distortion processing device further includes:

[0079] A determination unit, configured to determine the pose information of the target vehicle at the current moment according to the fused pose information;

[0080] A fourth calculation unit, configured to calculate a drift value according to the pose information at the current moment and the third pose information;

[0081] A second determination unit, configured to determine whether the drift value is greater than a preset drift threshold; if so, determine that acceleration correction needs to be performed on the IMU module of the target vehicle, and trigger the acceleration correction unit to perform acceleration correction processing on the IMU module according to the third pose information; if not, determine that the IMU module of the target vehicle operates normally and no acceleration correction is required.

[0082] Further, the acceleration correction unit includes:

[0083] A fourth calculation subunit, configured to calculate a correction amount in the vertical direction according to the pose information at the current moment and the third pose information;

[0084] A fourth acquisition subunit, configured to acquire acceleration information according to the IMU detection information;

[0085] The fourth acquisition subunit is further configured to acquire the original acceleration of the IMU module in the vertical direction according to the acceleration information;

[0086] A correction subunit, configured to correct the original acceleration according to the correction amount to obtain the acceleration of the corrected IMU in the vertical direction.

[0087] A third aspect of the present application provides an electronic device, including a memory and a processor. The memory is used to store a computer program, and the processor runs the computer program to enable the electronic device to execute the vehicle pose distortion processing method according to any one of the first aspects of the present application.

[0088] A fourth aspect of the present application provides a computer-readable storage medium, which stores computer program instructions. When the computer program instructions are read and run by a processor, the vehicle pose distortion processing method according to any one of the first aspects of the present application is executed.

[0089] The beneficial effects of the present application are as follows: For the problem of pose distortion caused by the rapid change of the unmanned vehicle in the vertical direction, this method can design a shear algorithm based on the acceleration information of the IMU to judge whether the unmanned vehicle changes rapidly in the vertical direction, thereby improving the simplicity and speed of calculation. At the same time, according to the rapidity of the unmanned vehicle in the vertical direction, the incorrect information is sheared, reducing unnecessary computing power consumption as much as possible, and further reducing the hardware and software costs of the unmanned vehicle, realizing the improvement of the algorithm calculation accuracy, and avoiding the generation of layering phenomena.

[0090] In addition, this method can reduce the resolution of the depth map generated by the laser point cloud through a local matching algorithm to generate an optimized descriptor, and then perform ICP matching on the pixel points of the optimized descriptor, so that the amount of data used can be reduced by one order of magnitude compared with the traditional method, thereby improving the real-time performance of the processing.

[0091] Finally, through the multi-sensor diagnostic algorithm, this method can judge whether there is a large drift in the IMU, and automatically correct the acceleration of the IMU when there is a large drift in the IMU, thereby improving the algorithm accuracy and providing more accurate data support for the rapid change of the unmanned vehicle in the vertical direction. Description of the Drawings

[0092] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following will briefly introduce the drawings required to be used in the embodiments of the present application. It should be understood that the following drawings only show some embodiments of the present application, and therefore should not be regarded as a limitation of the scope. For those of ordinary skill in the art, other related drawings can be obtained based on these drawings without creative efforts.

[0093] Figure 1 It is a schematic flowchart of a vehicle pose distortion processing method provided by an embodiment of the present application;

[0094] Figure 2 Schematic flowchart of another vehicle pose distortion processing method provided by an embodiment of the present application;

[0095] Figure 3 Multi-source SLAM system framework diagram of an application vehicle pose distortion processing method provided by an embodiment of the present application;

[0096] Figure 4 Schematic flowchart of an example of a vehicle pose distortion processing method provided by an embodiment of the present application;

[0097] Figure 5 Schematic structural diagram of a vehicle pose distortion processing device provided by an embodiment of the present application;

[0098] Figure 6 Schematic structural diagram of another vehicle pose distortion processing device provided by an embodiment of the present application. Detailed implementation manners

[0099] Next, the technical solutions in the embodiments of the present application will be described with reference to the accompanying drawings in the embodiments of the present application.

[0100] It should be noted that similar reference numerals and letters denote similar items in the following drawings. Therefore, once an item is defined in one drawing, it does not need to be further defined and explained in subsequent drawings. At the same time, in the description of the present application, the terms "first", "second", etc. are only used for differential description and cannot be construed as indicating or implying relative importance.

[0101] Embodiment 1

[0102] Please refer to Figure 1 , Figure 1 Schematic flowchart of a vehicle pose distortion processing method provided by this embodiment. Among them, the vehicle pose distortion processing method includes:

[0103] S101. Obtain the vehicle environment information of the target vehicle; wherein, the vehicle environment information at least includes lidar point cloud information, image information, and IMU detection information.

[0104] In this embodiment, the method preferentially obtains the environment information of the unmanned vehicle lidar, monocular camera, and IMU. Among them, the lidar obtains lidar point cloud information, the monocular camera obtains image information, and the IMU obtains acceleration and angular velocity information (collectively referred to as IMU detection information).

[0105] S102. When it is determined according to the IMU detection information that the target vehicle changes rapidly in the vertical direction, calculate the first pose information of the target vehicle according to the lidar point cloud information.

[0106] In this embodiment, the method designs a shear algorithm based on the acceleration information of the IMU to determine whether the unmanned vehicle changes rapidly in the vertical direction.

[0107] S103. Calculate the second pose information of the target vehicle according to the image information.

[0108] S104. Calculate the third pose information of the target vehicle according to the IMU detection information.

[0109] S105. Perform fusion processing on the first pose information and the second pose information to obtain the fused pose information.

[0110] In this embodiment, the method can be based on a local matching algorithm to separately solve the poses of the data obtained by the lidar and the monocular camera; and by fusing the poses obtained by the lidar and the monocular camera, the fused pose after fusion is obtained.

[0111] S106. When it is determined according to the fused pose information that the IMU module of the target vehicle needs to be corrected for acceleration, perform acceleration correction processing on the IMU module according to the third pose information.

[0112] In this embodiment, the method can be based on a multi-sensor diagnostic algorithm to determine whether there is a large drift in the IMU; and when there is a large drift in the IMU, correct the acceleration of the IMU.

[0113] In this embodiment, the execution subject of the method can be a computing device such as a computer or a server, and no limitation is made in this embodiment.

[0114] In this embodiment, the execution subject of the method can also be a smart device such as a smart phone or a tablet computer, and no limitation is made in this embodiment.

[0115] It can be seen that implementing the vehicle pose distortion processing method described in this embodiment can effectively detect in real time the pose distortion problem generated when the vehicle changes rapidly in the vertical direction, and at the same time correct the acceleration of the IMU when it is detected that there is a large drift in the IMU, reduce the vehicle pose error, so as to ensure the normal operation and safety of the vehicle.

[0116] Embodiment 2

[0117] Please refer to Figure 2 , Figure 2 which is a schematic flow chart of a vehicle pose distortion processing method provided in this embodiment. Among them, the vehicle pose distortion processing method includes:

[0118] S201. Detect the laser point cloud information through the lidar on the target vehicle.

[0119] In this embodiment, the lidar is a 3D lidar, specifically with 64 beams, a frequency of 10HZ, an angular resolution of 0.2°, and 10 frames of point cloud data are collected per second.

[0120] In this embodiment, the lidar point cloud information includes the distance information from the drone to the environment, and its arrangement is chaotic. The main sensor data collected is the distance D.

[0121] S202. Obtain image information through the monocular camera on the target vehicle.

[0122] In this embodiment, the monocular camera collects an environmental RGB image. Grayscale processing of the RGB image can generate a grayscale image, and the value of each pixel point represents the grayscale h.

[0123] S203. Collect IMU detection information through the IMU module on the target vehicle.

[0124] In this embodiment, the IMU detection information includes at least acceleration information and angular velocity information.

[0125] In this embodiment, the IMU module refers to an inertial measurement unit, which consists of three single-axis accelerometers and three single-axis gyroscopes. The accelerometers detect the acceleration signals of the object on the independent three axes of the carrier coordinate system, while the gyroscopes detect the angular velocity signals of the carrier relative to the navigation coordinate system. After processing these signals, the position and rotation information of the carrier can be calculated.

[0126] S204. Summarize the lidar point cloud information, image information, and IMU detection information to obtain environmental information.

[0127] In this embodiment, when the unmanned vehicle passes over a speed bump or other obstacles, the unmanned vehicle will quickly rise or fall in the vertical direction. If point cloud registration is performed using the point cloud information during this period, serious errors will occur. Therefore, this situation needs to be processed.

[0128] S205. Obtain acceleration information according to the IMU detection information.

[0129] S206. Calculate the extreme value of change according to the acceleration information.

[0130] In this embodiment, this method designs a shear algorithm based on the acceleration information of the IMU.

[0131] In this embodiment, this shear algorithm first calculates the extreme value of change using the acceleration information of the IMU. The specific calculation formula is as shown in the following formula:

[0132]

[0133] Among them, U represents the extreme value of change. The acceleration of the driverless vehicle in the z-axis at the current moment is represented, where t represents the current moment. The acceleration of the driverless vehicle in the z-axis at the previous moment is represented, where t-1 represents the previous moment.

[0134] S207. Determine the value of the change degree flag bit according to the extreme value of the change, and determine the number of laser point cloud frames to be clipped according to the value of the change degree flag bit.

[0135] In this embodiment, it can be judged whether the driverless vehicle has a large change in the vertical direction from the extreme value of the change. The change degree is mainly divided into three types, and the specific calculation formula is shown as follows:

[0136]

[0137] Among them, FLAG represents the change degree flag bit, and U represents the extreme value of the change.

[0138] In this embodiment, when the extreme value of the change U is less than 1, it indicates that the change degree of the driverless vehicle in the vertical direction is very small at this time, and the change degree flag bit FLAG is set to 0; when the extreme value of the change U is greater than or equal to 1 and less than 5, it indicates that the change degree of the driverless vehicle in the vertical direction is relatively small at this time, and the change degree flag bit FLAG is set to 1; when the extreme value of the change U is greater than or equal to 5, it indicates that the change degree of the driverless vehicle in the vertical direction is relatively large at this time, and the change degree flag bit FLAG is set to 2.

[0139] In this embodiment, this method can judge the change degree of the driverless vehicle in the vertical direction according to the value of the change degree flag bit. In order to reduce the interference of error information on the subsequent pose calculation, improve the pose accuracy and reduce the calculation amount. It is necessary to process the laser point cloud, and the specific processing formula is shown as follows:

[0140]

[0141] Among them, FLAG represents the change degree flag bit, m represents the number of laser point cloud frames to be clipped, α represents the adjustment parameter, the initial value is 0, and the value range is [0, 3].

[0142] If the value of the change degree flag bit FLAG is 0, the number of laser point cloud frames to be clipped is 0, indicating that the change degree of the driverless vehicle in the vertical direction is very small at this time, and the current frame is retained for subsequent pose calculation;

[0143] If the value of the change degree flag bit FLAG is 1, the number of laser point cloud frames to be clipped is 3, indicating that the change degree of the driverless vehicle in the vertical direction is relatively small at this time, and the current frame is retained for subsequent pose calculation;

[0144] If the value of the change degree flag FLAG is 2, the number of laser point cloud frames to be clipped is 5, indicating that the unmanned vehicle has a large change degree in the vertical direction at this time, and the current frame is retained for subsequent pose calculation.

[0145] In this embodiment, if a large error is found during the map building and positioning process, the method can further remove more laser point cloud frames by adjusting the value of the adjustment parameter, so as to reduce the influence of error information on the positioning and map building accuracy.

[0146] S208. Determine whether the number of laser point cloud frames to be clipped is greater than a preset number threshold. If so, execute step 209; if not, end this process.

[0147] In this embodiment, if the number of laser point cloud frames to be clipped is greater than the preset number threshold (such as m > 3), it is determined that the target vehicle has a rapid change in the vertical direction; if the number of laser point cloud frames to be clipped is not greater than the preset number threshold, it is determined that the target vehicle has not had a rapid change in the vertical direction.

[0148] S209. Perform two-dimensional projection on the laser point cloud information to obtain a point cloud depth map.

[0149] In this embodiment, the method can perform two-dimensional projection on the lidar frames in the laser point cloud information to generate a 64 * 1800 depth map (i.e., the point cloud depth map).

[0150] S210. Preprocess the point cloud depth map to obtain a preprocessed depth map.

[0151] In this embodiment, for the lidar, the number of laser points collected in one frame is 64 * 1800. To reduce the problem of distortion caused by the laser points collected at the edge of the lidar, the top and bottom two rows of laser points can be removed, and a 60 * 1800 depth map can be obtained.

[0152] S211. Generate an optimized descriptor based on the preprocessed depth map.

[0153] In this embodiment, to improve the calculation efficiency and generate an optimized descriptor, the method can reduce the resolution of the 60 * 1800 depth map to generate a 15 * 90 optimized descriptor. The pixel points in the optimized descriptor are the average of the depths of the middle four laser points in the 4 * 20 local map. The specific calculation formula is shown as follows:

[0154]

[0155] where d represents the depth of the pixel point in the optimized descriptor, d 1 、d 2 、d 3 、d 4Represents the depth of the four laser points in the middle of the 4*20 partial view.

[0156] S212. Calculate the centroid of all pixel points in the optimized descriptor.

[0157] In this embodiment, after obtaining the optimized descriptor, the method further uses the ICP algorithm to complete the inter-frame matching to calculate the corresponding pose.

[0158] In this embodiment, the algorithm first calculates the centroid of all pixel points of the current optimized descriptor and the previous frame's optimized descriptor:

[0159]

[0160]

[0161] Among them, , represents the centroid of the optimized descriptor, N represents the total number of pixel points, j represents the serial number, and represent different optimized descriptors.

[0162] S213. Perform centroid removal processing on the optimized descriptor according to the centroid of all pixel points to obtain the position information of all pixel points after centroid removal.

[0163] In this embodiment, the method continues to remove the centroid from the two optimized descriptors:

[0164]

[0165]

[0166] Among them, and represent the position information of all pixel points after centroid removal, and represent the position information of all pixel points before centroid removal, and i represents an integer.

[0167] S214. Construct the Hessian matrix according to the position information.

[0168] In this embodiment, construct the Hessian matrix H:

[0169] ;

[0170] Among them, and represent the position information of all pixel points after centroid removal, N represents the total number of pixel points, j represents the serial number, and i represents an integer.

[0171] S215. Perform SVD decomposition on the Hessian matrix to obtain the first pose information of the target vehicle.

[0172] In this embodiment, the method can perform SVD decomposition on the matrix to obtain the pose L (x l , y l , z l ) of the driverless vehicle. Among them, L (x l , y l , z l ) represents the translational position of the driverless vehicle calculated by the lidar, x l represents the coordinate of the x-axis of the driverless vehicle obtained by the lidar, y l represents the coordinate of the y-axis of the driverless vehicle obtained by the lidar, and z l represents the coordinate of the z-axis of the driverless vehicle obtained by the lidar.

[0173] S216. Calculate the second pose information of the target vehicle according to the image information.

[0174] In this embodiment, the method also uses the ICP algorithm to calculate the pose C (x c , y c , z c ) of the driverless vehicle for the grayscale image collected by the monocular camera. Among them, C (x c , y c , z c ) represents the translational position of the driverless vehicle calculated by the monocular camera, x c represents the coordinate of the x-axis of the driverless vehicle obtained by the monocular camera, y c represents the coordinate of the y-axis of the driverless vehicle obtained by the monocular camera, and z c represents the coordinate of the z-axis of the driverless vehicle obtained by the monocular camera.

[0175] S217. Obtain the acceleration information according to the IMU detection information.

[0176] S218. Calculate the third pose information of the target vehicle according to the acceleration information.

[0177] S219. Perform fusion processing on the first pose information and the second pose information to obtain the fused pose information.

[0178] In this embodiment, the method can fuse the poses obtained by the lidar and the monocular camera to obtain the filtered fused pose information.

[0179] In this embodiment, the pose obtained by the lidar is set as the state quantity, and the pose obtained by the monocular camera is set as the measurement quantity. Based on the update formula, the above pose information is fused. The specific update formula is as follows:

[0180] V L-C=A*L + B*C;

[0181] C = H*C 1 ;

[0182] Wherein, A represents the state transition matrix, B represents the control input matrix, C represents the pose obtained from the monocular camera, and C 1 represents the pose obtained from the monocular camera at the previous moment, L represents the pose obtained from the lidar, H represents the measurement matrix, and V L-C represents the pose obtained after filter fusion, wherein the pose obtained after fusion includes the environmental pose and the pose of the driverless vehicle.

[0183] S220. Determine the pose information of the target vehicle at the current moment according to the fused pose information.

[0184] S221. Calculate the drift value according to the pose information at the current moment and the third pose information.

[0185] In this embodiment, the method proposes a multi-sensor diagnosis algorithm for determining whether there is a large drift in the IMU. Specifically, based on the fused pose V L-C , the pose of the driverless vehicle at the current moment can be obtained , and the pose of the driverless vehicle calculated by the IMU can be obtained by integrating the acceleration of the IMU .

[0186] S222. Determine whether the drift value is greater than a preset drift threshold. If so, execute step S223; if not, end this process.

[0187] In this embodiment, if the drift value is greater than the preset drift threshold, it is determined that the acceleration of the IMU module of the target vehicle needs to be corrected. If the drift value is not greater than the preset drift threshold, it is determined that the IMU module of the target vehicle is operating normally and no acceleration correction is required.

[0188] In this embodiment, the method can use the drift function to determine whether there is a large drift in the IMU. The calculation formula of the drift function K is as follows:

[0189]

[0190] Wherein, the initial drift threshold is β;

[0191] When K is greater than β, it indicates that the IMU has a large drift due to long-term operation and the acceleration needs to be corrected. Otherwise, it indicates that the IMU is operating normally.

[0192] S223. Calculate the correction amount in the vertical direction according to the pose information at the current moment and the third pose information.

[0193] In this embodiment, since the vertical direction is only the z-axis, if there is a large drift in the IMU, it is necessary to correct the acceleration in the z-axis direction, and the correction amount The derivative calculation formula is as shown in the following formula:

[0194]

[0195] where represents the time interval between two laser point cloud frames.

[0196] S224. Obtain the acceleration information according to the IMU detection information.

[0197] S225. Obtain the original acceleration of the IMU module in the vertical direction according to the acceleration information.

[0198] S226. Correct the original acceleration according to the correction amount to obtain the corrected acceleration of the IMU in the vertical direction.

[0199] In this embodiment, this method can correct the acceleration of the original z-axis, and the calculation formula is as shown in the following formula:

[0200]

[0201] where represents the acceleration of the corrected IMU in the z-axis, is the original acceleration of the z-axis.

[0202] Please refer to Figure 3 , Figure 3 which shows a multi-source slam system framework diagram applying a vehicle pose distortion processing method. Among them, the data acquisition module is used to acquire the environmental information of the unmanned vehicle lidar, monocular camera, and IMU. Among them, the lidar obtains the laser point cloud information, the monocular camera obtains the image information, and the IMU obtains the acceleration and angular velocity information; the shear algorithm module is used to judge whether the unmanned vehicle changes rapidly in the vertical direction based on the acceleration information of the IMU and the shear algorithm; the pose calculation module is used to solve the poses of the data obtained by the lidar and the monocular camera respectively based on the local matching algorithm, and fuse the poses obtained by the lidar and the monocular camera to obtain the fused pose; the fault diagnosis and correction module is used to judge whether there is a large drift in the IMU based on the multi-sensor diagnosis algorithm, and correct the acceleration of the IMU when there is a large drift in the IMU.

[0203] Please refer to Figure 4 , Figure 4 which shows an example flow schematic diagram of a vehicle pose distortion processing method. Among them, this example flow can be applied and implemented in the above multi-source slam system to achieve the effects of fault diagnosis and information correction.

[0204] In this embodiment, the execution subject of the method may be a computing device such as a computer or a server, and no limitation is made in this embodiment.

[0205] In this embodiment, the execution subject of the method may also be a smart device such as a smart phone or a tablet computer, and no limitation is made in this embodiment.

[0206] It can be seen that implementing the vehicle pose distortion processing method described in this embodiment can effectively detect in real time the pose distortion problem generated when the vehicle changes rapidly in the vertical direction. At the same time, when it is detected that there is a large drift in the IMU, the acceleration of the IMU is corrected to reduce the vehicle pose error, thereby ensuring the normal operation and safety of the vehicle.

[0207] Embodiment 3

[0208] Please refer to Figure 5 , Figure 5 , which is a schematic structural diagram of a vehicle pose distortion processing device provided in this embodiment. As Figure 5 shown, the vehicle pose distortion processing device includes:

[0209] An acquisition unit 310, configured to acquire vehicle environment information of a target vehicle; wherein, the vehicle environment information includes at least laser point cloud information, image information, and IMU detection information;

[0210] A first calculation unit 320, configured to calculate first pose information of the target vehicle according to the laser point cloud information when it is determined according to the IMU detection information that the target vehicle changes rapidly in the vertical direction;

[0211] A second calculation unit 330, configured to calculate second pose information of the target vehicle according to the image information;

[0212] A third calculation unit 340, configured to calculate third pose information of the target vehicle according to the IMU detection information;

[0213] A fusion unit 350, configured to perform fusion processing on the first pose information and the second pose information to obtain fusion pose information;

[0214] An acceleration correction unit 360, configured to perform acceleration correction processing on the IMU module according to the third pose information when it is determined according to the fusion pose information that the IMU module of the target vehicle needs to be acceleration-corrected.

[0215] In this embodiment, the explanation of the vehicle pose distortion processing device may refer to the description in Embodiment 1 or Embodiment 2, and no further elaboration is made in this embodiment.

[0216] It can be seen that implementing the vehicle pose distortion processing device described in this embodiment can effectively detect in real time the pose distortion problem generated when the vehicle changes rapidly in the vertical direction. At the same time, when it is detected that there is a large drift in the IMU, the acceleration of the IMU is corrected to reduce the vehicle pose error, thereby ensuring the normal operation and safety of the vehicle.

[0217] Embodiment 4

[0218] Please refer to Figure 6 , Figure 6 which is a schematic structural diagram of a vehicle pose distortion processing device provided in this embodiment. As Figure 6 shown, the vehicle pose distortion processing device includes:

[0219] An acquisition unit 310, configured to acquire vehicle environment information of a target vehicle; wherein, the vehicle environment information at least includes lidar point cloud information, image information, and IMU detection information;

[0220] A first calculation unit 320, configured to calculate first pose information of the target vehicle according to the lidar point cloud information when it is determined according to the IMU detection information that the target vehicle changes rapidly in the vertical direction;

[0221] A second calculation unit 330, configured to calculate second pose information of the target vehicle according to the image information;

[0222] A third calculation unit 340, configured to calculate third pose information of the target vehicle according to the IMU detection information;

[0223] A fusion unit 350, configured to perform fusion processing on the first pose information and the second pose information to obtain fusion pose information;

[0224] An acceleration correction unit 360, configured to perform acceleration correction processing on the IMU module according to the third pose information when it is determined according to the fusion pose information that the IMU module of the target vehicle needs to be subjected to acceleration correction.

[0225] As an optional implementation manner, the acquisition unit 310 includes:

[0226] A detection subunit 311, configured to detect lidar point cloud information through a lidar on the target vehicle;

[0227] A first acquisition subunit 312, configured to acquire image information through a monocular camera on the target vehicle;

[0228] An acquisition subunit 313, configured to acquire IMU detection information through an IMU module on the target vehicle; wherein, the IMU detection information at least includes acceleration information and angular velocity information;

[0229] A summarization subunit 314, configured to summarize laser point cloud information, image information, and IMU detection information to obtain vehicle environment information.

[0230] As an alternative implementation, the vehicle pose distortion processing device further includes:

[0231] A first determination unit 370, configured to determine whether the target vehicle undergoes a rapid change in the vertical direction according to the IMU detection information.

[0232] As an alternative implementation, 370 includes:

[0233] A second acquisition subunit 371, configured to acquire acceleration information according to the IMU detection information;

[0234] A first calculation subunit 372, configured to calculate a change extreme value according to the acceleration information;

[0235] A determination subunit 373, configured to determine the value of a change degree flag bit according to the change extreme value;

[0236] The determination subunit 373 is further configured to determine the number of laser point cloud frames to be clipped according to the value of the change degree flag bit;

[0237] A judgment subunit 374, configured to judge whether the number of laser point cloud frames to be clipped is greater than a preset number threshold; if so, it is determined that the target vehicle undergoes a rapid change in the vertical direction; if not, it is determined that the target vehicle does not undergo a rapid change in the vertical direction.

[0238] As an alternative implementation, the first calculation unit 320 includes:

[0239] A projection subunit 321, configured to perform two-dimensional projection according to the laser point cloud information to obtain a point cloud depth map;

[0240] A preprocessing subunit 322, configured to preprocess the point cloud depth map to obtain a preprocessed depth map;

[0241] A generation subunit 323, configured to generate an optimized descriptor according to the preprocessed depth map;

[0242] A second calculation subunit 324, configured to calculate the centroid of all pixel points in the optimized descriptor;

[0243] A centroid removal subunit 325, configured to perform centroid removal processing on the optimized descriptor according to the centroid of all pixel points to obtain the position information of all pixel points after centroid removal;

[0244] A construction subunit 326, configured to construct a Hessian matrix according to the position information;

[0245] A decomposition subunit 327, configured to perform SVD decomposition on the Hessian matrix to obtain the first pose information of the target vehicle.

[0246] As an alternative implementation, the third calculation unit 340 includes:

[0247] A third acquisition subunit 341, configured to acquire acceleration information according to the IMU detection information;

[0248] A third calculation subunit 342, configured to calculate the third pose information of the target vehicle according to the acceleration information.

[0249] As an alternative implementation, the vehicle pose distortion processing device further includes:

[0250] A determination unit 380, configured to determine the pose information of the target vehicle at the current moment according to the fused pose information;

[0251] A fourth calculation unit 390, configured to calculate a drift value according to the pose information at the current moment and the third pose information;

[0252] A second determination unit 400, configured to determine whether the drift value is greater than a preset drift threshold; if so, determine that it is necessary to perform acceleration correction on the IMU module of the target vehicle, and trigger the acceleration correction unit 360 to perform acceleration correction processing on the IMU module according to the third pose information; if not, determine that the IMU module of the target vehicle operates normally and no acceleration correction is required.

[0253] As an alternative implementation, the acceleration correction unit 360 includes:

[0254] A fourth calculation subunit 361, configured to calculate a correction amount in the vertical direction according to the pose information at the current moment and the third pose information;

[0255] A fourth acquisition subunit 362, configured to acquire acceleration information according to the IMU detection information;

[0256] The fourth acquisition subunit 362 is further configured to acquire the original acceleration of the IMU module in the vertical direction according to the acceleration information;

[0257] A correction subunit 363, configured to correct the original acceleration according to the correction amount to obtain the corrected acceleration of the IMU in the vertical direction.

[0258] In this embodiment, the explanation of the vehicle pose distortion processing device may refer to the description in Embodiment 1 or Embodiment 2, and will not be elaborated herein.

[0259] It can be seen that implementing the vehicle pose distortion processing device described in this embodiment can effectively detect in real time the pose distortion problem generated when the vehicle changes rapidly in the vertical direction. At the same time, when it is detected that there is a large drift in the IMU, the acceleration of the IMU is corrected to reduce the vehicle pose error, thereby ensuring the normal operation and safety of the vehicle.

[0260] An embodiment of the present application provides an electronic device, including a memory and a processor. The memory is used to store a computer program, and the processor runs the computer program to enable the electronic device to execute the vehicle pose distortion processing method in Embodiment 1 or Embodiment 2 of the present application.

[0261] An embodiment of the present application provides a computer-readable storage medium, which stores computer program instructions. When the computer program instructions are read and run by a processor, the vehicle pose distortion processing method in Embodiment 1 or Embodiment 2 of the present application is executed.

[0262] In the several embodiments provided by the present application, it should be understood that the disclosed devices and methods can also be implemented in other ways. The device embodiments described above are merely illustrative. For example, the flowcharts and block diagrams in the accompanying drawings show the possible architectures, functions, and operations of devices, methods, and computer program products according to multiple embodiments of the present application. In this regard, each block in the flowchart or block diagram may represent a module, a program segment, or a part of code, and the module, program segment, or part of code contains one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the blocks may occur in a different order than marked in the accompanying drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in the reverse order, depending on the functions involved. It should also be noted that each block in the block diagram and / or flowchart, as well as the combination of blocks in the block diagram and / or flowchart, can be implemented by a dedicated hardware-based system for performing the specified functions or actions, or can be implemented by a combination of dedicated hardware and computer instructions.

[0263] In addition, in each embodiment of the present application, the functional modules can be integrated together to form an independent part, or each module can exist alone, or two or more modules can be integrated to form an independent part.

[0264] When the above-mentioned functions are implemented in the form of software function modules and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art or a part of this technical solution can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in various embodiments of this application. The aforementioned storage medium includes: various media such as USB flash drives, mobile hard disks, read-only memories (ROM, Read-Only Memory), random access memories (RAM, Random Access Memory), magnetic disks, or optical discs that can store program codes.

[0265] The above are only the embodiments of this application and are not used to limit the protection scope of this application. For those skilled in the art, this application can have various changes and modifications. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principle of this application shall be included in the protection scope of this application. It should be noted that similar reference numerals and letters represent similar items in the following drawings. Therefore, once an item is defined in one drawing, it does not need to be further defined and explained in subsequent drawings.

[0266] The above is only the specific implementation manner of this application, but the protection scope of this application is not limited thereto. Any person skilled in the art can easily think of changes or replacements within the technical scope disclosed by this application, and all should be covered by the protection scope of this application. Therefore, the protection scope of this application should be subject to the protection scope of the claims.

[0267] It should be noted that in this text, relational terms such as first and second are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the term "comprising", "including" or any other variant thereof is intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements not only includes those elements, but also includes other elements not expressly listed, or further includes elements inherent to such process, method, article or device. Without further limitations, an element defined by the statement "including a..." does not exclude the existence of additional identical elements in the process, method, article or device including the said element.

Claims

1. A method for processing vehicle posture distortion, characterized in that: include: Acquire vehicle environment information of the target vehicle; wherein the vehicle environment information at least includes laser point cloud information, image information and IMU detection information; Acquire acceleration information according to the IMU detection information; Calculating a change extreme value according to the acceleration information; Determine the value of the change degree flag bit according to the change extreme value; Determine the number of laser point cloud frames that need to be cut according to the value of the change degree flag; Determine whether the number of laser point cloud frames that need to be cut is greater than a preset number threshold; If yes, it is determined that the target vehicle has undergone a rapid change in the vertical direction; if no, it is determined that the target vehicle has not undergone a rapid change in the vertical direction; When it is determined according to the IMU detection information that the target vehicle changes rapidly in the vertical direction, the first position information of the target vehicle is calculated according to the laser point cloud information; Calculating second posture information of the target vehicle according to the image information; Calculating the third posture information of the target vehicle according to the IMU detection information; Fusing the first posture information and the second posture information to obtain fused posture information; When it is determined according to the fused posture information that acceleration correction needs to be performed on the IMU module of the target vehicle, acceleration correction processing is performed on the IMU module according to the third posture information.

2. The vehicle posture distortion processing method according to claim 1, characterized in that: The obtaining of vehicle environment information of the target vehicle includes: Detect laser point cloud information through the laser radar on the target vehicle; Acquiring image information through a monocular camera on the target vehicle; Collecting IMU detection information through an IMU module on the target vehicle; wherein the IMU detection information includes at least acceleration information and angular velocity information; The laser point cloud information, the image information and the IMU detection information are aggregated to obtain environmental information.

3. The vehicle posture distortion processing method according to claim 1, characterized in that: The step of calculating the first pose information of the target vehicle according to the laser point cloud information includes: Perform two-dimensional projection based on the laser point cloud information to obtain a point cloud depth map; Preprocessing the point cloud depth map to obtain a preprocessed depth map; generating an optimized descriptor according to the preprocessed depth map; Calculating the centroid of all pixels in the optimized descriptor; Performing a centroid removal process on the optimized descriptor according to the centroid of all the pixel points to obtain position information of all the pixel points after the centroid removal; Constructing a Hessian matrix according to the position information; Perform SVD decomposition on the Hessian matrix to obtain the first position information of the target vehicle.

4. The vehicle posture distortion processing method according to claim 1, characterized in that: The calculating the third posture information of the target vehicle according to the IMU detection information includes: Acquire acceleration information according to the IMU detection information; The third position information of the target vehicle is calculated according to the acceleration information.

5. The vehicle posture distortion processing method according to claim 1, characterized in that: The method further comprises: Determining the current moment position information of the target vehicle according to the fused position information; Calculating a drift value according to the current posture information and the third posture information; Determining whether the drift value is greater than a preset drift threshold; If yes, it is determined that the IMU module of the target vehicle needs to be corrected for acceleration, and the acceleration correction process of the IMU module is performed according to the third posture information; If not, it is determined that the IMU module of the target vehicle is operating normally and no acceleration correction is required.

6. The vehicle posture distortion processing method according to claim 5, characterized in that: The performing acceleration correction processing on the IMU module according to the third posture information includes: Calculating a correction amount in a vertical direction according to the current posture information and the third posture information; Acquire acceleration information according to the IMU detection information; Acquire the original acceleration of the IMU module in the vertical direction according to the acceleration information; The original acceleration is corrected according to the correction amount to obtain the corrected acceleration of the IMU in the vertical direction.

7. A vehicle posture distortion processing device, characterized in that: The vehicle posture distortion processing device comprises: An acquisition unit, used to acquire vehicle environment information of a target vehicle; wherein the vehicle environment information at least includes laser point cloud information, image information and IMU detection information; A first calculation unit is used to calculate the first position information of the target vehicle according to the laser point cloud information when it is determined according to the IMU detection information that the target vehicle changes rapidly in the vertical direction; A second calculation unit, used for calculating second posture information of the target vehicle according to the image information; A third calculation unit, used for calculating third posture information of the target vehicle according to the IMU detection information; A fusion unit, used for fusing the first posture information and the second posture information to obtain fused posture information; An acceleration correction unit, configured to perform acceleration correction processing on the IMU module according to the third posture information when it is determined that the IMU module of the target vehicle needs to be corrected according to the fused posture information; Wherein, the vehicle posture distortion processing device also includes: A first judgment unit is used to judge whether the target vehicle has a rapid change in the vertical direction according to the IMU detection information; Wherein, the first judgment unit includes: A second acquisition subunit is used to acquire acceleration information according to IMU detection information; A first calculation subunit, used for calculating the extreme value of change according to the acceleration information; A determination subunit, used to determine the value of the change degree flag bit according to the change extreme value; The determination subunit is further used to determine the number of laser point cloud frames that need to be cut according to the value of the change degree flag; The judging subunit is used to judge whether the number of laser point cloud frames that need to be cut is greater than a preset number threshold; if yes, it is determined that the target vehicle has undergone a rapid change in the vertical direction; if no, it is determined that the target vehicle has not undergone a rapid change in the vertical direction.

8. An electronic device, characterized in that: The electronic device includes a memory and a processor, the memory is used to store a computer program, and the processor runs the computer program to enable the electronic device to execute the vehicle posture distortion processing method according to any one of claims 1 to 6.

9. A readable storage medium, characterized in that: The readable storage medium stores computer program instructions, and when the computer program instructions are read and executed by a processor, the vehicle posture distortion processing method described in any one of claims 1 to 6 is executed.

Images