Vehicle pose distortion processing method and device

Abstract

The invention provides a vehicle pose distortion processing method and device. The method comprises the following steps: acquiring vehicle environment information of a target vehicle; when it is judged that the target vehicle rapidly changes in the vertical direction according to the IMU detection information, first pose information of the target vehicle is calculated according to the laser point cloud information; calculating second pose information of the target vehicle according to the image information; calculating third pose information of the target vehicle according to the IMU detection information; performing fusion processing on the first pose information and the second pose information to obtain fused pose information; when it is judged that acceleration correction needs to be carried out on the IMU module of the target vehicle according to the fused pose information, acceleration correction processing is carried out on the IMU module according to the third pose information. According to the method and the device, the rapid change of the vehicle in the vertical direction can be effectively detected in real time, the acceleration of the IMU is corrected when the IMU drifts greatly, and the pose error of the vehicle is reduced, so that the normal operation and the safety of the vehicle are ensured.

Description

Technical Field

[0001] The present application relates to the field of vehicle detection technology, and in particular to a method and device for processing vehicle posture distortion. Background Art

[0002] In recent years, with the rapid development of unmanned driving technology, unmanned vehicle technology has shown its great application potential in many fields such as dock automation, service industry and warehouse management. However, unmanned vehicles often need to face complex and changeable environment and road conditions in the process of performing tasks, which puts forward more stringent requirements on various key technologies of unmanned vehicles, especially positioning and mapping technologies. In the existing methods, drift detection is performed according to the predicted posture of the current frame to preliminarily determine whether the unmanned vehicle has drifted; at the same time, the normal state detection is performed on the current frame to further confirm whether the unmanned vehicle is in normal operation. Based on the above detection results, the existing method will further perform corresponding state judgment on the current frame, and adopt means such as factor graph optimization processing to obtain the posture optimization result of the current frame. Subsequently, the posture prediction of the current radar point cloud frame and the NDT point cloud registration processing are performed according to the optimized posture results, so as to achieve precise control of the posture of the unmanned vehicle. However, in practice, it is found that existing methods lack a special detection mechanism for vertical changes of IMU sensors, and also lack a correction and feedback mechanism for IMU sensors, which will cause the drift to continue to increase, leading to large posture errors. Summary of the invention

[0003] The purpose of the embodiments of the present application is to provide a method and device for processing vehicle posture distortion, which can effectively detect the posture distortion problem caused by the rapid change of the vehicle in the vertical direction in real time, and at the same time correct the acceleration of the IMU when a large drift of the IMU is detected, thereby reducing the vehicle posture error and ensuring the normal operation and safety of the vehicle.

[0004] The first aspect of the present application provides a vehicle posture distortion processing method, comprising: Acquire vehicle environment information of the target vehicle; wherein the vehicle environment information at least includes laser point cloud information, image information and IMU detection information; When it is determined according to the IMU detection information that the target vehicle changes rapidly in the vertical direction, the first position information of the target vehicle is calculated according to the laser point cloud information; Calculating second posture information of the target vehicle according to the image information; Calculating the third posture information of the target vehicle according to the IMU detection information; Fusing the first posture information and the second posture information to obtain fused posture information; When it is determined according to the fused posture information that acceleration correction needs to be performed on the IMU module of the target vehicle, acceleration correction processing is performed on the IMU module according to the third posture information.

[0005] In the above implementation process, the method can first obtain the vehicle environment information of the target vehicle; when it is determined that the target vehicle changes rapidly in the vertical direction according to the IMU detection information, the first pose information of the target vehicle is calculated according to the laser point cloud information; then, the second pose information of the target vehicle is calculated according to the image information; and the third pose information of the target vehicle is calculated according to the IMU detection information; finally, the first pose information and the second pose information are fused to obtain the fused pose information; when it is determined that the IMU module of the target vehicle needs to be corrected for acceleration according to the fused pose information, the IMU module is corrected for acceleration according to the third pose information. It can be seen that the method can effectively detect the pose distortion problem caused by the rapid change of the vehicle in the vertical direction in real time, and at the same time, correct the acceleration of the IMU when a large drift of the IMU is detected, reduce the vehicle pose error, and thus ensure the normal operation and safety of the vehicle.

[0006] Furthermore, the obtaining of vehicle environment information of the target vehicle includes: Detect laser point cloud information through the laser radar on the target vehicle; Acquiring image information through a monocular camera on the target vehicle; Collecting IMU detection information through an IMU module on the target vehicle; wherein the IMU detection information includes at least acceleration information and angular velocity information; The laser point cloud information, the image information and the IMU detection information are aggregated to obtain environmental information.

[0007] Further, judging whether the target vehicle changes rapidly in the vertical direction according to the IMU detection information includes: Acquire acceleration information according to the IMU detection information; Calculating a change extreme value according to the acceleration information; Determine the value of the change degree flag bit according to the change extreme value; Determine the number of laser point cloud frames that need to be cut according to the value of the change degree flag; Determine whether the number of laser point cloud frames that need to be cut is greater than a preset number threshold; If yes, it is determined that the target vehicle undergoes a rapid change in the vertical direction; If not, it is determined that the target vehicle does not change rapidly in the vertical direction.

[0008] Further, the first pose information of the target vehicle is calculated according to the laser point cloud information, including: Perform two-dimensional projection based on the laser point cloud information to obtain a point cloud depth map; Preprocessing the point cloud depth map to obtain a preprocessed depth map; generating an optimized descriptor according to the preprocessed depth map; Calculating the centroid of all pixels in the optimized descriptor; Performing a centroid removal process on the optimized descriptor according to the centroid of all the pixel points to obtain position information of all the pixel points after the centroid removal; Constructing a Hessian matrix according to the position information; Perform SVD decomposition on the Hessian matrix to obtain the first position information of the target vehicle.

[0009] Further, the calculating the third posture information of the target vehicle according to the IMU detection information includes: Acquire acceleration information according to the IMU detection information; The third position information of the target vehicle is calculated according to the acceleration information.

[0010] Furthermore, the method further comprises: Determining the current moment position information of the target vehicle according to the fused position information; Calculating a drift value according to the current posture information and the third posture information; Determining whether the drift value is greater than a preset drift threshold; If yes, it is determined that the IMU module of the target vehicle needs to be corrected for acceleration, and the acceleration correction process of the IMU module is performed according to the third posture information; If not, it is determined that the IMU module of the target vehicle is operating normally and no acceleration correction is required.

[0011] Further, performing acceleration correction processing on the IMU module according to the third posture information includes: Calculating a correction amount in a vertical direction according to the current posture information and the third posture information; Acquire acceleration information according to the IMU detection information; Acquire the original acceleration of the IMU module in the vertical direction according to the acceleration information; The original acceleration is corrected according to the correction amount to obtain the corrected acceleration of the IMU in the vertical direction.

[0012] A second aspect of the present application provides a vehicle posture distortion processing device, the vehicle posture distortion processing device comprising: An acquisition unit, used to acquire vehicle environment information of a target vehicle; wherein the vehicle environment information at least includes laser point cloud information, image information and IMU detection information; A first calculation unit is used to calculate the first position information of the target vehicle according to the laser point cloud information when it is determined according to the IMU detection information that the target vehicle changes rapidly in the vertical direction; A second calculation unit, used for calculating second posture information of the target vehicle according to the image information; A third calculation unit, used for calculating third posture information of the target vehicle according to the IMU detection information; A fusion unit, used for fusing the first posture information and the second posture information to obtain fused posture information; An acceleration correction unit is used to perform acceleration correction processing on the IMU module according to the third posture information when it is determined that the IMU module of the target vehicle needs to be corrected according to the fused posture information.

[0013] Furthermore, the acquisition unit includes: A detection subunit, used for detecting laser point cloud information through a laser radar on a target vehicle; A first acquisition subunit, used for acquiring image information through a monocular camera on the target vehicle; A collection subunit, used for collecting IMU detection information through an IMU module on the target vehicle; wherein the IMU detection information includes at least acceleration information and angular velocity information; The aggregation subunit is used to aggregate the laser point cloud information, the image information and the IMU detection information to obtain vehicle environment information.

[0014] Furthermore, the vehicle posture distortion processing device also includes: The first judgment unit is used to judge whether the target vehicle changes rapidly in the vertical direction according to the IMU detection information.

[0015] Furthermore, the first determining unit includes: A second acquisition subunit, used to acquire acceleration information according to the IMU detection information; A first calculation subunit, configured to calculate a change extreme value according to the acceleration information; A determination subunit, used to determine the value of a change degree flag bit according to the change extreme value; The determination subunit is further used to determine the number of laser point cloud frames that need to be cut according to the value of the change degree flag; The judging subunit is used to judge whether the number of laser point cloud frames that need to be cut is greater than a preset number threshold; if so, it is determined that the target vehicle has undergone rapid changes in the vertical direction; if not, it is determined that the target vehicle has not undergone rapid changes in the vertical direction.

[0016] Furthermore, the first computing unit includes: A projection subunit, used for performing two-dimensional projection according to the laser point cloud information to obtain a point cloud depth map; A preprocessing subunit, used to preprocess the point cloud depth map to obtain a preprocessed depth map; A generating subunit, configured to generate an optimized descriptor according to the preprocessed depth map; A second calculation subunit, used to calculate the centroid of all pixels in the optimized descriptor; A centroid removal subunit, used for performing centroid removal processing on the optimized descriptor according to the centroids of all the pixels to obtain position information of all the pixels after the centroids are removed; A construction subunit, used for constructing a Hessian matrix according to the position information; The decomposition subunit is used to perform SVD decomposition on the Hessian matrix to obtain the first position information of the target vehicle.

[0017] Furthermore, the third calculation unit includes: A third acquisition subunit, used to acquire acceleration information according to the IMU detection information; The third calculation subunit is used to calculate the third posture information of the target vehicle according to the acceleration information.

[0018] Furthermore, the vehicle posture distortion processing device also includes: A determination unit, used to determine the current moment position and posture information of the target vehicle according to the fused position and posture information; a fourth calculation unit, configured to calculate a drift value according to the current posture information and the third posture information; The second judgment unit is used to judge whether the drift value is greater than a preset drift threshold; if so, it is determined that the IMU module of the target vehicle needs to be corrected by acceleration, and the acceleration correction unit is triggered to perform acceleration correction processing on the IMU module according to the third posture information; if not, it is determined that the IMU module of the target vehicle operates normally and does not need acceleration correction.

[0019] Furthermore, the acceleration correction unit comprises: a fourth calculation subunit, configured to calculate a correction amount in a vertical direction according to the current posture information and the third posture information; A fourth acquisition subunit, used to acquire acceleration information according to the IMU detection information; The fourth acquisition subunit is further used to acquire the original acceleration of the IMU module in the vertical direction according to the acceleration information; The correction subunit is used to correct the original acceleration according to the correction amount to obtain the corrected acceleration of the IMU in the vertical direction.

[0020] A third aspect of the present application provides an electronic device, including a memory and a processor, wherein the memory is used to store a computer program, and the processor runs the computer program to enable the electronic device to execute the vehicle posture distortion processing method described in any one of the first aspects of the present application.

[0021] A fourth aspect of the present application provides a computer-readable storage medium storing computer program instructions. When the computer program instructions are read and executed by a processor, the vehicle posture distortion processing method described in any one of the first aspect of the present application is executed.

[0022] The beneficial effects of the present application are as follows: in order to address the posture distortion problem caused by the rapid changes in the vertical direction of the unmanned vehicle, the method can design a shearing algorithm based on the acceleration information of the IMU to determine whether the unmanned vehicle has undergone rapid changes in the vertical direction, thereby improving the simplicity and speed of the calculation. At the same time, according to the speed of the unmanned vehicle in the vertical direction, the erroneous information is sheared to minimize unnecessary computing power consumption, while further reducing the hardware and software costs of the unmanned vehicle, thereby improving the calculation accuracy of the algorithm and avoiding the occurrence of stratification.

[0023] In addition, this method can reduce the resolution of the depth map generated by the laser point cloud through a local matching algorithm to generate an optimized descriptor, and then perform ICP matching on the pixels of the optimized descriptor, thereby reducing the amount of data used by an order of magnitude compared to traditional methods, thereby improving the real-time processing.

[0024] Finally, through the multi-sensor diagnostic algorithm, this method can determine whether there is a large drift in the IMU, and automatically correct the acceleration of the IMU if there is a large drift, thereby improving the accuracy of the algorithm and providing more accurate data support for the rapid changes of the unmanned vehicle in the vertical direction. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the embodiments of the present application will be briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present application and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.

[0026] Figure 1 A flow chart of a vehicle posture distortion processing method provided in an embodiment of the present application; Figure 2 A flowchart of another method for processing vehicle posture distortion provided in an embodiment of the present application; Figure 3 A multi-source SLAM system framework diagram for a vehicle posture distortion processing method provided in an embodiment of the present application; Figure 4 A schematic diagram of an example flow of a method for processing vehicle posture distortion provided in an embodiment of the present application; Figure 5 A schematic diagram of the structure of a vehicle posture distortion processing device provided in an embodiment of the present application; Figure 6 A schematic diagram of the structure of another vehicle posture distortion processing device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0027] The technical solutions in the embodiments of the present application will be described below in conjunction with the drawings in the embodiments of the present application.

[0028] It should be noted that similar reference numerals and letters represent similar items in the following drawings, so once an item is defined in one drawing, it does not need to be further defined and explained in subsequent drawings. At the same time, in the description of this application, the terms "first", "second", etc. are only used to distinguish the description and cannot be understood as indicating or implying relative importance.

[0029] Example 1 Please see Figure 1 , Figure 1 A flow chart of a method for processing vehicle posture distortion provided in this embodiment. The method for processing vehicle posture distortion includes: S101. Acquire vehicle environment information of a target vehicle; wherein the vehicle environment information at least includes laser point cloud information, image information, and IMU detection information.

[0030] In this embodiment, the method gives priority to obtaining environmental information from the unmanned vehicle's laser radar, monocular camera, and IMU. The laser radar obtains laser point cloud information, the monocular camera obtains image information, and the IMU obtains acceleration and angular velocity information (collectively referred to as IMU detection information).

[0031] S102: When it is determined based on the IMU detection information that the target vehicle changes rapidly in the vertical direction, the first position information of the target vehicle is calculated based on the laser point cloud information.

[0032] In this embodiment, the method designs a shearing algorithm based on the acceleration information of the IMU to determine whether the unmanned vehicle undergoes rapid changes in the vertical direction.

[0033] S103. Calculate second position information of the target vehicle according to the image information.

[0034] S104. Calculate the third posture information of the target vehicle according to the IMU detection information.

[0035] S105 , fusing the first pose information and the second pose information to obtain fused pose information.

[0036] In this embodiment, the method can solve the posture of the data obtained by the laser radar and the monocular camera respectively based on the local matching algorithm; and obtain the fused posture by fusing the postures obtained by the laser radar and the monocular camera.

[0037] S106. When it is determined based on the fused posture information that acceleration correction needs to be performed on the IMU module of the target vehicle, acceleration correction processing is performed on the IMU module based on the third posture information.

[0038] In this embodiment, the method can determine whether the IMU has a large drift based on a multi-sensor diagnostic algorithm; and correct the acceleration of the IMU when the IMU has a large drift.

[0039] In this embodiment, the execution subject of the method may be a computing device such as a computer or a server, and no limitation is made in this embodiment.

[0040] In this embodiment, the execution subject of the method may also be a smart device such as a smart phone, a tablet computer, etc., which is not limited in this embodiment.

[0041] It can be seen that the implementation of the vehicle posture distortion processing method described in this embodiment can effectively detect the posture distortion problem caused by the rapid change of the vehicle in the vertical direction in real time, and at the same time correct the acceleration of the IMU when a large drift of the IMU is detected, thereby reducing the vehicle posture error and ensuring the normal operation and safety of the vehicle.

[0042] Example 2 Please see Figure 2 , Figure 2 A flow chart of a method for processing vehicle posture distortion provided in this embodiment. The method for processing vehicle posture distortion includes: S201, detecting laser point cloud information through a laser radar on a target vehicle.

[0043] In this embodiment, the laser radar is a 3D laser radar, specifically a 64-beam laser radar, a frequency of 10 Hz, an angular resolution of 0.2°, and collects 10 frames of point cloud data per second.

[0044] In this embodiment, the laser point cloud information includes the distance information from the drone to the environment, and its arrangement is disorderly. The collected sensor data is mainly the distance D.

[0045] S202: Acquire image information through a monocular camera on the target vehicle.

[0046] In this embodiment, the monocular camera captures an RGB image of the environment, and grayscale processing of the RGB image can generate a grayscale image, and the value of each pixel represents the grayscale h.

[0047] S203, collecting IMU detection information through the IMU module on the target vehicle.

[0048] In this embodiment, the IMU detection information includes at least acceleration information and angular velocity information.

[0049] In this embodiment, the IMU module refers to an inertial measurement unit, which is composed of three single-axis accelerometers and three single-axis gyroscopes. The accelerometer detects the acceleration signal of the object in the independent three-axis of the carrier coordinate system, and the gyroscope detects the angular velocity signal of the carrier relative to the navigation coordinate system. After processing these signals, the position and rotation information of the carrier can be calculated.

[0050] S204, summarizing laser point cloud information, image information and IMU detection information to obtain environmental information.

[0051] In this embodiment, when the unmanned vehicle passes through a speed bump or other obstacle, the unmanned vehicle will rise or fall rapidly in the vertical direction. If point cloud registration is performed using the point cloud information during this period, serious errors will occur, so this situation needs to be handled.

[0052] S205. Acquire acceleration information according to IMU detection information.

[0053] S206. Calculate the extreme value of change according to the acceleration information.

[0054] In this embodiment, the method designs a shearing algorithm based on the acceleration information of the IMU.

[0055] In this embodiment, the shearing algorithm first uses the acceleration information of the IMU to calculate the extreme value of the change. The specific calculation formula is shown as follows:

[0056] Among them, U represents the extreme value of change, represents the acceleration of the unmanned vehicle on the z-axis at the current moment, t represents the current moment, represents the acceleration of the unmanned vehicle on the z-axis at the previous moment, and t-1 represents the previous moment.

[0057] S207, determining the value of the change degree flag according to the change extreme value, and determining the number of laser point cloud frames that need to be cut according to the value of the change degree flag.

[0058] In this embodiment, the extreme value of change can be used to determine whether the unmanned vehicle has undergone a significant change in the vertical direction. The degree of change is mainly divided into three types. The specific calculation formula is shown in the following formula:

[0059] Among them, FLAG represents the change degree flag, and U represents the change extreme value.

[0060] In this embodiment, when the change extreme value U is less than 1, it means that the degree of change of the unmanned vehicle in the vertical direction is very small, and the change degree flag FLAG is set to 0; when the change extreme value U is greater than or equal to 1 and less than 5, it means that the degree of change of the unmanned vehicle in the vertical direction is small, and the change degree flag FLAG is set to 1; when the change extreme value U is greater than or equal to 5, it means that the degree of change of the unmanned vehicle in the vertical direction is large, and the change degree flag FLAG is set to 2.

[0061] In this embodiment, the method can determine the degree of change of the unmanned vehicle in the vertical direction according to the value of the change degree flag. In order to reduce the interference of erroneous information on subsequent posture calculation, improve posture accuracy and reduce the amount of calculation, the laser point cloud needs to be processed. The specific processing formula is shown in the following formula:

[0062] Among them, FLAG represents the change degree flag, m represents the number of laser point cloud frames that need to be cut, and α represents the adjustment parameter, with an initial value of 0 and a value range of [0, 3].

[0063] If the value of the change degree flag FLAG is 0, the number of laser point cloud frames that need to be cut is 0, which means that the vertical change degree of the unmanned vehicle is very small at this time, and the current frame is retained for subsequent pose calculation; If the value of the change degree flag FLAG is 1, the number of laser point cloud frames that need to be cut is 3, which means that the change degree of the unmanned vehicle in the vertical direction is small at this time, and the current frame is retained for subsequent pose calculation; If the value of the change degree flag FLAG is 2, the number of laser point cloud frames that need to be cut is 5, which means that the degree of change of the unmanned vehicle in the vertical direction is large at this time, and the current frame is retained for subsequent pose calculation.

[0064] In this embodiment, if a large error is found during the mapping and positioning process, the method can further remove more laser point cloud frames by adjusting the value of the adjustment parameter, so as to reduce the impact of the erroneous information on the positioning and mapping accuracy.

[0065] S208, determining whether the number of laser point cloud frames to be cut is greater than a preset number threshold, if so, executing step 209; if not, terminating the process.

[0066] In this embodiment, if the number of laser point cloud frames that need to be cut is greater than a preset number threshold (such as m>3), it is determined that the target vehicle has undergone rapid changes in the vertical direction; if the number of laser point cloud frames that need to be cut is not greater than the preset number threshold, it is determined that the target vehicle has not undergone rapid changes in the vertical direction.

[0067] S209: Perform two-dimensional projection based on the laser point cloud information to obtain a point cloud depth map.

[0068] In this embodiment, the method can perform two-dimensional projection on the laser radar frame in the laser point cloud information to generate a 64*1800 depth map (ie, a point cloud depth map).

[0069] S210: pre-process the point cloud depth map to obtain a pre-processed depth map.

[0070] In this embodiment, for the laser radar, the number of laser points collected in one frame is 64*1800. In order to reduce the problem of distortion of laser points collected at the edge of the laser radar, the top and bottom two rows of laser points can be removed, and a 60*1800 depth map can be obtained.

[0071] S211 . Generate an optimized descriptor according to the preprocessed depth map.

[0072] In this embodiment, in order to improve the calculation efficiency and generate an optimized descriptor, the method can reduce the resolution of the 60*1800 depth map to generate a 15*90 optimized descriptor. The pixel point in the optimized descriptor is the average value of the depth of the four middle laser points in the 4*20 local map. The specific calculation formula is shown as follows:

[0073] Among them, d represents the depth of the pixel in the optimized descriptor, and d1, d2, d3, and d4 represent the depths of the four laser points in the middle of the 4*20 local image.

[0074] S212: Calculate the centroid of all pixels in the optimized descriptor.

[0075] In this embodiment, after obtaining the optimized descriptor, the method further uses the ICP algorithm to complete inter-frame matching to calculate the corresponding posture.

[0076] In this embodiment, the algorithm first calculates the centroid of all pixels of the current optimized descriptor and the previous frame optimized descriptor:

[0077]

[0078] in, , represents the centroid of the optimized descriptor, N represents the total number of pixels, j represents the sequence number, and Indicates different optimization descriptors.

[0079] S213, performing a centroid removal process on the optimized descriptor according to the centroids of all the pixels, and obtaining position information of all the pixels after the centroids are removed.

[0080] In this embodiment, the method continues to remove the centroids of the two optimized descriptors:

[0081]

[0082] in, and Represents the position information of all pixels after removing the centroid. and It represents the position information of all pixels after the centroid is removed, and i represents an integer.

[0083] S214. Construct a Hessian matrix according to the position information.

[0084] In this embodiment, the Hessian matrix H is constructed: ; in, and It represents the position information of all pixels after removing the centroid, N represents the total number of pixels, j represents the sequence number, and i represents an integer.

[0085] S215, performing SVD decomposition on the Hessian matrix to obtain the first position information of the target vehicle.

[0086] In this embodiment, the method can perform SVD decomposition on the matrix to obtain the position L (x l ,y l , z l ). Among them, L (x l ,y l , z l ) represents the translation position of the unmanned vehicle calculated by the laser radar, x l represents the x-axis coordinate of the unmanned vehicle obtained by the laser radar, y l represents the y-axis coordinate of the unmanned vehicle obtained by the laser radar, z l Represents the z-axis coordinate of the unmanned vehicle obtained by the laser radar.

[0087] S216. Calculate second position information of the target vehicle according to the image information.

[0088] In this embodiment, the method also uses the ICP algorithm to calculate the position C (x c ,y c , z c ). Among them, C (x c ,y c , z c ) represents the translation position of the unmanned vehicle calculated by the monocular camera, x c represents the x-axis coordinate of the unmanned vehicle obtained by the monocular camera, y c represents the y-axis coordinate of the unmanned vehicle obtained by the monocular camera, z c Represents the z-axis coordinate of the unmanned vehicle obtained by the monocular camera.

[0089] S217. Obtain acceleration information according to IMU detection information.

[0090] S218. Calculate the third posture information of the target vehicle according to the acceleration information.

[0091] S219, fusing the first pose information and the second pose information to obtain fused pose information.

[0092] In this embodiment, the method can fuse the postures obtained by the laser radar and the monocular camera to obtain filtered fused posture information.

[0093] In this embodiment, the posture obtained by the laser radar is set as the state quantity, and the posture obtained by the monocular camera is set as the measurement quantity. The above posture information is fused based on the update formula. The specific update formula is as follows: V L-C =A*L+B*C; C=H*C1; Among them, A represents the state transfer matrix, B represents the control input matrix, C represents the pose obtained by the monocular camera, C1 represents the pose obtained by the monocular camera at the previous moment, L represents the pose obtained by the lidar, H represents the measurement matrix, V L-C It represents the pose obtained after filtering and fusion, wherein the pose obtained after fusion includes the pose of the environment and the pose of the unmanned vehicle.

[0094] S220. Determine the current position and posture information of the target vehicle according to the fused position and posture information.

[0095] S221. Calculate the drift value according to the current posture information and the third posture information.

[0096] In this embodiment, the method proposes a multi-sensor diagnosis algorithm for determining whether the IMU has a large drift. Specifically, the algorithm is based on the fusion posture V L-C , we can get the current position of the unmanned vehicle By integrating the acceleration of IMU, the position and posture of the unmanned vehicle calculated by IMU can be obtained. .

[0097] S222: Determine whether the drift value is greater than a preset drift threshold. If so, execute step S223; if not, end this process.

[0098] In this embodiment, if the drift value is greater than the preset drift threshold, it is determined that acceleration correction is required for the IMU module of the target vehicle. If the drift value is not greater than the preset drift threshold, it is determined that the IMU module of the target vehicle is operating normally and no acceleration correction is required.

[0099] In this embodiment, the method can use the drift function to determine whether the IMU has a large drift. The calculation formula of the drift function K is as follows:

[0100] Among them, the initialization drift threshold is β; When K is greater than β, it indicates that the IMU has a large drift due to long-term operation and needs to correct the acceleration. Otherwise, it indicates that the IMU operates normally.

[0101] S223. Calculate the correction amount in the vertical direction according to the current posture information and the third posture information.

[0102] In this embodiment, since the vertical direction is only the z-axis, if the IMU has a large drift, the acceleration in the z-axis direction needs to be corrected. The derivative calculation formula is as follows:

[0103] in, Indicates the interval time between two laser point cloud frames.

[0104] S224. Obtain acceleration information according to IMU detection information.

[0105] S225. Obtain the original acceleration of the IMU module in the vertical direction according to the acceleration information.

[0106] S226. Correct the original acceleration according to the correction amount to obtain the corrected acceleration of the IMU in the vertical direction.

[0107] In this embodiment, this method can correct the original z-axis acceleration, and the calculation formula is shown as follows:

[0108] in, Indicates the corrected IMU acceleration on the z-axis, is the original acceleration on the z-axis.

[0109] Please see Figure 3 , Figure 3 The multi-source SLAM system framework diagram for applying a vehicle posture distortion processing method is shown. The data acquisition module is used to obtain the environmental information of the unmanned vehicle laser radar, monocular camera and IMU, wherein the laser radar obtains laser point cloud information, the monocular camera obtains image information, and the IMU obtains acceleration and angular velocity information; the shearing algorithm module is used to determine whether the unmanned vehicle has a rapid change in the vertical direction based on the acceleration information and shearing algorithm of the IMU; the posture calculation module is used to solve the posture of the data obtained by the laser radar and the monocular camera respectively based on the local matching algorithm, and fuse the posture obtained by the laser radar and the monocular camera to obtain the fused posture; the fault diagnosis and correction module is used to determine whether the IMU has a large drift based on the multi-sensor diagnosis algorithm, and correct the acceleration of the IMU when the IMU has a large drift.

[0110] Please see Figure 4 , Figure 4 The following is a schematic diagram of an example process of a method for processing vehicle posture distortion, wherein the example process can be applied to the multi-source SLAM system mentioned above to achieve the effects of fault diagnosis and information correction.

[0111] In this embodiment, the execution subject of the method may be a computing device such as a computer or a server, and no limitation is made in this embodiment.

[0112] In this embodiment, the execution subject of the method may also be a smart device such as a smart phone, a tablet computer, etc., which is not limited in this embodiment.

[0113] It can be seen that the implementation of the vehicle posture distortion processing method described in this embodiment can effectively detect the posture distortion problem caused by the rapid change of the vehicle in the vertical direction in real time, and at the same time correct the acceleration of the IMU when a large drift of the IMU is detected, thereby reducing the vehicle posture error and ensuring the normal operation and safety of the vehicle.

[0114] Example 3 Please see Figure 5 , Figure 5 FIG. 1 is a schematic diagram of the structure of a vehicle posture distortion processing device provided in this embodiment. Figure 5 As shown, the vehicle posture distortion processing device includes: The acquisition unit 310 is used to acquire vehicle environment information of the target vehicle; wherein the vehicle environment information at least includes laser point cloud information, image information and IMU detection information; The first calculation unit 320 is used to calculate the first position information of the target vehicle according to the laser point cloud information when it is determined according to the IMU detection information that the target vehicle changes rapidly in the vertical direction; A second calculation unit 330, used to calculate second position information of the target vehicle according to the image information; A third calculation unit 340, used to calculate third posture information of the target vehicle according to the IMU detection information; A fusion unit 350 is used to fuse the first pose information and the second pose information to obtain fused pose information; The acceleration correction unit 360 is used to perform acceleration correction processing on the IMU module according to the third posture information when it is determined that the IMU module of the target vehicle needs to be corrected according to the fused posture information.

[0115] In this embodiment, the explanation of the vehicle posture distortion processing device can refer to the description in Example 1 or Example 2, and will not be further elaborated in this embodiment.

[0116] It can be seen that the vehicle posture distortion processing device described in this embodiment can effectively detect the posture distortion problem caused by the rapid change of the vehicle in the vertical direction in real time, and at the same time correct the acceleration of the IMU when a large drift of the IMU is detected, thereby reducing the vehicle posture error and ensuring the normal operation and safety of the vehicle.

[0117] Example 4 Please see Figure 6 , Figure 6 FIG. 1 is a schematic diagram of the structure of a vehicle posture distortion processing device provided in this embodiment. Figure 6 As shown, the vehicle posture distortion processing device includes: The acquisition unit 310 is used to acquire vehicle environment information of the target vehicle; wherein the vehicle environment information at least includes laser point cloud information, image information and IMU detection information; The first calculation unit 320 is used to calculate the first position information of the target vehicle according to the laser point cloud information when it is determined according to the IMU detection information that the target vehicle changes rapidly in the vertical direction; A second calculation unit 330, used to calculate second position information of the target vehicle according to the image information; A third calculation unit 340, used to calculate third posture information of the target vehicle according to the IMU detection information; A fusion unit 350 is used to fuse the first pose information and the second pose information to obtain fused pose information; The acceleration correction unit 360 is used to perform acceleration correction processing on the IMU module according to the third posture information when it is determined that the IMU module of the target vehicle needs to be corrected according to the fused posture information.

[0118] As an optional implementation, the acquiring unit 310 includes: The detection subunit 311 is used to detect laser point cloud information through a laser radar on the target vehicle; A first acquisition subunit 312 is used to acquire image information through a monocular camera on the target vehicle; The acquisition subunit 313 is used to acquire IMU detection information through the IMU module on the target vehicle; wherein the IMU detection information at least includes acceleration information and angular velocity information; The aggregation subunit 314 is used to aggregate the laser point cloud information, image information and IMU detection information to obtain vehicle environment information.

[0119] As an optional implementation, the vehicle posture distortion processing device further includes: The first judgment unit 370 is used to judge whether the target vehicle has a rapid change in the vertical direction according to the IMU detection information.

[0120] As an optional implementation, 370 includes: The second acquisition subunit 371 is used to acquire acceleration information according to the IMU detection information; A first calculation subunit 372, configured to calculate a change extreme value according to acceleration information; A determination subunit 373, used to determine the value of the change degree flag bit according to the change extreme value; The determination subunit 373 is further used to determine the number of laser point cloud frames that need to be cut according to the value of the change degree flag; The judgment subunit 374 is used to judge whether the number of laser point cloud frames that need to be cut is greater than a preset number threshold; if so, it is determined that the target vehicle has undergone rapid changes in the vertical direction; if not, it is determined that the target vehicle has not undergone rapid changes in the vertical direction.

[0121] As an optional implementation, the first calculation unit 320 includes: The projection subunit 321 is used to perform two-dimensional projection according to the laser point cloud information to obtain a point cloud depth map; A preprocessing subunit 322 is used to preprocess the point cloud depth map to obtain a preprocessed depth map; A generating subunit 323, configured to generate an optimized descriptor according to the preprocessed depth map; A second calculation subunit 324 is used to calculate the centroid of all pixels in the optimized descriptor; A centroid removal subunit 325 is used to remove the centroid of the optimized descriptor according to the centroid of all pixels to obtain the position information of all pixels after the centroid is removed; A construction subunit 326, for constructing a Hessian matrix according to the position information; The decomposition subunit 327 is used to perform SVD decomposition on the Hessian matrix to obtain the first position information of the target vehicle.

[0122] As an optional implementation, the third calculation unit 340 includes: The third acquisition subunit 341 is used to acquire acceleration information according to the IMU detection information; The third calculation subunit 342 is used to calculate the third posture information of the target vehicle according to the acceleration information.

[0123] As an optional implementation, the vehicle posture distortion processing device further includes: A determination unit 380, configured to determine the current position and posture information of the target vehicle according to the fused position and posture information; The fourth calculation unit 390 is used to calculate the drift value according to the current posture information and the third posture information; The second judgment unit 400 is used to judge whether the drift value is greater than a preset drift threshold; if so, it is determined that the IMU module of the target vehicle needs to be corrected by acceleration, and the acceleration correction unit 360 is triggered to perform acceleration correction processing on the IMU module according to the third posture information; if not, it is determined that the IMU module of the target vehicle is operating normally and no acceleration correction is required.

[0124] As an optional implementation, the acceleration correction unit 360 includes: The fourth calculation subunit 361 is used to calculate the correction amount in the vertical direction according to the current posture information and the third posture information; The fourth acquisition subunit 362 is used to acquire acceleration information according to the IMU detection information; The fourth acquisition subunit 362 is further used to acquire the original acceleration of the IMU module in the vertical direction according to the acceleration information; The correction subunit 363 is used to correct the original acceleration according to the correction amount to obtain the corrected acceleration of the IMU in the vertical direction.

[0125] In this embodiment, the explanation of the vehicle posture distortion processing device can refer to the description in Example 1 or Example 2, and will not be further elaborated in this embodiment.

[0126] It can be seen that the vehicle posture distortion processing device described in this embodiment can effectively detect the posture distortion problem caused by the rapid change of the vehicle in the vertical direction in real time, and at the same time correct the acceleration of the IMU when a large drift of the IMU is detected, thereby reducing the vehicle posture error and ensuring the normal operation and safety of the vehicle.

[0127] An embodiment of the present application provides an electronic device, including a memory and a processor, wherein the memory is used to store a computer program, and the processor runs the computer program to enable the electronic device to execute the vehicle posture distortion processing method in embodiment 1 or embodiment 2 of the present application.

[0128] An embodiment of the present application provides a computer-readable storage medium storing computer program instructions. When the computer program instructions are read and executed by a processor, the vehicle posture distortion processing method in Embodiment 1 or Embodiment 2 of the present application is executed.

[0129] In several embodiments provided in the present application, it should be understood that the disclosed devices and methods can also be implemented in other ways. The device embodiments described above are merely schematic. For example, the flowcharts and block diagrams in the accompanying drawings show the possible architecture, functions and operations of the devices, methods and computer program products according to multiple embodiments of the present application. In this regard, each box in the flowchart or block diagram can represent a module, a program segment or a part of a code, and the module, a program segment or a part of a code contains one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two consecutive boxes can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flowchart, and the combination of boxes in the block diagram and / or flowchart can be implemented with a dedicated hardware-based system that performs a specified function or action, or can be implemented with a combination of dedicated hardware and computer instructions.

[0130] In addition, the functional modules in the various embodiments of the present application may be integrated together to form an independent part, or each module may exist separately, or two or more modules may be integrated to form an independent part.

[0131] If the function is implemented in the form of a software function module and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application can be essentially or partly embodied in the form of a software product that contributes to the prior art. The computer software product is stored in a storage medium and includes several instructions for a computer device (which can be a personal computer, server, or network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, and other media that can store program codes.

[0132] The above description is only an embodiment of the present application and is not intended to limit the scope of protection of the present application. For those skilled in the art, the present application may have various changes and variations. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present application should be included in the scope of protection of the present application. It should be noted that similar reference numerals and letters represent similar items in the following drawings, so once an item is defined in one drawing, it does not need to be further defined and explained in the subsequent drawings.

[0133] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

[0134] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.

Claims

1. A method for processing vehicle posture distortion, characterized in that: include: Acquire vehicle environment information of the target vehicle; wherein the vehicle environment information at least includes laser point cloud information, image information and IMU detection information; When it is determined according to the IMU detection information that the target vehicle changes rapidly in the vertical direction, the first position information of the target vehicle is calculated according to the laser point cloud information; Calculating second posture information of the target vehicle according to the image information; Calculating the third posture information of the target vehicle according to the IMU detection information; Fusing the first posture information and the second posture information to obtain fused posture information; When it is determined according to the fused posture information that acceleration correction needs to be performed on the IMU module of the target vehicle, acceleration correction processing is performed on the IMU module according to the third posture information.

2. The vehicle posture distortion processing method according to claim 1, characterized in that: The obtaining of vehicle environment information of the target vehicle includes: Detect laser point cloud information through the laser radar on the target vehicle; Acquiring image information through a monocular camera on the target vehicle; Collecting IMU detection information through an IMU module on the target vehicle; wherein the IMU detection information includes at least acceleration information and angular velocity information; The laser point cloud information, the image information and the IMU detection information are aggregated to obtain environmental information.

3. The vehicle posture distortion processing method according to claim 1, characterized in that: The method further comprises: Determine whether the target vehicle changes rapidly in the vertical direction according to the IMU detection information; Wherein, judging whether the target vehicle changes rapidly in the vertical direction according to the IMU detection information includes: Acquire acceleration information according to the IMU detection information; Calculating a change extreme value according to the acceleration information; Determine the value of the change degree flag bit according to the change extreme value; Determine the number of laser point cloud frames that need to be cut according to the value of the change degree flag; Determine whether the number of laser point cloud frames that need to be cut is greater than a preset number threshold; If yes, it is determined that the target vehicle undergoes a rapid change in the vertical direction; If not, it is determined that the target vehicle does not change rapidly in the vertical direction.

4. The vehicle posture distortion processing method according to claim 1, characterized in that: The step of calculating the first pose information of the target vehicle according to the laser point cloud information includes: Perform two-dimensional projection based on the laser point cloud information to obtain a point cloud depth map; Preprocessing the point cloud depth map to obtain a preprocessed depth map; generating an optimized descriptor according to the preprocessed depth map; Calculating the centroid of all pixels in the optimized descriptor; Performing a centroid removal process on the optimized descriptor according to the centroids of all the pixels to obtain position information of all the pixels after the centroids are removed; Constructing a Hessian matrix according to the position information; Perform SVD decomposition on the Hessian matrix to obtain the first position information of the target vehicle.

5. The vehicle posture distortion processing method according to claim 1, characterized in that: The calculating the third posture information of the target vehicle according to the IMU detection information includes: Acquire acceleration information according to the IMU detection information; The third position information of the target vehicle is calculated according to the acceleration information.

6. The vehicle posture distortion processing method according to claim 1, characterized in that: The method further comprises: Determining the current moment position information of the target vehicle according to the fused position information; Calculating a drift value according to the current posture information and the third posture information; Determining whether the drift value is greater than a preset drift threshold; If yes, it is determined that the IMU module of the target vehicle needs to be corrected for acceleration, and the acceleration correction process of the IMU module is performed according to the third posture information; If not, it is determined that the IMU module of the target vehicle is operating normally and no acceleration correction is required.

7. The vehicle posture distortion processing method according to claim 6, characterized in that: The performing acceleration correction processing on the IMU module according to the third posture information includes: Calculating a correction amount in a vertical direction according to the current posture information and the third posture information; Acquire acceleration information according to the IMU detection information; Acquire the original acceleration of the IMU module in the vertical direction according to the acceleration information; The original acceleration is corrected according to the correction amount to obtain the corrected acceleration of the IMU in the vertical direction.

8. A vehicle posture distortion processing device, characterized in that: The vehicle posture distortion processing device comprises: An acquisition unit, used to acquire vehicle environment information of a target vehicle; wherein the vehicle environment information at least includes laser point cloud information, image information and IMU detection information; A first calculation unit is used to calculate the first position information of the target vehicle according to the laser point cloud information when it is determined according to the IMU detection information that the target vehicle changes rapidly in the vertical direction; A second calculation unit, used for calculating second posture information of the target vehicle according to the image information; A third calculation unit, used for calculating third posture information of the target vehicle according to the IMU detection information; A fusion unit, used for fusing the first posture information and the second posture information to obtain fused posture information; An acceleration correction unit is used to perform acceleration correction processing on the IMU module according to the third posture information when it is determined that the IMU module of the target vehicle needs to be corrected according to the fused posture information.

9. An electronic device, characterized in that: The electronic device includes a memory and a processor, the memory is used to store a computer program, and the processor runs the computer program to enable the electronic device to execute the vehicle posture distortion processing method according to any one of claims 1 to 7.

10. A readable storage medium, characterized in that: The readable storage medium stores computer program instructions, and when the computer program instructions are read and executed by a processor, the vehicle posture distortion processing method described in any one of claims 1 to 7 is executed.

Images