Vehicle posture correction method and system based on deep learning

Abstract

The present invention relates to the technical field of vehicle posture correction, and in particular to a vehicle posture correction method and system based on deep learning. The present invention proposes the following scheme: a nine-axis gyroscope sensor fixed at the bottom of the truck center console is used to collect and correct the truck motion data, and the data is processed in combination with a Kalman filter algorithm to accurately identify the current posture of the vehicle, and a camera and a laser radar are used to identify the road conditions ahead, such as curves and road conditions. The posture data is combined to evaluate the stability risk of the cargo. If there is a risk, the vehicle posture will be corrected to ensure driving safety. The scheme is suitable for improving the driving safety of heavy trucks and reducing accidents caused by cargo instability.

Description

Technical Field

[0001] The present invention relates to the technical field of vehicle posture correction, and in particular to a vehicle posture correction method and system based on deep learning. Background Art

[0002] In the field of modern transportation and logistics, the safety monitoring of heavy trucks has always been a key link in improving driving safety and ensuring the integrity of goods. Vehicle posture recognition, as one of the core technologies, is of great significance for preventing rollovers and reducing cargo losses. Traditional posture recognition technologies mostly rely on simple sensor data, such as accelerometers and gyroscopes, but these methods have limitations in accuracy and reliability, especially under complex road conditions. With the development of deep learning technology, its powerful data processing and pattern recognition capabilities provide new solutions for the stability assessment of transported goods. However, existing technologies still have shortcomings such as inaccurate error correction and incomplete road condition recognition, and further innovation and improvement are urgently needed.

[0003] For example, a Chinese patent with authorization announcement number CN110515110B provides a method, device, equipment and computer-readable storage medium for data evaluation in the field of unmanned driving. The method includes generating a reference position posture data set of the vehicle based at least on satellite navigation data related to the vehicle. The method also includes generating a test position posture data set of the vehicle based at least on a part of the satellite navigation data, and the accuracy of the test position posture data set is lower than the reference position posture data set. The method also includes obtaining a correction position posture data set for the test position posture data set. The method also includes evaluating the correction performance for the test position posture data set based on the reference position posture data set and the correction position posture data set. The disclosed embodiment can achieve efficient and rapid evaluation of the correction result.

[0004] The above patents all have the problems raised by this background technology: they do not consider the impact of the gravity distribution of goods on the stability of goods during truck transportation. Under complex and changeable road conditions, such as sharp turns and uneven roads, the vehicle posture cannot be corrected in time, affecting the stability of cargo transportation. To solve the above problems, this application designs a vehicle posture correction method and system based on deep learning. Summary of the invention

[0005] The technical problem to be solved by the present invention is to address the shortcomings of the existing technology and provide a vehicle posture correction method and system based on deep learning. The nine-axis gyroscope sensor fixed at the bottom of the truck's center console collects and corrects the truck's motion data, and processes the data in combination with the Kalman filter algorithm to accurately identify the vehicle's current posture. The camera and lidar are used to identify the road conditions ahead, such as curves and road conditions, and the posture data is combined to evaluate the stability risk of the cargo. If there is a risk, the vehicle posture will be corrected to ensure driving safety. The system is suitable for improving the driving safety of heavy trucks and reducing accidents caused by cargo instability.

[0006] To achieve the above object, the present invention provides the following technical solutions:

[0007] A vehicle posture correction method based on deep learning, the method comprising:

[0008] The nine-axis gyroscope sensor is used to collect the truck's motion data and identify the current posture data;

[0009] Identify the road conditions ahead of the truck while it is driving, and determine whether there is a stability risk for the cargo based on the road conditions ahead and current posture data combined with gravity distribution parameters;

[0010] If there is a stability risk with the cargo, it will be displayed through the on-board system and the driving posture of the truck will be corrected.

[0011] The nine-axis gyroscope sensor includes a three-axis accelerometer, a three-axis gyroscope and a three-axis magnetometer. The truck motion data includes acceleration data, angular velocity data and magnetic field data. The identification of current posture data includes motion data correction and posture data identification. The specific steps of posture data identification are:

[0012] Convert the truck motion data from the carrier coordinate system to the navigation coordinate system and calculate the acceleration attitude value , magnetic attitude value and the initial quaternion ;

[0013] The acceleration attitude value, magnetic attitude value and initial quaternion are processed by a Kalman filter to calculate an updated quaternion, and the current attitude data is calculated according to the initial quaternion and the updated quaternion. The calculation formula of the current attitude data is:

[0014] ,

[0015] in, Indicates the current posture data, represents the similarity function, Indicates updating the quaternion. represents the Jacobian matrix and T represents the matrix transpose.

[0016] The identification of the road condition ahead of the truck during driving includes curve state identification and road surface condition identification. The curve state identification obtains the image ahead of the truck through an image sensor, extracts a lane line set according to the Hough transform, identifies the lane lines of the truck through a parameter voting algorithm, and calculates the curvature of the lane lines. The road surface state identification obtains the three-dimensional point cloud ahead of the truck through a laser radar, performs random consistency sampling, plane fitting, discrete degree calculation and cluster analysis on the three-dimensional point cloud data, and calculates the geometric parameters of the point cloud data.

[0017] The specific steps of the curve state recognition are as follows:

[0018] The image sensor installed on the front of the truck collects multi-view images of the road ahead;

[0019] The lane lines are detected by edge detection algorithm and cascade residual regression tree to obtain a lane line set, and the lane line in front of the truck is extracted from the lane line set by parameter voting algorithm;

[0020] The lane line in front of the truck is optimized by the RANSAC algorithm, and the lane line is fitted by the cubic spline interpolation method to obtain the lane line curve. The road curvature is calculated according to the lane line curve. The calculation formula of the road curvature is:

[0021] ,

[0022] in, represents the road curvature, represents the first-order derivative value of the lane curve, Represents the second-order derivative value of the lane curve;

[0023] The road curvature is compared with the curve curvature threshold. If it is greater than or equal to the curve curvature threshold, it means that the curve ahead of the truck is a sharp curve. If it is less than the curve curvature threshold, it means that the curve ahead of the truck is a gentle curve.

[0024] The specific steps of road surface state recognition are as follows:

[0025] The three-dimensional point cloud data is randomly sampled by random consistency sampling to obtain the road surface plane equation, and the laser points that are greater than or equal to 2 cm away from the road surface plane equation are marked as observed point cloud data;

[0026] The neighborhood points of the observed point cloud data are calculated according to the Lagrange multiplier method, the observed point cloud data and the neighborhood points are discretely determined, and the discrete degree of the observed point cloud data is calculated. The calculation formula for the discrete degree of the observed point cloud data is:

[0027] ,

[0028] in, represents the discrete degree of the observed point cloud data, A represents the covariance matrix of the observed point cloud data, n represents the normal vector of the observed point cloud data, represents the arccosine function, j represents a single neighborhood point of the observed point cloud data, M represents the total number of neighborhood points of the observed point cloud data, Represents the normal vector of the jth neighborhood point of the observed point cloud data, represents the maximum value function, Represents the Euclidean distance between the observed point cloud data and the jth neighboring point of the observed point cloud data;

[0029] Perform coplanarity judgment and cluster analysis on the observation point cloud data, merge the observation point cloud data that meet the coplanarity conditions and belong to the same category and allocate point set space, distinguish the laser points belonging to different road conditions into different categories, and obtain the laser point set;

[0030] The geometric parameters of the classes in the laser point set are calculated, and the road surface state is output according to the geometric parameters.

[0031] The gravity distribution parameters are obtained by modeling the gravity distribution of the truck, including cargo mass, cargo center of gravity and cargo size. The method for judging whether the cargo has stability risk includes constructing a risk assessment network, taking the road condition ahead, current posture data and gravity distribution parameters as input parameters of the risk assessment network, training the input parameters through the risk assessment network, and outputting a risk assessment result.

[0032] The risk assessment network includes an input layer, a hidden layer and an output layer;

[0033] The input layer is used to fuse the front road condition and the current posture data through a normalized embedded Gaussian function to predict the dynamic response of the truck;

[0034] The hidden layer is used to extract the characteristics of gravity distribution parameters, calculate the cargo force analysis results, and calculate the cargo movement probability based on the dynamic response of the truck and the cargo force analysis results;

[0035] The output layer is used to convert the cargo movement probability into a risk probability value through an activation function.

[0036] A vehicle posture correction system based on deep learning, the system comprising a posture data recognition module, a road condition recognition module, a stability assessment module, a risk warning module and a posture correction module;

[0037] The posture data recognition module is configured with a posture recognition strategy, and the posture recognition strategy is used to recognize the current posture of the truck based on the truck motion data;

[0038] The road condition recognition module is configured with a curve state recognition strategy and a road surface state recognition strategy. The curve state recognition strategy is used to calculate the lane line curvature and determine the curve state according to the lane line curve. The road surface state recognition strategy is used to calculate the geometric parameters of the point cloud data and determine the road surface state according to the geometric parameters.

[0039] The stability assessment module is used to construct a risk assessment network, taking the road conditions ahead, current posture data and gravity distribution parameters as input parameters, training the input parameters through the risk assessment network, and outputting risk assessment results;

[0040] The risk prompt module is used to display stability risks through the vehicle system, including risk prompts and warnings;

[0041] The posture correction module is used to correct the driving posture of the truck, including speed control, lane adjustment and posture control;

[0042] The posture data recognition module comprises:

[0043] Data acquisition unit: uses a nine-axis gyroscope sensor to collect the truck's motion data in real time, including three-axis acceleration data, three-axis angular velocity data, and three-axis magnetic field data;

[0044] Data conversion and correction unit: This unit converts the truck motion data collected in the carrier coordinate system into data in the navigation coordinate system, and calculates the acceleration attitude value, magnetic attitude value and initial quaternion;

[0045] Attitude calculation unit: The Kalman filter is used to calculate and update the quaternion, and the current attitude data is calculated in combination with the initial quaternion;

[0046] The road condition recognition module includes a curve state recognition unit and a road surface condition recognition unit, the curve state recognition strategy includes a curve recognition logic, the curve recognition logic is configured in the curve state recognition unit, and the road surface condition recognition strategy includes a road surface recognition logic, the road surface recognition logic is configured in the road surface condition recognition unit;

[0047] The curve state recognition unit acquires the image in front of the truck through an image sensor, extracts a lane line set using Hough transform, recognizes the lane line through a parameter voting algorithm, and finally calculates the curvature of the lane line to determine the state of the curve ahead;

[0048] The road condition recognition unit uses a laser radar to obtain the front three-dimensional point cloud data, performs random consistency sampling, plane fitting, discrete degree calculation and cluster analysis, and calculates geometric parameters to identify the road condition;

[0049] The stability assessment module comprises:

[0050] Gravity distribution parameter modeling unit: by modeling the gravity distribution of the truck, the cargo mass, center of gravity position and size parameters are obtained;

[0051] Risk assessment network construction unit: constructs a risk assessment network consisting of an input layer, a hidden layer, and an output layer, which is used to receive the road conditions ahead, current posture data, and gravity distribution parameters as inputs, and outputs the stability risk assessment results of the cargo through training;

[0052] The risk warning module includes:

[0053] Display unit: when the output result of the stability assessment module indicates that there is a stability risk, the corresponding prompt information, including the risk level and warning signal, is displayed on the interface of the vehicle system;

[0054] Audio warning unit: issues warning signals via the audio system to alert the driver to possible risk situations;

[0055] The posture correction module comprises:

[0056] Speed ​​control unit: adjusts vehicle speed according to stability risk assessment results, slowing down or increasing speed to reduce rollover risk;

[0057] Lane adjustment unit: When driving on a curve, the lane position is adjusted according to the identified curve status to ensure safe passage through the curve;

[0058] Posture control unit: controls the vehicle's posture by adjusting the suspension system, braking system, etc. to ensure cargo stability and driving safety.

[0059] Compared with the prior art, the present invention has the following beneficial effects:

[0060] 1. The present invention uses dual-mode sensors of cameras and lidars to fuse image and point cloud data through a deep learning model, thus achieving all-round and high-precision recognition of the road conditions ahead. This technology can not only identify a variety of curves and road conditions, but also dynamically adjust the recognition strategy in real time to adapt to different driving environments, thereby improving the accuracy of cargo stability risk prediction;

[0061] 2. The present invention adapts to different loading conditions by combining the gravity distribution parameters of the cargo in the truck, whether the cargo is evenly distributed or asymmetrically distributed, and provides corresponding stability evaluation. BRIEF DESCRIPTION OF THE DRAWINGS

[0062] Other features, objects and advantages of the present invention will become more apparent from the detailed description of non-limiting embodiments made with reference to the following drawings:

[0063] Figure 1This is a flow chart of a vehicle posture correction method based on deep learning according to Embodiment 1 of the present invention;

[0064] Figure 2 This is a flow chart of posture data recognition in Example 1 of the present invention;

[0065] Figure 3 This is a flow chart of a curve recognition method according to Embodiment 1 of the present invention;

[0066] Figure 4 This is a flow chart of a road condition identification method according to Embodiment 1 of the present invention;

[0067] Figure 5 This is a risk assessment network structure diagram of Example 1 of the present invention;

[0068] Figure 6 This is a module diagram of a vehicle posture correction system based on deep learning according to Example 2 of the present invention. DETAILED DESCRIPTION

[0069] The technical solutions in the embodiments of the present invention will be described clearly and completely below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all the embodiments.

[0070] Example 1

[0071] See also Figure 1 , an embodiment provided by the present invention: a vehicle posture correction method based on deep learning, the specific steps of the method are as follows:

[0072] S1: The nine-axis gyroscope sensor fixed in the truck is used to collect the truck motion data and correct the truck motion data;

[0073] In this step, the nine-axis gyroscope sensor is fixed to the bottom of the center console of the truck, so that the truck motion data can be collected in real time during the driving process of the truck, and it is far away from the engine and other high-temperature areas, which is also convenient for wiring and installation. The nine-axis gyroscope sensor includes a three-axis accelerometer, a three-axis gyroscope and a three-axis magnetometer. The truck motion data includes acceleration data, angular velocity data and magnetic field data. Since the nine-axis gyroscope sensor cannot ensure that the sensor module remains absolutely parallel to the direction of travel of the truck during welding, and the measured value is easily affected by its own principles and design factors, these inevitable errors will cause the sensor unit to produce zero drift. Therefore, the truck motion data is corrected by a special correction method to improve the accuracy of the measurement data;

[0074] S2: Process the corrected truck motion data according to the Kalman filter algorithm to identify the current posture data;

[0075] S3: Identify the road conditions ahead of the truck during driving, and determine whether there is a stability risk for the cargo based on the road conditions ahead and the current posture data;

[0076] In this step, the road condition ahead includes a curve state and a road surface state, the curve state includes a sharp curve and a gentle curve, and the road surface state includes a pothole road surface and an uphill and downhill road surface. A variety of sensors installed in front of the truck, such as a camera and a laser radar, are used to collect images and radar information of the road ahead in real time. The image collected by the camera is analyzed through an image processing algorithm to identify the curvature of the road ahead. The road surface state is confirmed based on the high-precision distance information provided by the laser radar. Based on the curvature of the road ahead (sharp curvature or gentle curvature) and the road surface state (pothole road surface or uphill and downhill road surface), combined with the current posture data, the stability of the cargo is comprehensively analyzed. By accurately identifying the road condition ahead and the current posture data of the truck, the stability risk of the cargo is timely determined;

[0077] S4: If there is a stability risk with the cargo, it will be displayed through the on-board system and the driving posture of the truck will be corrected;

[0078] The correction of the truck motion data includes acceleration correction, angular velocity correction and magnetic field correction. The acceleration correction corrects the acceleration data by a six-position calibration method, the angular velocity correction corrects the angular velocity data by a zero drift correction, and the magnetic field correction corrects the magnetic field data by an ellipsoid fitting method;

[0079] See also Figure 2 , the flow chart of posture data recognition in the embodiment of the present invention, the specific steps of S2 are as follows:

[0080] S2.1: Convert the acceleration data on the three orthogonal axes of the accelerometer to the navigation coordinate system. By converting the acceleration data to the navigation coordinate system, the error caused by the different installation positions and directions of the sensor is eliminated, and the accuracy of the attitude data is improved. The acceleration attitude value is calculated based on the gravity acceleration data and acceleration data in the navigation coordinate system. The calculation formula of the acceleration attitude value is:

[0081] ,

[0082] in, Indicates the acceleration attitude value, Represents the x-axis projection vector of the gravity acceleration data in the navigation coordinate system. Represents the y-axis projection vector of the gravity acceleration data in the navigation coordinate system. Represents the z-axis projection vector of the gravity acceleration data in the navigation coordinate system. represents matrix multiplication, Represents the transformation matrix between the sensor carrier coordinate system and the navigation coordinate system, Represents the x-axis projection vector of the acceleration data in the sensor carrier coordinate system, Represents the y-axis projection vector of the acceleration data in the sensor carrier coordinate system, Represents the z-axis projection vector of the acceleration data in the sensor carrier coordinate system;

[0083] S2.2: Convert the three-axis output values ​​of the magnetometer in the carrier coordinate system to the navigation coordinate system. Since the x-axis in the navigation coordinate system is aligned with the true north direction, project the magnetic output values ​​of the y-axis and z-axis in the navigation coordinate system to the x-axis to calculate the magnetic attitude value. The calculation formula of the magnetic attitude value is:

[0084]

[0085] in, represents the magnetic attitude value, e represents the exponential function, represents the discretized Laplace-Beltharami operator, Represents the x-axis projection vector of the magnetic field data in the navigation coordinate system, Represents the y-axis projection vector of the magnetic field data in the navigation coordinate system, Represents the z-axis projection vector of the magnetic field data in the navigation coordinate system, Represents the angle between the x-axis and y-axis of the magnetic field data in the navigation coordinate system. Represents the angle between the x-axis and the z-axis of the magnetic field data in the navigation coordinate system;

[0086] S2.3: Convert the three-axis angular velocity data measured by the three-axis gyroscope to the navigation coordinate system according to the direction cosine matrix, calculate the correction value of the angular velocity through the PI controller, and calculate the initial quaternion with a real part of zero according to the correction value ;

[0087] in, represents the initial quaternion, Represents the corrected x-axis projection vector of the angular velocity in the navigation coordinate system, Represents the corrected y-axis projection vector of the angular velocity in the navigation coordinate system, Represents the corrected angular velocity z-axis projection vector in the navigation coordinate system;

[0088] The PI controller includes a proportional part and an integral part. The proportional part calculates the error between the current angular velocity and the target angular velocity through a proportional coefficient. The integral part is used to integrate the error of the proportional part and calculate a correction value according to the integral coefficient. The PI controller has the characteristics of fast response and can correct the angular velocity in real time, thereby ensuring the real-time performance of posture recognition.

[0089] S2.4: The acceleration attitude value, magnetic attitude value and initial quaternion are processed by a Kalman filter to calculate an updated quaternion, and the current attitude data is calculated according to the initial quaternion and the updated quaternion. The calculation formula of the current attitude data is:

[0090] ,

[0091] in, Indicates the current posture data, represents the similarity function, Indicates updating the quaternion. represents the Jacobian matrix, T represents the matrix transpose, wherein the similarity function includes a Gaussian function, a logarithmic function and a square root function;

[0092] The Kalman filter is a recursive algorithm that can estimate the dynamic state of a system in a noisy environment. Its basic idea is to use the mathematical model and observation data of the system to continuously correct the state estimate through two steps of prediction and update. This embodiment establishes an attitude calculation model, designs an extended Kalman filter based on quaternions, establishes a state equation with the initial quaternion as the state quantity, and an observation equation with the acceleration attitude value and the magnetic attitude value as the observation quantity, and adopts a construction method of an adaptive measurement noise covariance matrix to reduce the interference of the carrier linear acceleration and the surrounding local magnetic field, thereby improving the attitude measurement accuracy;

[0093] The curve state obtains the image in front of the truck through the image sensor, extracts the lane line set according to the Hough transform, identifies the lane line of the truck through the parameter voting algorithm, and calculates the curvature of the lane line. The road surface state obtains the three-dimensional point cloud in front of the truck through the laser radar, processes the three-dimensional point cloud data, and identifies the road surface condition in front;

[0094] See also Figure 3 , a flow chart of a method for identifying a curve according to an embodiment of the present invention, wherein the specific steps for identifying the curve state are as follows:

[0095] S3.1: The image sensor installed on the front of the truck collects multi-view images of the road ahead. The multi-view images capture a comprehensive image of the road ahead, providing richer and more detailed information and reducing the information loss and errors caused by a single view.

[0096] S3.2: Grayscale the multi-view images of the road ahead. Grayscale processing reduces the complexity of image data, retains the main road feature information, and provides a basis for subsequent edge detection and image fusion. The grayscale multi-view images are preprocessed according to the adaptive Gaussian filter function. The adaptive Gaussian filter can adaptively adjust the filter parameters according to the local features in the image, effectively smooth the noise in the image, and retain important edge information;

[0097] S3.3: According to the edge detection algorithm, the edge detection algorithm includes a Canny operator, the Canny operator dynamically adjusts the high and low thresholds, selects the optimal threshold according to the local characteristics of the image, enhances the edge connectivity, fills the broken edges using morphological operations, detects the road edge of the pre-processed grayscale multi-view image, obtains the edge image, fuses the multi-view edge images by the maximum fusion method, calculates the fused edge image, and enhances the continuity and integrity of the road edge;

[0098] S3.4: According to the fused edge image, lane lines are detected by cascaded residual regression tree to obtain a lane line set, and the lane line in front of the truck is extracted from the lane line set by a parameter voting algorithm;

[0099] S3.5: The lane line in front of the truck is optimized by the RANSAC algorithm, and the lane line is fitted by the cubic spline interpolation method to obtain the lane line curve. The road curvature is calculated according to the lane line curve. The calculation formula of the road curvature is:

[0100] ,

[0101] in, represents the road curvature, represents the first-order derivative value of the lane curve, Represents the second-order derivative value of the lane curve;

[0102] S3.6: Compare the road curvature with the curve curvature threshold. If it is greater than or equal to the curve curvature threshold, it indicates that the curve ahead of the truck is a sharp curve. If it is less than the curve curvature threshold, it indicates that the curve ahead of the truck is a gentle curve.

[0103] The specific steps of S3.4 are as follows:

[0104] S3.4.1: Construct a cascaded residual regression tree, take the average shape of all fused edge images as the initial shape for regression, input it into the leaf node of the first cascaded residual regression tree, calculate the difference between the true shape and the average shape of each fused edge image, take the variance of the difference of the images in the leaf node, and use the result as the residual of the leaf node;

[0105] S3.4.2: Save the residuals of all leaf nodes, complete the construction of the first tree, update the current shape of each fused edge image to the initial residual shape, input it into the leaf node of the second cascaded residual regression tree, repeatedly calculate the residuals of the leaf nodes, extract the residuals of the leaf nodes of the second cascaded residual regression tree according to the least squares method, obtain the position of the key points of the lane line, mark and crop in the fused edge image, and obtain the lane line set;

[0106] S3.4.3: Define the lane line parameter space, and perform weighted voting on each lane line in the parameter space, so that the lane line closer to the center line of the image (i.e., the direction of vehicle travel) has a higher weight;

[0107] S3.4.4: The voting results and voting weights of each parameter pair are recorded through the accumulator matrix, and a set of suspected lane lines is extracted based on local maximum detection. The set of suspected lane lines is checked for consistency through geometric constraints to extract the lane line in front of the truck. The geometric constraints include: the two lane lines should be parallel, the slope and spacing of the lane lines should comply with the road design specifications, and the length and continuity of the lane lines should meet the road design requirements.

[0108] See also Figure 4 The specific steps of identifying the road surface state are as follows:

[0109] S4.1: Scan the moving direction of the truck through the laser radar sensor to obtain the three-dimensional point cloud data of the front area, and pre-process the three-dimensional point cloud data;

[0110] Usually the road surface can be approximated as a spatial plane, and the projected area on the road surface All laser points within the road surface can be determined as points projected on the road surface, that is, changes within 2 cm of convexity or concavity on the road surface plane will not affect the driving of the truck. Therefore, the road surface plane equation can be fitted through random consistency sampling, and the laser points less than 2 cm away from the road surface plane equation are classified as in-plane points, and the laser points greater than or equal to 2 cm away from the road surface plane equation are classified as out-of-plane points. The degree of dispersion of the out-of-plane points is calculated, and the road surface state is distinguished by clustering the degree of dispersion. Through random consistency sampling, plane fitting, dispersion calculation and cluster analysis, the quality and structure of point cloud data are significantly improved;

[0111] S4.2: Randomly sample the three-dimensional point cloud data through random consistency sampling to improve the overall consistency and accuracy of the point cloud data, obtain the road surface plane equation, mark the laser points that are greater than or equal to 2 cm away from the road surface plane equation as observation point cloud data, calculate the neighborhood points of the observation point cloud data according to the Lagrange multiplier method, make discrete judgments on the observation point cloud data and the neighborhood points, and calculate the discrete degree of the observation point cloud data. The calculation formula for the discrete degree of the observation point cloud data is:

[0112] ,

[0113] in, represents the discrete degree of the observed point cloud data, A represents the covariance matrix of the observed point cloud data, n represents the normal vector of the observed point cloud data, represents the arccosine function, j represents a single neighborhood point of the observed point cloud data, M represents the total number of neighborhood points of the observed point cloud data, Represents the normal vector of the jth neighborhood point of the observed point cloud data, represents the maximum value function, Represents the Euclidean distance between the observed point cloud data and the jth neighboring point of the observed point cloud data;

[0114] S4.3: Perform coplanarity judgment and cluster analysis on the observation point cloud data, merge the observation point cloud data that meet the coplanarity condition and belong to the same category and allocate point set space, distinguish the laser points belonging to different road conditions into different categories, and obtain the laser point set;

[0115] S4.4: Calculate the geometric parameters of the class in the laser point set, the geometric parameters including the y-axis sign of the laser point ,depth and size range , the depth is determined by calculating the absolute distance from each laser point in the class to the plane, selecting the maximum absolute distance as the depth parameter, and the size range is determined by calculating the x-axis distribution range of the laser points in the class;

[0116] S4.5: Output the road surface state according to the geometric parameters. and <0 and , it is judged as a pothole road. Pothole roads usually have a large depth and are concave downwards. and >0 and , it is judged as a raised road surface. and <0 and , it is judged as a downhill road. and >0 and , it is determined to be an uphill road. Through detailed geometric parameter calculation and analysis, different road conditions can be accurately distinguished and determined.

[0117] The specific steps of preprocessing the three-dimensional point cloud data are as follows:

[0118] S4.1.1: Calculate the y-axis coordinates of all 3D point cloud data in the 3D point cloud data set. If the y-axis coordinate of a point is greater than 0, delete the point from the 3D point cloud data set. By deleting all points with y-axis coordinates greater than 0, irrelevant upper point cloud data is removed, and the point cloud data of the ground and objects is retained, thereby improving the validity of the data.

[0119] S4.1.2: When the truck is driving forward, it only needs the road surface information ahead, and the point cloud data of the LiDAR contains 360-degree environmental information. The path interval is set to retain only the 3D point cloud data within the 4m wide strip ahead of the vehicle. This reduces the calculation pressure of the algorithm and can further improve the road surface fitting effect.

[0120] S4.1.3: traverse and query the 3D point cloud data set, delete the 3D point cloud data with the same radar rotation angle and echo distance, and for each point, check whether its rotation angle and echo distance already exist in the hash table. If so, mark it as a duplicate point and delete it; otherwise, record the rotation angle and echo distance of the point to ensure that each point cloud data is unique under the same rotation angle and echo distance, and optimize the structure of the data set;

[0121] S4.1.4: Determine the neighborhood point set of each point through the neighborhood search method, use principal component analysis to determine the normal direction of each point, aggregate the 3D point cloud data in the normal direction, adjust the position and coordinates of the sampling points, and thus denoise the 3D point cloud data. By aggregating the point cloud data in the normal direction, the data surface is smoothed, the noise points are reduced, and the smoothness and accuracy of the data are improved;

[0122] The gravity distribution parameters are obtained by gravity distribution modeling of the truck, including cargo mass, cargo center of gravity and cargo size. The gravity distribution modeling includes cargo placement, center of gravity position measurement and volume division. The cargo placement includes determining the specific placement position and cargo size of the cargo on the truck through precision measuring tools and drawings during cargo loading, and also includes using high-strength restraint belts, partitions, anti-slip mats and other fixing tools to fix the cargo in the truck cargo compartment to ensure that the center of gravity position of the cargo will not change during transportation. The center of gravity position measurement includes using a multi-point weighing system to install high-precision weighing sensors at the four corners of the truck to measure the weight of the cargo at different support points respectively, and calculate the center of gravity position of the cargo based on the weight. , the volume division is used to divide the cargo into a number of three-dimensional small units, determine the gap space according to the three-dimensional small units, and obtain the cargo mass;

[0123] The method for determining whether the cargo has a stability risk includes constructing a risk assessment network, taking the road condition ahead, current posture data, and gravity distribution parameters as input parameters of the risk assessment network, training the input parameters through the risk assessment network, and outputting risk assessment results, thereby achieving an accurate assessment of the stability of the cargo in the truck.

[0124] See also Figure 5 , a structure diagram of a risk assessment network according to an embodiment of the present invention, wherein the risk assessment network comprises an input layer, a hidden layer and an output layer;

[0125] The input layer is used to fuse the front road condition and the current posture data through a normalized embedded Gaussian function to predict the dynamic response of the truck. The input layer also includes an LSTM model, which processes the contextual contact information of the road condition change in the long time series in the fusion feature through the LSTM model, integrates and outputs through the fully connected Dropout, and obtains the dynamic response of the truck;

[0126] The hidden layer is used to extract gravity distribution parameter features, calculate cargo force analysis results, and calculate cargo movement probability according to the dynamic response of the truck and the cargo force analysis results. The hidden layer includes a CNN network, and the CNN network is used to extract gravity distribution parameter features, expand the gravity distribution parameters through a 3×3 convolution kernel, calculate the gravity distribution parameters according to the maximum pooling function, and finally flatten the output of the pooling function into a one-dimensional vector through a flattening function to obtain gravity distribution parameter features. The hidden layer also includes an MLP network, which is used to input the gravity distribution parameter features into the fully connected layer of the MLP network, analyze the cargo force through the neurons of the fully connected layer, and obtain the cargo force conditions under different road conditions. The force conditions include inertial force, centrifugal force and gravity;

[0127] The output layer is used to convert the cargo movement probability into a risk probability value through an activation function, including a tipping risk probability value and a slipping risk probability value;

[0128] The display through the vehicle-mounted system includes risk prompts and warnings. The risk prompts display the stability risk information of the cargo in real time through the vehicle-mounted system, including real-time data of tipping risk and slipping risk. The prompt information includes the risk level and current road conditions. The warning is used to automatically trigger the warning system when a high risk is detected, and prompt the driver to pay attention through sound and vision, and recommend safe driving routes and speeds in combination with the vehicle-mounted navigation system;

[0129] The correction includes speed control, lane adjustment and posture control. The speed control dynamically adjusts the speed of the truck according to the real-time risk analysis results to reduce the risk of overturning and sliding of the cargo, slows down on sharp bends and uphill and downhill sections to reduce centrifugal force and inertia. The lane adjustment is used to suggest the driver to change lanes and avoid uneven sections when a pothole road is identified. The posture control adjusts the height and stiffness of the vehicle suspension in real time through the vehicle's active suspension system to reduce vehicle tilt and yaw. The body stability control system and anti-rollover control system adjust the vehicle's traction and braking torque, control the vehicle's yaw angle, keep the vehicle stable, and perform posture adjustments to maintain vehicle stability.

[0130] Example 2

[0131] See also Figure 6, the present invention provides an embodiment: a vehicle posture correction system based on deep learning, the system comprising a posture data recognition module, a road condition recognition module, a stability assessment module, a risk warning module and a posture correction module;

[0132] The posture data recognition module is configured with a posture recognition strategy, and the posture recognition strategy is used to recognize the current posture of the truck based on the truck motion data;

[0133] The road condition recognition module is configured with a curve state recognition strategy and a road surface state recognition strategy. The curve state recognition strategy is used to calculate the lane line curvature and determine the curve state according to the lane line curve. The road surface state recognition strategy is used to calculate the geometric parameters of the point cloud data and determine the road surface state according to the geometric parameters.

[0134] The stability assessment module is used to construct a risk assessment network, taking the road conditions ahead, current posture data and gravity distribution parameters as input parameters, training the input parameters through the risk assessment network, and outputting risk assessment results;

[0135] The risk prompt module is used to display stability risks through the vehicle-mounted system, including risk prompts and warnings;

[0136] The posture correction module is used to correct the driving posture of the truck, including speed control, lane adjustment and posture control.

[0137] The posture data recognition module comprises:

[0138] Data acquisition unit: uses a nine-axis gyroscope sensor to collect the truck's motion data in real time, including three-axis acceleration data, three-axis angular velocity data, and three-axis magnetic field data;

[0139] Data conversion and correction unit: This unit converts the truck motion data collected in the carrier coordinate system into data in the navigation coordinate system, and calculates the acceleration attitude value, magnetic attitude value and initial quaternion;

[0140] Attitude calculation unit: The Kalman filter is used to calculate and update the quaternion, and the current attitude data is calculated in combination with the initial quaternion.

[0141] The road condition recognition module includes a curve state recognition unit and a road surface condition recognition unit. The curve state recognition strategy includes a curve recognition logic, which is configured in the curve state recognition unit. The curve recognition logic analyzes the image collected by the camera through an image processing algorithm to recognize the curvature of the road ahead. The road surface condition recognition strategy includes a road surface recognition logic, which is configured in the road surface condition recognition unit. The road surface recognition logic confirms the road surface condition according to the high-precision distance information provided by the laser radar.

[0142] The curve state recognition unit acquires the image in front of the truck through an image sensor, extracts a lane line set using Hough transform, recognizes the lane line through a parameter voting algorithm, and finally calculates the curvature of the lane line to determine the state of the curve ahead;

[0143] The road condition recognition unit uses a laser radar to obtain the three-dimensional point cloud data in front, performs random consistency sampling, plane fitting, discrete degree calculation and cluster analysis, and calculates geometric parameters to identify the road condition.

[0144] The stability assessment module comprises:

[0145] Gravity distribution parameter modeling unit: by modeling the gravity distribution of the truck, the cargo mass, center of gravity position and size parameters are obtained;

[0146] Risk assessment network construction unit: constructs a risk assessment network consisting of an input layer, a hidden layer, and an output layer, which is used to receive the road conditions ahead, current posture data, and gravity distribution parameters as input, and outputs the stability risk assessment results of the cargo through training.

[0147] The risk warning module includes:

[0148] Display unit: when the output result of the stability assessment module indicates that there is a stability risk, the corresponding prompt information, including the risk level and warning signal, is displayed on the interface of the vehicle system;

[0149] Audio warning unit: issues a warning signal via the audio system to alert the driver to possible risk situations.

[0150] The posture correction module comprises:

[0151] Speed ​​control unit: adjusts vehicle speed according to stability risk assessment results, slowing down or increasing speed to reduce rollover risk;

[0152] Lane adjustment unit: When driving on a curve, the lane position is adjusted according to the identified curve status to ensure safe passage through the curve;

[0153] Posture control unit: controls the vehicle's posture by adjusting the suspension system, braking system, etc. to ensure cargo stability and driving safety.

[0154] Although the embodiments of the present invention have been shown and described above, it is to be understood that the above embodiments are exemplary and are not to be construed as limitations of the present invention. A person skilled in the art may change, modify, replace and vary the above embodiments within the scope of the present invention.

Claims

1. A vehicle posture correction method based on deep learning, characterized in that: The method comprises: The nine-axis gyroscope sensor is used to collect the truck's motion data and identify the current posture data; Identify the road conditions ahead of the truck while it is driving, and determine whether there is a stability risk for the cargo based on the road conditions ahead and current posture data combined with gravity distribution parameters; If there is a stability risk with the cargo, it will be displayed through the vehicle system and the driving posture of the truck will be corrected; The nine-axis gyroscope sensor includes a three-axis accelerometer, a three-axis gyroscope and a three-axis magnetometer. The truck motion data includes acceleration data, angular velocity data and magnetic field data. The identification of current posture data includes motion data correction and posture data identification. The specific steps of posture data identification are: Convert the truck motion data from the carrier coordinate system to the navigation coordinate system and calculate the acceleration attitude value , magnetic attitude value and the initial quaternion ; The acceleration attitude value, magnetic attitude value and initial quaternion are processed by a Kalman filter to calculate an updated quaternion, and the current attitude data is calculated according to the initial quaternion and the updated quaternion. The calculation formula of the current attitude data is: ， in, Indicates the current posture data, represents the similarity function, Indicates updating the quaternion. represents the Jacobian matrix, T represents the matrix transpose; The identification of the road condition ahead of the truck during driving includes curve state identification and road surface state identification. The curve state identification obtains the image ahead of the truck through an image sensor, extracts a lane line set according to an edge detection algorithm and a cascaded residual regression tree, identifies the lane line of the truck through a parameter voting algorithm, and calculates the curvature of the lane line. The road surface state identification obtains a three-dimensional point cloud ahead of the truck through a laser radar, performs random consistency sampling, plane fitting, discrete degree calculation and cluster analysis on the three-dimensional point cloud data, and calculates the geometric parameters of the point cloud data; The gravity distribution parameters are obtained by performing gravity distribution modeling on the truck, including cargo mass, cargo center of gravity and cargo size. The method for judging whether the cargo has stability risk includes constructing a risk assessment network, taking the road condition ahead, current posture data and gravity distribution parameters as input parameters of the risk assessment network, training the input parameters through the risk assessment network, and outputting a risk assessment result; The risk assessment network includes an input layer, a hidden layer and an output layer; The input layer is used to fuse the front road condition and the current posture data through a normalized embedded Gaussian function to predict the dynamic response of the truck; The hidden layer is used to extract the characteristics of gravity distribution parameters, calculate the cargo force analysis results, and calculate the cargo movement probability based on the dynamic response of the truck and the cargo force analysis results; The output layer is used to convert the cargo movement probability into a risk probability value through an activation function.

2. The vehicle posture correction method based on deep learning according to claim 1, characterized in that: The specific steps of the curve state recognition are as follows: The image sensor installed on the front of the truck collects multi-view images of the road ahead; The lane lines are detected by edge detection algorithm and cascade residual regression tree to obtain a lane line set, and the lane line in front of the truck is extracted from the lane line set by parameter voting algorithm; The lane line in front of the truck is optimized by the RANSAC algorithm, and the lane line is fitted by the cubic spline interpolation method to obtain the lane line curve. The road curvature is calculated according to the lane line curve. The calculation formula of the road curvature is: ， in, represents the road curvature, represents the first-order derivative value of the lane curve, Represents the second-order derivative value of the lane curve; The road curvature is compared with the curve curvature threshold. If it is greater than or equal to the curve curvature threshold, it means that the curve ahead of the truck is a sharp curve. If it is less than the curve curvature threshold, it means that the curve ahead of the truck is a gentle curve.

3. The vehicle posture correction method based on deep learning according to claim 1 is characterized in that: The specific steps of road surface state recognition are as follows: The three-dimensional point cloud data is randomly sampled by random consistency sampling to obtain the road surface plane equation, and the laser points that are greater than or equal to 2 cm away from the road surface plane equation are marked as observed point cloud data; The neighborhood points of the observed point cloud data are calculated according to the Lagrange multiplier method, the observed point cloud data and the neighborhood points are discretely determined, and the discrete degree of the observed point cloud data is calculated. The calculation formula for the discrete degree of the observed point cloud data is: ， in, represents the discrete degree of the observed point cloud data, A represents the covariance matrix of the observed point cloud data, n represents the normal vector of the observed point cloud data, represents the arccosine function, j represents a single neighborhood point of the observed point cloud data, M represents the total number of neighborhood points of the observed point cloud data, Represents the normal vector of the jth neighborhood point of the observed point cloud data, represents the maximum value function, Represents the Euclidean distance between the observed point cloud data and the jth neighboring point of the observed point cloud data; Perform coplanarity judgment and cluster analysis on the observation point cloud data, merge the observation point cloud data that meet the coplanarity conditions and belong to the same category and allocate point set space, distinguish the laser points belonging to different road conditions into different categories, and obtain the laser point set; The geometric parameters of the classes in the laser point set are calculated, and the road surface state is output according to the geometric parameters.

4. A vehicle posture correction system based on deep learning, implemented based on the vehicle posture correction method based on deep learning as claimed in any one of claims 1 to 3, characterized in that: The system includes a posture data recognition module, a road condition recognition module, a stability assessment module, a risk warning module and a posture correction module; The posture data recognition module is configured with a posture recognition strategy, and the posture recognition strategy is used to recognize the current posture of the truck based on the truck motion data; The road condition recognition module is configured with a curve state recognition strategy and a road surface state recognition strategy. The curve state recognition strategy is used to calculate the lane line curvature and determine the curve state according to the lane line curve. The road surface state recognition strategy is used to calculate the geometric parameters of the point cloud data and determine the road surface state according to the geometric parameters. The stability assessment module is used to construct a risk assessment network, taking the road conditions ahead, current posture data and gravity distribution parameters as input parameters, training the input parameters through the risk assessment network, and outputting risk assessment results.

5. The vehicle posture correction system based on deep learning according to claim 4 is characterized in that: The system also includes a risk warning module and a posture correction module; The risk prompt module is used to display stability risks through the vehicle system, including risk prompts and warnings; The posture correction module is used to correct the driving posture of the truck, including speed control, lane adjustment and posture control.

6. The vehicle posture correction system based on deep learning according to claim 4, characterized in that: The road condition recognition module includes a curve state recognition unit and a road surface condition recognition unit, the curve state recognition strategy includes a curve recognition logic, the curve recognition logic is configured in the curve state recognition unit, and the road surface condition recognition strategy includes a road surface recognition logic, the road surface recognition logic is configured in the road surface condition recognition unit; The curve state recognition unit acquires the image in front of the truck through an image sensor, extracts a lane line set through an edge detection algorithm and a cascaded residual regression tree, recognizes the lane line through a parameter voting algorithm, and finally calculates the curvature of the lane line to determine the state of the curve ahead; The road condition recognition unit uses a laser radar to obtain the three-dimensional point cloud data in front, performs random consistency sampling, plane fitting, discrete degree calculation and cluster analysis, and calculates geometric parameters to identify the road condition.

Images