Image measurement vehicle roll angle method suitable for structured pavement condition

Abstract

The invention provides a method for measuring a roll angle of a vehicle through an image suitable for a structured pavement condition, and aims to compensate for accumulative errors and vibration influences of low-cost IMU measurement. According to the method, firstly, the geometric distance between laser beams is measured through an improved image processing method, and then high-precision roll angle measurement is achieved based on a regression function. In order to further improve the measurement accuracy, the collected laser image is input into the improved MobileNetV4 model, and the feature extraction and dynamic prediction capabilities are optimized through the technologies of multi-scale convolution, adaptive convolution kernel, dynamic feature selection and weighting and the like, so that the prediction of the roll angle of the vehicle is realized. The two methods are combined, a weighted average fusion mechanism is used, the weight is dynamically adjusted, and the precision and stability of inclination angle measurement are further improved. According to the method, high precision can be kept under the structured pavement condition, and compared with a low-cost IMU sensor measurement method, the method has good stability and real-time performance. The research provides a new means for the monitoring of the dynamic behaviors of the vehicle, and is of great significance to the development of vehicle safety and driving assistance systems.

Description

Technical Field

[0001] The invention relates to the technical field of vehicle posture monitoring, and in particular to a method for measuring vehicle roll angle based on an image of laser beam geometric distance under structured road conditions. Background Art

[0002] Vehicle rollover is an extremely dangerous traffic accident, especially for heavy vehicles such as trucks, which are prone to rollover when turning because of their high center of gravity and heavy load. Rollover accidents not only cause serious property damage, but may also endanger the lives of drivers and passengers. Studies have shown that the occurrence of vehicle rollover accidents is closely related to multiple factors, including the vehicle's driving state, road conditions, vehicle posture, etc. In anti-rollover technology, vehicle stability and posture control are key, and rollover prediction and warning have become the core of effective prevention and control. Modern active safety control systems such as active suspension and differential braking avoid rollover accidents by adjusting the vehicle state in real time.

[0003] In these control systems, real-time monitoring of the vehicle's roll angle plays a vital role. As a key parameter for evaluating the vehicle's dynamic behavior and stability, the roll angle can directly reflect whether the vehicle is close to the critical point of rollover, helping the system to adjust the vehicle's driving state in time to prevent rollover. Especially under extreme conditions such as high-speed turns and sharp obstacle avoidance, accurate measurement of the roll angle is of great significance for improving vehicle stability and ensuring driving safety.

[0004] At present, the measurement of vehicle roll angle mainly relies on various sensors, including accelerometers, gyroscopes, etc. Although these sensors can provide measurement results, they are often limited by cost, installation complexity and environmental adaptability. First of all, accelerometers and gyroscopes are relatively complex in the integration and calibration process, which not only increases the installation and maintenance costs, but also has high requirements for the installation location and vehicle structure. In addition, during long-term operation, these sensors are easily affected by environmental factors such as temperature and humidity, and drift occurs, resulting in reduced measurement accuracy. Especially when the vehicle is running for a long time, frequently accelerating and decelerating, or encountering complex road conditions, the error accumulation of the sensor will be further aggravated, making it difficult to ensure the stability and accuracy of the tilt angle measurement. These problems make the application of traditional sensors in roll angle measurement have certain limitations, and it is difficult to maintain accurate and reliable measurement results in various complex driving environments.

[0005] Therefore, how to overcome the limitations of existing technologies and achieve accurate, stable and real-time roll angle measurement has become a key challenge to improve the performance of anti-rollover control systems. Summary of the invention

[0006] In view of the limitations of the existing technology, the present invention proposes a method for measuring the vehicle roll angle based on the image of the laser beam geometric distance under structured road conditions. Compared with the existing technology, this method shows significant advantages in long-term measurement accuracy and environmental adaptability, and provides more stable and accurate measurement results in applications that require high-precision real-time roll angle information, and has broad application prospects.

[0007] The present invention adopts the following technical solutions:

[0008] A method for measuring vehicle roll angle based on an image of laser beam geometric distance under structured road conditions, comprising the following contents:

[0009] Step S1, install two laser emitters along the transverse axis on the outside of the bottom of the vehicle suspension so that they form parallel linear light spots on the ground. Install an imaging device (such as a camera) near one of the laser emitters. Adjust the position of the laser emitter so that the light beam is within the field of view of the imaging device and parallel to the longitudinal axis of the vehicle. The imaging device continuously captures the image of the laser beam on the ground to detect the roll angle of the vehicle in real time.

[0010] Step S2: Install the laser transmitter and imaging device on the platform to simulate the installation configuration of the vehicle suspension. During the experiment, the vehicle roll is simulated by tilting the platform, and the tilt angle is recorded by the tilt sensor, and the collected laser images are calibrated. Repeat the experiment under common structured road conditions to establish a more comprehensive data set to facilitate the completion of tilt angle measurement adapted to structured road conditions.

[0011] Step S3: Based on the data set established in step S2, the collected images are processed based on Canny edge detection and Hough transform algorithms to calculate the geometric distance between the two laser beams, and then a regression model of the inclination angle-laser light distance is constructed, and the least squares method is used for fitting to accurately predict the vehicle roll angle under different road conditions.

[0012] Step S4: In order to improve the accuracy and environmental adaptability of tilt angle detection, the present invention predicts the roll angle of the vehicle through an improved MobileNetV4 model. The model adopts an adaptive convolution kernel and a dynamic feature selection mechanism to enhance the processing capability of complex ground reflection information, and supports quantization and pruning technology to make it lightweight and convenient for real-time deployment on the vehicle platform.

[0013] Step S5. Finally, the present invention utilizes a fusion mechanism based on weighted average to combine the S3 image processing with the prediction results of the S4 deep learning model, and dynamically adjusts the weights through adaptive correction to improve the accuracy and stability of vehicle roll angle measurement.

[0014] Further, the specific steps of step S1 are as follows:

[0015] Step S11, laser light source arrangement: two laser emitters are installed horizontally side by side along the transverse axis of the vehicle on the outside of the bottom of the vehicle body, ensuring that the laser beams they emit can form clear, parallel linear light spots on the ground.

[0016] Step S12, imaging unit setting: select an imaging device (such as a camera) and install it close to one of the laser emitters, ensuring that the vertical height of the device from the ground is between 30 and 50 cm. The imaging device should support the capture of RGB 24-bit color images and have a resolution of at least 480×320 to ensure image quality.

[0017] Step S13, light spot positioning: adjust the positions of the two laser emitters so that the linear light spots formed by them on the ground are within the field of view of the imaging device, and ensure that each light ray is parallel to the longitudinal axis of the vehicle.

[0018] Step S14, image capture: In order to detect the roll angle of the vehicle in real time, the imaging device continuously captures the image of the laser spot on the ground at a speed of about 30 frames per second.

[0019] Further, the specific steps of step S2 are as follows:

[0020] Step S21: After the laser transmitter and imaging device are installed, the laser road image data under common structured road conditions are collected by constructing an experimental device. In order to simulate the roll of the vehicle while ensuring safety, the present invention uses a platform instead of the vehicle's suspension, and the laser transmitter and imaging device are installed on the platform to ensure that the installation position is consistent with the installation on the actual vehicle. In addition, a high-precision tilt sensor is installed under the platform for calibration of the data set.

[0021] Step S22: During the experiment, the platform is tilted to simulate the vehicle roll, and the roll angle is controlled between -15 degrees and 15 degrees. By adjusting the tilt angle of the platform, nearly 80 laser road surface images are taken under the same road surface conditions, and the angles corresponding to these images are evenly distributed between -15 degrees and 15 degrees. The roll angle recorded in real time by the tilt sensor is used to calibrate the collected image data to ensure the accuracy of the data set.

[0022] Step S23: After the establishment of the data set for a single road type, the experimental scope is further expanded to cover common structured road conditions: asphalt, mud, cement and gravel roads. For each road type, the corresponding laser road images are systematically collected according to the established method. In this way, a comprehensive data set can be constructed, which can represent various road conditions that a vehicle may encounter during actual driving, thereby providing a solid data foundation for the accurate measurement of the vehicle roll angle.

[0023] Further, the specific steps of step S3 are as follows:

[0024] First, the image collected by the imaging device is converted into a grayscale image to reduce the data dimension and improve the processing efficiency. Then, in order to remove the noise and detail interference in the image, the present invention adopts the adaptive Gaussian blur technology and dynamically adjusts the smoothing parameters to further improve the image quality and provide high-quality input data for subsequent feature extraction.

[0025] Next, the present invention uses the Canny edge detection algorithm to extract edge information in the image. The specific method is as follows: first, the pre-processed image is subjected to gradient calculation, and the area with the most obvious brightness change in the image is identified by calculating the brightness change in the horizontal and vertical directions, that is, the edge area most likely to be the laser beam, and the non-maximum suppression strategy is adopted to screen out the strongest local edges to avoid repeated identification and blurred edge responses. The edges are classified by a dual threshold strategy: that is, high-intensity edges are immediately identified, and low-intensity edges are determined to be effective by their connection relationship with high-intensity edges. This dual threshold mechanism effectively suppresses the noise contours in the background, ensuring that only clear edges related to the target features are retained. Through the edge detection algorithm, the output edge image only contains edges with structural significance, laying the foundation for the subsequent laser beam feature extraction.

[0026] Most of the noise can be removed by edge detection, but the outline of some ground background is still included. In order to accurately extract the two laser beams, the present invention first analyzes the edge detection results and further eliminates isolated points or small edge areas. This step calculates the area or perimeter of each edge area and excludes those areas that are smaller than expected or do not meet the characteristics of the laser beam based on the set threshold, thereby reducing background interference and highlighting the laser beam characteristics. The Hough transform is then used to extract the laser beam features in the image. The Hough transform maps each edge point in the image to the parameter space and identifies the point set that conforms to the straight line equation. The specific steps are as follows:

[0027] Polar coordinate space mapping: For each edge point, according to its image coordinates (x, y), it is converted into line parameters (ρ, θ) in polar coordinates, where

[0028] ρ=x·cos(θ)+y·sin(θ)

[0029] In this process, each edge point will form a curve in the ρ-θ parameter space.

[0030] When multiple edge points are collinear, the curves they form in the ρ-θ space will intersect at the same point. This intersection represents a straight line in the image. By accumulating the number of occurrences of these intersections, we can find the points that appear the most in the parameter space, which correspond to the significant straight lines in the image.

[0031] In order to ensure the accuracy of detection, the present invention introduces a dynamic threshold adjustment mechanism in the Hough transform, which only retains the points whose cumulative times in the parameter space exceed a certain threshold. This threshold is dynamically adjusted according to the specific application environment and the intensity of the laser beam, ensuring that only significant straight lines are detected while ignoring irrelevant background contours.

[0032] After the straight line detection is completed, multiple straight lines may be detected in the image. In order to accurately extract the two laser beams, the present invention uses the following strategies for screening and optimization:

[0033] (1) Redundant line detection: First, the distance between each pair of detected lines is calculated, usually based on their ρ parameter (i.e., the distance from the line to the image origin). For lines with a distance that is too small, they are judged to be redundant detection results of the same line; then, the θ angle difference between each two lines is calculated. If the angle difference between the two lines is less than a certain set threshold and their distance difference is also within a certain range, they are considered to be the same line and merged. This step generates a more stable line parameter by calculating the mean or median.

[0034] (2) Laser beam screening: To ensure that only two laser beams are extracted, the present invention further screens the lines based on the geometric position, the length of the line, and the relationship with the emission direction of the laser emitter. According to the preset geometric model, two lines that meet the expected position and direction of the laser beam can be quickly identified, and other background lines that do not meet the conditions are excluded.

[0035] After completing the image processing step, the centerline equations of the two laser beams can be obtained. Then, the horizontal axis is set to the actual roll angle of the vehicle, which is recorded by the tilt sensor during image acquisition. The vertical axis is set to the pixel position of the centerline of the two laser beams, which is the horizontal position of the centerline in the image calculated after the image processing step. Then, the least squares method is used to fit the curve to these data points. The least squares method determines the best fit curve by minimizing the sum of squares of the errors, thereby establishing a regression model that can accurately represent the relationship between the vehicle roll angle and the centerline pixel position.

[0036] Further, the specific steps of step S4 are as follows:

[0037] After completing the construction of the basic regression model, in order to further improve the accuracy and environmental adaptability of the vehicle roll angle measurement system, the present invention predicts the roll angle of the vehicle through an improved MobileNetV4 model. MobileNetV4 adopts an inverted bottleneck block (IRB) structure, which effectively reduces the number of model parameters and calculations through depthwise separable convolution while maintaining good performance. In addition, MobileNetV4 further enhances the model's ability to capture multi-scale features by adopting a hierarchical interconnection approach and a feature fusion strategy.

[0038] In order to better meet the specific needs of vehicle roll angle measurement, especially to provide stable and reliable measurement results under structured road conditions, this paper improves on the basis of MobileNetV4. First, an adaptive convolution kernel mechanism is introduced, which enables the model to dynamically adjust the parameters of the convolution kernel according to the shape and intensity of the laser beam in the input image, so as to more accurately extract features related to the roll angle. This optimization significantly improves the model's ability to process complex ground reflection information, enhances the accuracy of prediction and adaptability under different road conditions.

[0039] Secondly, considering the resource constraints that the model may face in actual deployment, the present invention fully considers the compatibility of quantization and pruning technology when designing the network structure. By selecting activation functions and data representation methods suitable for quantization, and constructing a network structure that is easy to prune, the optimized MobileNetV4 model can reduce the size of the model and computing resource consumption while maintaining high-precision prediction, and realize real-time processing on the on-board computing platform. This feature enables the present invention to perform real-time data fusion with the vehicle's existing IMU system to provide more accurate and stable roll angle measurement results.

[0040] In addition, the present invention also introduces a dynamic feature selection and attention mechanism. In the feature fusion stage, the model can learn the importance of different features to the roll angle prediction, and select and weight features based on these importance. This mechanism enables the model to pay more attention to features that have a significant impact on the prediction results, while ignoring those irrelevant or noisy features, thereby further improving the accuracy of the prediction. This dynamic feature selection capability enables the present invention to perform well in long-term measurements and effectively compensates for the cumulative errors that may occur in the long-term use of the IMU.

[0041] Through the improved MobileNetV4 network, a high-precision and environmentally adaptable vehicle roll angle measurement model can be obtained. It can accurately analyze various structured road scenes, improve prediction accuracy through adaptive convolution kernels and dynamic feature selection, and based on lightweight model design, it can facilitate the deployment of the model on the vehicle computing platform. This deep learning method complements traditional IMU measurements, especially in the case of frequent acceleration and deceleration or long-term driving of the vehicle, and can provide more stable and accurate roll angle information, providing reliable data support for vehicle stability control and safety systems.

[0042] Further, the specific steps of step S5 are as follows:

[0043] After completing the design of the MobileNetV4 model, the present invention integrates the model with the image processing method to further improve the accuracy and environmental adaptability of vehicle roll angle measurement. The specific integration steps are as follows:

[0044] First, the present invention introduces a fusion mechanism based on weighted average to combine the advantages of image processing methods and deep learning models. In this mechanism, the weights of image processing results and deep learning prediction results are first adjusted dynamically according to the current road conditions, lighting conditions and other external environmental factors. For scenes with good lighting conditions and smooth roads, the image processing method has a higher weight, while in scenes with complex lighting and rugged roads, the weight of the deep learning model is increased accordingly to cope with complex background and noise interference.

[0045] Subsequently, at each time step, the weighted average method is used to fuse the image processing results and the deep learning prediction results to generate a final roll angle prediction value.

[0046] Assume that the prediction result of the image processing method is θ img , the prediction result of the deep learning model is θ dl , then the final roll angle prediction value can be calculated by the following formula:

[0047] θ final =w img (t)·θ img +w dl (t)·θ dl

[0048] In order to achieve adaptive dynamic adjustment, the present invention also introduces an adaptive correction mechanism to further optimize the accuracy of the fusion results. The mechanism analyzes the fusion results of multiple consecutive time steps and dynamically adjusts the weight value. In this strategy, the system will adjust the weight of the prediction results of the image processing method and the prediction results of the deep learning model in real time according to the changes in the external environment, such as light intensity, road surface flatness, and vehicle speed.

[0049] According to the roll angle prediction formula above, its weight value is updated by the following formula:

[0050]

[0051]

[0052] Among them, α is the adjustment coefficient, which controls the sensitivity of the system to errors. A larger α value will make the system more sensitive to errors, resulting in more drastic changes in weights; while a smaller α value will make the weight adjustment smoother.

[0053] The normalization factor Z(t) is used to ensure that the sum of the two weights is 1, and the calculation formula is as follows:

[0054] Z(t)=exp(-α·|θ img (t)-θ final (t)|)+exp(-α·|θ dl (t)-θ final (t)|)

[0055] In this way, the system is able to automatically adjust the weights at each time step based on the prediction accuracy of different methods. For example, when the prediction error of the image processing method is small, w img (t+1) will increase, which means that the system will rely more on the results of image processing. On the contrary, if the prediction error of the deep learning model is small, w dl (t+1) will increase, making the system more dependent on the predictions of the deep learning model.

[0056] The beneficial effect of the present invention is that this comprehensive method not only achieves high-precision geometric analysis, but also improves adaptability to common structured road conditions. The technology has the ability to predict the vehicle roll angle in real time and can effectively compensate for the accumulated error of IMU in long-term measurement. Compared with the prior art, the present invention shows significant advantages in long-term measurement accuracy and environmental adaptability, and provides an efficient and reliable complementary solution for vehicle stability control and road safety. BRIEF DESCRIPTION OF THE DRAWINGS

[0057] Figure 1 The figure is a flow chart of the vehicle roll angle measurement system of the present invention.

[0058] Figure 2 It is a schematic diagram of the installation of the inclination angle measurement of the present invention.

[0059] Figure 3 It is a process schematic diagram of the image processing technology of the present invention. DETAILED DESCRIPTION

[0060] The present invention will be further explained below in conjunction with the accompanying drawings and specific implementations. It should be understood that the following specific implementations are only used to illustrate the present invention and are not used to limit the scope of the present invention.

[0061] Figure 1 This is a flow chart of the vehicle roll angle measurement system of the present invention. The present invention will be further described below in conjunction with the accompanying drawings.

[0062] Step 1: Install two laser emitters on the outside of the bottom of the vehicle suspension along the transverse axis so that they form parallel linear spots on the ground. Install an imaging device (such as a camera) near one of the laser emitters. Adjust the position of the laser emitter so that the beam is within the field of view of the imaging device and parallel to the longitudinal axis of the vehicle. The imaging device continuously captures the image of the laser beam on the ground to detect the roll angle of the vehicle in real time.

[0063] Step 2: Install the laser transmitter and imaging equipment on the platform to simulate the installation configuration of the vehicle suspension. During the experiment, the vehicle roll is simulated by tilting the platform, and the tilt angle is recorded using the inclination sensor to calibrate the collected laser images. Repeat the experiment under common structured road conditions to establish a more comprehensive data set to facilitate the completion of inclination measurements adapted to structured road conditions.

[0064] Step 3. Based on the data set established in step 2, the collected images are processed based on Canny edge detection and Hough transform algorithms to calculate the geometric distance between the two laser beams. Then, a regression model of the inclination angle-laser light distance is constructed and fitted using the least squares method to accurately predict the vehicle roll angle under different road conditions.

[0065] Step 4: In order to improve the accuracy and environmental adaptability of tilt angle detection, the present invention predicts the roll angle of the vehicle through an improved MobileNetV4 deep learning model. The model adopts adaptive convolution kernel and dynamic feature selection mechanism to enhance the processing ability of complex ground reflection information, and supports quantization and pruning technology to make it lightweight and convenient for real-time deployment on the vehicle platform.

[0066] Step 5. Finally, the present invention utilizes a fusion mechanism based on weighted average to combine the image processing in step 3 with the prediction results of the deep learning model in step 4, and dynamically adjusts the weights through adaptive correction to improve the accuracy and stability of vehicle roll angle measurement.

[0067] The schematic diagram of the inclination measurement installation in step 1 is as follows Figure 2 As shown, the specific implementation is:

[0068] First, the laser light source is arranged: that is, two laser emitters are installed horizontally side by side along the transverse axis of the vehicle on the outside of the bottom of the vehicle suspension to ensure that the laser beams they emit can form clear, parallel linear spots on the ground. Then the imaging unit is set up: that is, an imaging device (such as a camera) is selected and installed close to one of the laser emitters, ensuring that the vertical height of the device from the ground is between 30 and 50 cm. The imaging device should support the capture of RGB24-bit color images and have a resolution of at least 480×320 to ensure image quality. Then the beam positioning is completed: that is, the positions of the two laser emitters are adjusted so that the linear spots they form on the ground are within the field of view of the imaging device, and each beam is parallel to the longitudinal axis of the vehicle. Finally, image capture is achieved: in order to detect the roll angle of the vehicle in real time, the imaging device continuously captures the image of the laser beam on the ground at a speed of about 30 frames per second.

[0069] The specific implementation of step 2 is:

[0070] After the laser transmitter and imaging device are installed, the laser road image data under common structured road conditions are collected by constructing an experimental device. In order to simulate the roll of the vehicle while ensuring safety, the present invention uses a platform instead of the vehicle's suspension, and the laser transmitter and imaging device are installed on the platform to ensure that the installation position is consistent with the installation on the actual vehicle. In addition, a high-precision tilt sensor is installed under the platform for calibration of the data set.

[0071] During the experiment, the platform was tilted to simulate the vehicle roll, and the roll angle was controlled between -15 degrees and 15 degrees. By adjusting the tilt angle of the platform, nearly 80 laser road surface images were taken under the same road conditions, and the corresponding angles of these images were evenly distributed between -15 degrees and 15 degrees. The roll angle recorded in real time by the tilt sensor is used to calibrate the collected image data to ensure the accuracy of the data set.

[0072] After completing the establishment of the data set for a single road type, the experiment was further expanded to cover common structured road conditions: asphalt, mud, cement, and gravel. For each road type, the corresponding laser road images were systematically collected according to the established method. In this way, a comprehensive data set can be constructed that can represent various road conditions that vehicles may encounter during actual driving, thus providing a solid data foundation for the accurate measurement of vehicle roll angle.

[0073] The process diagram of the image processing technology in step 3 is as follows Figure 3 As shown, the specific implementation is:

[0074] First, the image collected by the imaging device is converted into a grayscale image to reduce the data dimension and improve the processing efficiency. In order to remove noise and detail interference in the image, the present invention adopts adaptive Gaussian blur technology and dynamically adjusts the smoothing parameters to further improve the image quality and provide high-quality input data for subsequent feature extraction.

[0075] Next, the present invention uses the Canny edge detection algorithm to extract edge information in the image. The specific method is as follows: first, the pre-processed image is subjected to gradient calculation, and the area with the most obvious brightness change in the image is identified by calculating the brightness change in the horizontal and vertical directions, that is, the edge area most likely to be the laser beam, and the non-maximum suppression strategy is adopted to screen out the strongest local edges to avoid repeated identification and blurred edge responses. The edges are classified by a dual threshold strategy: that is, high-intensity edges are immediately identified, and low-intensity edges are determined to be effective by their connection relationship with high-intensity edges. This dual threshold mechanism effectively suppresses the noise contours in the background, ensuring that only clear edges related to the target features are retained. Through the edge detection algorithm, the output edge image only contains edges with structural significance, laying the foundation for the subsequent laser beam feature extraction.

[0076] Most of the noise can be removed by edge detection, but the outline of some ground background is still included. In order to accurately extract the two laser beams, the present invention first analyzes the edge detection results and further eliminates isolated points or small edge areas. This step calculates the area or perimeter of each edge area and excludes those areas that are smaller than expected or do not meet the characteristics of the laser beam based on the set threshold, thereby reducing background interference and highlighting the laser beam characteristics. The Hough transform is then used to extract the laser beam features in the image. The Hough transform maps each edge point in the image to the parameter space and identifies the point set that conforms to the straight line equation. The specific steps are as follows:

[0077] Polar coordinate space mapping: For each edge point, according to its image coordinates (x, y), it is converted into line parameters (ρ, θ) in polar coordinates, where

[0078] ρ=x·cos(θ)+y·sin(θ)

[0079] In this process, each edge point will form a curve in the ρ-θ parameter space.

[0080] When multiple edge points are collinear, the curves they form in the ρ-θ space will intersect at the same point. This intersection represents a straight line in the image. By accumulating the number of occurrences of these intersections, we can find the points that appear the most in the parameter space, which correspond to the significant straight lines in the image.

[0081] In order to ensure the accuracy of detection, the present invention introduces a dynamic threshold adjustment mechanism in the Hough transform, which only retains the points whose cumulative times in the parameter space exceed a certain threshold. This threshold is dynamically adjusted according to the specific application environment and the intensity of the laser beam, ensuring that only significant straight lines are detected while ignoring irrelevant background contours.

[0082] After the straight line detection is completed, multiple straight lines may be detected in the image. In order to accurately extract the two laser beams, the present invention uses the following strategies for screening and optimization:

[0083] (1) Redundant line detection: First, the distance between each pair of detected lines is calculated, and the difference is calculated based on their ρ parameter (i.e., the distance from the line to the image origin). For lines with a distance that is too small, they are judged to be redundant detection results of the same line; then, the θ angle difference between each two lines is calculated. If the angle difference between the two lines is less than a certain set threshold and their distance difference is also within a certain range, they are considered to be the same line and merged. This step generates a more stable line parameter by calculating the mean or median.

[0084] (2) Line feature screening: In order to ensure that only two laser beams are extracted, the present invention further screens the lines based on the geometric position, the length of the line, and the relationship with the emission direction of the laser emitter. According to the preset geometric model, two lines that meet the expected position and direction of the laser beam can be quickly identified, and other background lines that do not meet the conditions are excluded.

[0085] After completing the image processing step, the centerline equations of the two laser beams can be obtained. Then, the horizontal axis is set to the actual roll angle of the vehicle, which is recorded by the tilt sensor during image acquisition. The vertical axis is set to the pixel position of the centerline of the two laser beams, which is the horizontal position of the centerline in the image calculated after the image processing step. Then, the least squares method is used to fit the curve to these data points. The least squares method determines the best fit curve by minimizing the sum of squares of the errors, thereby establishing a regression model that can accurately represent the relationship between the vehicle roll angle and the centerline pixel position.

[0086] The specific implementation of step 4 is as follows:

[0087] After completing the construction of the basic regression model, in order to further improve the accuracy and environmental adaptability of the vehicle roll angle measurement system, the present invention predicts the roll angle of the vehicle through an improved MobileNetV4 model. MobileNetV4 adopts an inverted bottleneck block (IRB) structure, which effectively reduces the number of model parameters and calculations through depthwise separable convolution while maintaining good performance. In addition, MobileNetV4 further enhances the model's ability to capture multi-scale features by adopting a hierarchical interconnection approach and a feature fusion strategy.

[0088] In order to better meet the specific needs of vehicle roll angle measurement, especially to provide stable and reliable measurement results under different road conditions, this paper improves on the basis of MobileNetV4. First, an adaptive convolution kernel mechanism is introduced, which enables the model to dynamically adjust the parameters of the convolution kernel according to the shape and intensity of the laser beam in the input image, so as to more accurately extract features related to the roll angle. This optimization significantly improves the model's ability to process complex ground reflection information, enhances the accuracy of prediction and adaptability under different road conditions.

[0089] Secondly, considering the resource constraints that the model may face in actual deployment, the present invention fully considers the compatibility of quantization and pruning technology when designing the network structure. By selecting activation functions and data representation methods suitable for quantization, and constructing a network structure that is easy to prune, the optimized MobileNetV4 model can reduce the size of the model and computing resource consumption while maintaining high-precision predictions, so as to achieve real-time processing on the vehicle computing platform. This feature enables the present invention to perform real-time data fusion with the vehicle's existing IMU system to provide more accurate and stable roll angle measurement results.

[0090] In addition, the present invention also introduces a dynamic feature selection and attention mechanism. In the feature fusion stage, the model can learn the importance of different features to the roll angle prediction, and select and weight features according to their importance. This mechanism enables the model to pay more attention to features that have a significant impact on the prediction results, while ignoring those features that are irrelevant or have large noise, thereby further improving the accuracy of the prediction. This dynamic feature selection capability enables the present invention to perform well in long-term measurements, effectively compensating for the cumulative errors that may occur in the long-term use of the IMU.

[0091] Through the improved MobileNetV4 network, a high-precision and environmentally adaptable vehicle roll angle measurement model can be obtained. It can accurately analyze common structured road scenes, improve prediction accuracy through adaptive convolution kernels and dynamic feature selection, and based on lightweight model design, it can facilitate the deployment of the model on the vehicle computing platform. This deep learning method complements traditional IMU measurements, especially in the case of frequent acceleration and deceleration or long-term driving of the vehicle, and can provide more stable and accurate roll angle information, providing reliable data support for vehicle stability control and safety systems.

[0092] The specific implementation of step five is:

[0093] After completing the design of the MobileNetV4 model, the present invention integrates the model with the image processing method to further improve the accuracy and environmental adaptability of vehicle roll angle measurement. The specific integration steps are as follows:

[0094] First, the present invention introduces a fusion mechanism based on weighted average to combine the advantages of image processing methods and deep learning models. In this mechanism, the weights of image processing results and deep learning prediction results are first adjusted dynamically according to the current road conditions, lighting conditions and other external environmental factors. For scenes with good lighting conditions and smooth roads, the image processing method has a higher weight, while in scenes with complex lighting and rugged roads, the weight of the deep learning model is increased accordingly to cope with complex background and noise interference.

[0095] Subsequently, at each time step, the weighted average method is used to fuse the image processing results and the deep learning prediction results to generate a final roll angle prediction value.

[0096] Assume that the prediction result of the image processing method is θ img , the prediction result of the deep learning model is θ dl , then the final roll angle prediction value can be calculated by the following formula:

[0097] θ final =w img (t)·θ img +w dl (t)·θ dl

[0098] In order to achieve adaptive dynamic adjustment, the present invention also introduces an adaptive correction mechanism to further optimize the accuracy of the fusion results. The mechanism analyzes the fusion results of multiple consecutive time steps and dynamically adjusts the weight value. In this strategy, the system will adjust the weight of the prediction results of the image processing method and the prediction results of the deep learning model in real time according to the changes in the external environment, such as light intensity, road surface flatness, and vehicle speed.

[0099] According to the roll angle prediction formula above, its weight value is updated by the following formula:

[0100]

[0101]

[0102] Among them, α is the adjustment coefficient, which controls the sensitivity of the system to errors. A larger α value will make the system more sensitive to errors, resulting in more drastic changes in weights; while a smaller α value will make the weight adjustment smoother.

[0103] The normalization factor Z(t) is used to ensure that the sum of the two weights is 1, and the calculation formula is as follows:

[0104] Z(t)=exp(-α·|θ img (t)-θ final (t)|)+exp(-α·|θ dl (t)-θ final (t)|)

[0105] In this way, the system is able to automatically adjust the weights at each time step based on the prediction accuracy of different methods. For example, when the prediction error of the image processing method is small, w img (t+1) will increase, which means that the system will rely more on the results of image processing. On the contrary, if the prediction error of the deep learning model is small, w dl (t+1) will increase, making the system more dependent on the predictions of the deep learning model.

[0106] Ultimately, this adaptive weight adjustment mechanism ensures that the system can automatically optimize the fusion results under different environmental and road conditions, improving the overall accuracy and environmental adaptability of the vehicle roll angle prediction. Through real-time fusion with IMU data, the method of the present invention can effectively compensate for the accumulated error of IMU and provide more reliable and accurate roll angle information for the vehicle stability control system.

[0107] It should be noted that the above content only illustrates the technical idea of ​​the present invention and cannot be used to limit the protection scope of the present invention. For ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principle of the present invention. These improvements and modifications all fall within the protection scope of the claims of the present invention.

Claims

1. A method for measuring vehicle roll angle using images on structured road conditions, characterized in that: The following steps are involved: Step S1, installing two laser emitters along the transverse axis on the outer side of the bottom of the vehicle suspension so that they form parallel linear light spots on the ground; installing an imaging device near one of the laser emitters; adjusting the position of the laser emitter so that the light beam is within the field of view of the imaging device and parallel to the longitudinal axis of the vehicle; the imaging device continuously captures the image of the laser beam on the ground to detect the roll angle of the vehicle in real time; Step S2, installing the laser transmitter and the imaging device on the platform to simulate the installation configuration of the vehicle suspension; during the experiment, simulating the vehicle roll by tilting the platform, and using the inclination sensor to record the inclination angle, and calibrating the collected laser images; repeating the experiment under common structured road conditions to establish a more comprehensive data set, so as to complete the inclination measurement suitable for structured road conditions; Step S3, based on the data set established in step S2, the collected images are processed based on Canny edge detection and Hough transform algorithms to calculate the geometric distance between the two laser beams, and a relationship model between the inclination angle and the laser beam distance is constructed, and the least squares method is used for fitting to accurately predict the vehicle roll angle under structured road conditions; Step S4: In order to improve the accuracy and environmental adaptability of tilt angle detection, the roll angle of the vehicle is predicted by using an improved MobileNetV4 model; the improved MobileNetV4 model adopts an adaptive convolution kernel and a dynamic feature selection mechanism to enhance the processing capability of complex ground reflection information, and supports quantization and pruning technology to make it lightweight and convenient for real-time deployment on the vehicle platform; Step S5: Using a fusion mechanism based on weighted average, the image processing in step S3 is combined with the prediction results of the deep learning model in step S4, and the weights are dynamically adjusted through adaptive correction to improve the accuracy and stability of the vehicle roll angle measurement.

2. The method for measuring vehicle roll angle using images applicable to structured road conditions according to claim 1, characterized in that: The specific steps of step S1 are as follows: Step S11, laser light source arrangement: two laser emitters are installed side by side horizontally along the transverse axis of the vehicle on the outside of the bottom of the vehicle suspension to ensure that a clear and parallel linear light spot is formed on the ground; Step S12, imaging unit setting: select an imaging device and install it near one of the laser transmitters, ensuring that the vertical height of the device from the ground is between 30 and 50 centimeters; the imaging device supports shooting RGB 24-bit color images and has a resolution of at least 480×320 to ensure image quality; Step S13, light spot positioning: adjusting the positions of the two laser emitters so that the linear light spots formed by them on the ground are within the field of view of the imaging device, and ensuring that each light beam is parallel to the longitudinal axis of the vehicle; Step S14, image capture: In order to detect the roll angle of the vehicle in real time, the imaging device continuously captures the image of the laser beam on the ground at a speed of about 30 frames per second.

3. The method for measuring vehicle roll angle using images applicable to structured road conditions according to claim 1, characterized in that: The specific steps of step S2 are as follows: Step S21: After the laser transmitter and the imaging device are installed, the laser road image data under common structured road conditions are collected by constructing an experimental device; in order to simulate the roll scene of the vehicle while ensuring safety, a platform is used instead of the vehicle suspension, and the laser transmitter and the imaging device are installed on the platform, ensuring that the installation position is consistent with the installation on the actual vehicle; In addition, a high-precision tilt sensor was installed under the platform for calibration of the data set; Step S22, during the experiment, the platform is tilted to simulate the vehicle roll, and the roll angle is controlled between -15 degrees and 15 degrees; by adjusting the tilt angle of the platform, 80 laser road surface images are taken under the same road surface conditions, and the angles corresponding to these images are evenly distributed between -15 degrees and 15 degrees; the roll angle recorded in real time by the tilt sensor is used to calibrate the collected image data to ensure the accuracy of the data set; Step S23: After the establishment of the data set for a single road type is completed, the experimental scope is further expanded to cover other structured road conditions: asphalt, mud, cement and gravel roads; for each road type, the corresponding laser road surface images are systematically collected according to the established method; through this method, a comprehensive data set is constructed, which can represent various road conditions that the vehicle may encounter during actual driving, thereby providing a solid data foundation for the accurate measurement of the vehicle roll angle.

4. The method for measuring vehicle roll angle using images applicable to structured road conditions according to claim 1, characterized in that: The specific steps of step S3 are as follows: First, the images collected by the imaging device are converted into grayscale images to reduce the data dimension and improve the processing efficiency. To remove the noise interference in the image, the adaptive Gaussian blur technology is used to dynamically adjust the smoothing parameters to further improve the image quality and provide high-quality input data for subsequent feature extraction. Next, the Canny edge detection algorithm is used to extract edge information in the image. The specific method is as follows: first, the gradient of the preprocessed image is calculated, and the area with the most obvious brightness change in the image is identified by calculating the brightness change in the horizontal and vertical directions, that is, the area most likely to be the edge of the laser beam, and the non-maximum suppression strategy is used to screen out the strongest local edges to avoid repeated identification and blurred edge responses; the edges are classified through a dual threshold strategy: that is, high-intensity edges are immediately identified, and low-intensity edges are determined to be effective through their connection relationship with high-intensity edges; this dual threshold mechanism effectively suppresses the noise contours in the background, ensuring that only clear edges related to the target features are retained; through the edge detection algorithm, the output edge image only contains edges with structural significance, laying the foundation for the subsequent laser beam feature extraction; Most of the noise is removed by edge detection, but some contours in the ground background are still included. In order to accurately extract the two laser beams, the edge detection results are first analyzed to further eliminate isolated points or small edge areas. This step calculates the area or perimeter of each edge area and excludes areas that are smaller than expected or do not meet the characteristics of the laser beam based on the set threshold, thereby reducing background interference and highlighting the characteristics of the laser beam. The Hough transform is then used to extract the laser beam features in the image. The Hough transform maps each edge point in the image to the parameter space and identifies the point set that meets the straight line equation. The specific steps are as follows: Polar coordinate space mapping: For each edge point, according to its image coordinates (x, y), it is converted into line parameters (ρ, θ) in polar coordinates, where ρ=x·cos(θ)++y·sin(θ) In this process, each edge point will form a curve in the ρ-θ parameter space; When multiple edge points are collinear, the curves they form in the ρ-θ space intersect at the same point; this intersection represents a straight line in the image; by accumulating the number of occurrences of these intersections, find the points with the most occurrences in the parameter space, which correspond to the significant straight lines in the image; In order to ensure the accuracy of detection, a dynamic threshold adjustment mechanism is introduced in the Hough transform, which only retains the points whose cumulative times in the parameter space exceed a certain threshold; this threshold is dynamically adjusted according to the specific application environment and the intensity of the laser beam to ensure that only significant straight lines are detected while ignoring irrelevant background contours; After the straight line detection is completed, multiple straight lines may be detected in the image. In order to accurately extract the two laser beams, the following strategies are used for screening and optimization: (1) Redundant line detection: First, the distance between each pair of detected lines is calculated, and the difference is calculated based on their ρ parameters. For lines with a distance that is too small, they are considered to be redundant detection results of the same line. Then, the θ angle difference between each pair of lines is calculated. If the angle difference between the two lines is less than a certain threshold and their distance difference is also within a certain range, they are considered to be the same line and merged. This step generates a more stable line parameter by calculating the mean or median. (2) Laser beam screening: To ensure that only two laser beams are extracted, the lines are further screened based on the geometric position, the length of the line, and the relationship with the emission direction of the laser emitter. According to the preset geometric model, the two lines that meet the expected position and direction of the laser beam are quickly identified, and other background lines that do not meet the conditions are excluded; After completing the image processing step, the centerline equations of the two laser beams are obtained; then, the horizontal axis is set to the actual roll angle of the vehicle, and these angle data are synchronously recorded by the tilt sensor during image acquisition; the vertical axis is set to the pixel position of the centerline of the two laser beams, which is the horizontal position of the centerline in the image calculated after the image processing step; then, the least squares method is used to fit the curve to these data points; the least squares method determines the best fitting curve by minimizing the sum of squares of the errors, thereby establishing a regression model that can accurately represent the relationship between the vehicle roll angle and the centerline pixel position.

5. The method for measuring vehicle roll angle using images applicable to structured road conditions according to claim 2, characterized in that: The specific steps of step S4 are as follows: In order to further improve the accuracy and environmental adaptability of the vehicle roll angle measurement system, the improved MobileNetV4 model is used to predict the vehicle roll angle; MobileNetV4 adopts an inverted bottleneck block IRB structure and effectively reduces the number of model parameters and calculations through depthwise separable convolution while maintaining good performance; in addition, MobileNetV4 further enhances the model's ability to capture multi-scale features by adopting a hierarchical interconnection approach and feature fusion strategy; In order to better meet the needs of vehicle roll angle measurement, especially to provide stable and reliable measurement results under structured road conditions, improvements were made on the basis of MobileNetV4. First, an adaptive convolution kernel mechanism was introduced, which enables the model to dynamically adjust the parameters of the convolution kernel according to the shape and intensity of the laser beam in the input image, thereby more accurately extracting features related to the roll angle. This optimization significantly improves the model's ability to process complex ground reflection information, enhancing prediction accuracy and adaptability under structured road conditions; Secondly, considering the resource constraints faced by the model in actual deployment, the compatibility of quantization and pruning technology is fully considered when designing the network structure; by selecting activation functions and data representation methods suitable for quantization, and building a network structure that is easy to prune, the optimized MobileNetV4 model can reduce the size of the model and computing resource consumption while maintaining high-precision prediction, so as to achieve real-time processing on the vehicle computing platform; this feature enables the present invention to perform real-time data fusion with the vehicle's existing IMU system, providing more accurate and stable roll angle measurement results; In addition, dynamic feature selection and attention mechanisms are introduced; In the feature fusion stage, the model can learn the importance of different features to the roll angle prediction, and select and weight the features according to their importance. This mechanism enables the model to pay more attention to the features that have a significant impact on the prediction results, while ignoring those that are irrelevant or have large noise, further improving the accuracy of the prediction. This dynamic feature selection capability enables the present invention to perform well in long-term measurements, effectively compensating for the cumulative errors that may occur in the long-term use of the IMU. Through the improved MobileNetV4 network, a high-precision and environmentally adaptable vehicle roll angle measurement model is obtained; it can accurately analyze structured road scenes, improve prediction accuracy through adaptive convolution kernels and dynamic feature selection, and based on lightweight model design, it can facilitate the deployment of the model on the on-board computing platform; this deep learning method complements traditional IMU measurements, especially in cases where the vehicle frequently accelerates and decelerates or drives for a long time, it can provide more stable and accurate roll angle information, and provide reliable data support for vehicle stability control and safety systems.

6. The method for measuring vehicle roll angle using images applicable to structured road conditions according to claim 3, characterized in that: The specific steps of step S5 are as follows: After the optimization of the MobileNetV4 model is completed, the model is fused with the image processing method to further improve the accuracy and environmental adaptability of vehicle roll angle measurement; the specific fusion steps are as follows: Firstly, a fusion mechanism based on weighted average is introduced to combine the advantages of image processing methods and deep learning models. In this mechanism, the weights of image processing results and deep learning prediction results are dynamically adjusted according to the current road conditions, lighting conditions and other external environmental factors. For scenes with good lighting conditions and smooth roads, the image processing method has a higher weight, while in scenes with complex lighting and rough roads, the weight of the deep learning model increases accordingly to cope with complex backgrounds and noise interference; Then, at each time step, the weighted average method is used to fuse the image processing results and the deep learning prediction results to generate a final roll angle prediction value; Assume that the prediction result of the image processing method is θ img , the prediction result of the deep learning model is θ dl , then the final predicted roll angle θ final It can be calculated by the following formula: i final =w img (t)·θ img +w dl (t)·θ dl In order to achieve adaptive dynamic adjustment, an adaptive correction mechanism is introduced to further optimize the accuracy of the fusion results. This mechanism analyzes the fusion results of multiple consecutive time steps and dynamically adjusts the weight value. In this strategy, the system will adjust the weight of the prediction results of the image processing method and the prediction results of the deep learning model in real time according to the changes in the external environment, including light intensity, road surface flatness, and vehicle speed factors. According to the roll angle prediction formula, its weight value W img and w dl They are updated by the following formulas respectively: Among them, α is the adjustment coefficient, which controls the sensitivity of the system to errors; a larger α value will make the system more sensitive to errors, resulting in more drastic changes in weights; while a smaller α value will make the weight adjustment smoother; The normalization factor Z(t) is used to ensure that the sum of the two weights is 1, and the calculation formula is as follows: Z(t)=exp(-α·|θ img (t)-θ final (t)|)+exp(-α·|θ dl (t)-θ final (t)|) In this way, the system can automatically adjust the weights at each time step according to the prediction accuracy of different methods; when the prediction error of the image processing method is small, w img (t+1) will increase, which means that the system will rely more on the results of image processing; on the contrary, if the prediction error of the deep learning model is small, w dl (t+1) will increase, making the system more dependent on the predictions of the deep learning model; Ultimately, this adaptive weight adjustment mechanism ensures that the system can automatically optimize the fusion results under different environments and road conditions, improving the overall accuracy and environmental adaptability of the vehicle's roll angle prediction; through real-time fusion with IMU data, it can effectively compensate for the IMU's accumulated errors and provide the vehicle stability control system with more reliable and accurate roll angle information.

Images