Automobile Tire Pressure Regulation Method and System Based on Multimodal Fusion and Attention Mechanism

Abstract

The present invention discloses an automobile tire pressure regulation method and system based on multimodal fusion and attention mechanism, belonging to the field of tire pressure regulation, and comprising the following steps: S1, collecting multimodal data and preprocessing; S2, constructing and training a multimodal fusion model based on historical multimodal data, the multimodal fusion model including a road surface type recognition sub-model, a temperature change sub-model, and a tire state sub-model; S3, inputting the real-time collected multimodal data into the multimodal fusion model to obtain the road surface type, the prediction result of the tire thermal expansion effect, and the evaluation result of the tire wear level, and then combining the gated attention mechanism to weight multi-source features to dynamically generate tire pressure regulation suggestions. By adopting the above-mentioned automobile tire pressure regulation method and system based on multimodal fusion and attention mechanism, through multi-source data collection, data preprocessing, multimodal fusion model construction, real-time monitoring, fault warning, etc., precise control and all-round monitoring of the automobile tire pressure are realized.

Description

Technical Field

[0001] The present invention relates to the technical field of tire pressure regulation, and particularly to an automobile tire pressure regulation method and system based on multi-modal fusion and attention mechanism. Background Art

[0002] Automobile tire pressure is one of the important factors affecting driving safety. A reasonable tire pressure setting can reduce the rolling resistance of automobile tires, improve tire life, reduce the risk of tire blowout, and at the same time reduce the load on the shock absorption system and improve the grip and braking performance of the automobile. In the prior art, generally, the automobile tire is inflated to reach a preset pressure and then driven at this preset pressure. The preset pressure is usually a fixed value determined under standard conditions. In fact, during the driving process of the vehicle, various driving scenarios will be encountered. For example, when driving on rough terrains such as unpaved rural roads or mountain roads, the tires need to withstand irregular impacts and lateral forces of different degrees. The fixed preset pressure may not be sufficient to adapt to these extreme conditions, resulting in an increased risk of tire wear and puncture.

[0003] In view of the above problems, the prior art has further developed an automobile tire pressure control method and system disclosed in CN202410541703.9. This invention patent first obtains the current tire pressure of the automobile and the road surface image in the driving direction of the automobile, then determines the corresponding tire pressure conversion coefficient according to the current tire pressure and the road surface image, and finally adjusts the automobile tire pressure according to the tire pressure conversion coefficient, so that the tire pressure of the automobile can be adjusted in real time according to the road surface condition in the driving direction of the automobile and the current tire pressure of the automobile, so that when the automobile is driving under different road surface conditions, its tire pressure can be maintained at the optimal tire pressure, improving the driving safety of the automobile.

[0004] However, the factors affecting tire pressure are complex, including weather changes such as thermal expansion and contraction, and the influence of tire wear on tire pressure. Adjusting tire pressure solely based on road surface conditions has poor adjustment effects and cannot adapt to complex driving environments. Summary of the Invention

[0005] The purpose of the present invention is to provide an automobile tire pressure regulation method and system based on multi-modal fusion and attention mechanism to solve the above technical problems.

[0006] To achieve the above purpose, the present invention provides an automobile tire pressure regulation method based on multi-modal fusion and attention mechanism, including the following steps:

[0007] S1. Collect multi-modal data and preprocess it: Synchronously collect vehicle driving information, road surface image information, environmental information, and tire information;

[0008] S2. Construct and train a multi-modal fusion model based on historical multi-modal data. The multi-modal fusion model includes a road surface type recognition sub-model, a temperature change sub-model, and a tire status sub-model;

[0009] S3. Input the real-time collected multi-modal data into the multi-modal fusion model to obtain the road surface type, the prediction result of the tire thermal expansion effect, and the evaluation result of the tire wear grade. Then, combined with the gated attention mechanism to weight multi-source features, dynamically generate tire pressure regulation suggestions.

[0010] Preferably, in step S1, the vehicle driving information includes vehicle speed, wheel speed, acceleration, and vehicle body inclination data;

[0011] The environmental information includes temperature and humidity, rainfall, air pressure, and road surface temperature distribution image;

[0012] The tire information includes tire pressure waveform information and tire ground pressure distribution information;

[0013] The preprocessing includes performing gamma correction, CLAHE enhancement, and sliding window mean filtering on the road surface image information and the road surface temperature distribution image in sequence, and performing time series data enhancement processing on the tire information, vehicle driving information, temperature and humidity, rainfall, and air pressure.

[0014] Preferably, in step S2, the road surface type recognition sub-model is an EfficientNet-B5 model trained with the labeled road surface image information;

[0015] The temperature change sub-model is a bidirectional LSTM model trained with historical temperature and humidity data;

[0016] The tire status sub-model is a three-dimensional convolutional network trained with historical tire information.

[0017] Preferably, during the training process of the EfficientNet-B5 model, the first 10 layers of the EfficientNet-B5 network framework are frozen, and the learning rate is set to 1e-5 to update the training parameters of the top 3 layers.

[0018] Preferably, step S3 specifically includes the following steps:

[0019] S31. Calculate the weight factors:

[0020] (1);

[0021] In the formula, , and respectively represent the weight factors of the road surface type, the tire thermal expansion effect, and the tire wear grade; represents the temperature change rate; represents the precipitation intensity; represents the friction coefficient; represents the slope; represents the tire wear grade; represents the tire pressure fluctuation range; represents the unevenness degree of the ground pressure distribution;

[0022] S32. Multimodal fusion:

[0023] (2);

[0024] In the formula, represents the fused feature vector; represents the road surface type feature vector; represents the tire thermal expansion effect feature vector; represents the tire wear grade feature vector;

[0025] S33. Calculate the weights by the gated attention mechanism

[0026] S331. Calculate the attention score vector through a multi-layer perceptron :

[0027] (3);

[0028] In the formula, represents the multi-layer perceptron calculation function;

[0029] Use the Softmax function to normalize the attention scores to obtain the attention weight vector:

[0030] (4);

[0031] In the formula, represents the th attention weight corresponding to the feature; and respectively represent the th feature and the th feature corresponding attention score vectors; represents the dimension of the fused feature vector ;

[0032] S34. Calculate the weighted feature vector :

[0033] (5);

[0034] In the formula, represents element-wise multiplication;

[0035] S37. Calculate the recommended value for tire pressure regulation through a linear layer :

[0036] (6);

[0037] In the formula, represents the weight vector of the linear layer, and the weight vector of the linear layer has the same dimension as ; represents the bias term.

[0038] Preferably, in step S3, when the tire pressure exceeds the threshold, an alarm is triggered, and the nearest repair point is recommended based on the in-vehicle navigation.

[0039] A system for an automotive tire pressure regulation method based on multimodal fusion and attention mechanism includes:

[0040] A multimodal data and preprocessing module for synchronously collecting vehicle driving information, road surface image information, environmental information, and tire information;

[0041] A multimodal fusion model construction and training module for constructing and training a multimodal fusion model based on historical multimodal data, and the multimodal fusion model includes a road surface type recognition sub-model, a temperature change sub-model, and a tire status sub-model;

[0042] A tire pressure regulation recommendation generation module for inputting the real-time collected multimodal data into the multimodal fusion model to obtain the road surface type, tire thermal expansion effect prediction result, and tire wear level evaluation result, and then dynamically generating a tire pressure regulation recommendation by combining the gated attention mechanism to weight multi-source features;

[0043] A trigger alarm module for triggering an alarm when the tire pressure exceeds the threshold and recommending the nearest repair point based on the in-vehicle navigation.

[0044] Preferably, the multimodal data and preprocessing module includes a vehicle driving information collection unit, a road surface image information collection unit, an environmental information collection unit, and a tire information collection unit. Among them, the vehicle driving information collection unit includes a nine-axis IMU sensor arranged on the vehicle body and a wheel speed sensor for collecting wheel speed information, and the nine-axis IMU sensor is used to collect vehicle speed, acceleration, and vehicle body inclination data;

[0045] The road surface image information collection unit includes a vehicle-mounted camera with a resolution of 1280×720 arranged on the vehicle body;

[0046] The environmental information collection unit includes a vehicle-mounted temperature and humidity sensor, a rain sensor, a barometric pressure sensor, and a FLIR thermal imager for collecting road surface temperature distribution images;

[0047] The tire information acquisition unit includes an implantable MEMS sensor for acquiring tire pressure waveform information and a pressure sensor for acquiring tire ground pressure distribution information.

[0048] Preferably, the temperature and humidity sensor is an SHT35 sensor, the rainfall sensor is an integrated optical rain gauge, the air pressure sensor is a BMP-280 sensor, and the resolution of the FLIR thermal imager is 640×512.

[0049] Therefore, the present invention adopts the above-mentioned automotive tire pressure regulation method and system based on multi-modal fusion and attention mechanism, and the beneficial effects are as follows:

[0050] 1. Comprehensively and accurately obtain various types of data during vehicle driving, providing a solid data basis for subsequent precise tire pressure adjustment. At the same time, preprocess the collected data, including image data enhancement (gamma correction, CLAHE enhancement, sliding window mean filtering, etc.) and time series data enhancement, improve the data quality, facilitate the model to extract features, reduce noise interference, avoid overfitting, and further improve the response inference speed and accuracy of the model;

[0051] 2. Construct a multi-modal fusion model architecture including three sub-models of road surface recognition, temperature change and tire status, comprehensively consider the influence of various factors such as weather, road surface and tire conditions on tire pressure, and then weight multi-source features through the gated attention mechanism, and adjust the weights of each factor by itself, so that the tire pressure adjustment suggestion is more suitable for the actual driving environment, effectively reduce tire wear, and extend the service life of the tire.

[0052] In summary, the present invention can not only dynamically adjust the tire pressure according to various factors in a complex driving environment, reduce tire wear, and extend the service life of the tire and the vehicle, but also ensure driving safety in real time, give an early warning in time when the tire pressure is abnormal and provide maintenance convenience. At the same time, accurate road surface recognition helps to optimize the vehicle use and maintenance costs, and has significant advantages and important application values in the field of automotive tire pressure control.

[0053] Next, through the drawings and embodiments, the technical solutions of the present invention will be further described in detail. Brief Description of the Drawings

[0054] Figure 1 It is a flowchart of the automotive tire pressure regulation method based on multi-modal fusion and attention mechanism of the present invention. Detailed Embodiments

[0055] In order to make the objectives, technical solutions, and advantages of the embodiments of the present invention more clearly understood, the following further details the embodiments of the present invention in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely used to explain the embodiments of the present invention and are not used to limit the embodiments of the present invention. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts fall within the scope of protection of this application. Examples of the embodiments are shown in the accompanying drawings, where the same or similar reference numerals represent the same or similar elements or elements with the same or similar functions throughout.

[0056] It should be noted that the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or server that includes a series of steps or units does not necessarily have to be limited to those steps or units clearly listed, but may include other steps or units not clearly listed or inherent to these processes, methods, products, or devices.

[0057] The following details the embodiments of the present invention in conjunction with the accompanying drawings.

[0058] As Figure 1 shown, the method for regulating vehicle tire pressure based on multimodal fusion and attention mechanism includes the following steps:

[0059] S1. Collect multimodal data and preprocess it: Synchronously collect vehicle driving information, road surface image information, environmental information, and tire information;

[0060] In step S1, the vehicle driving information includes vehicle speed, wheel speed, acceleration, and body inclination data;

[0061] The environmental information includes temperature and humidity, rainfall, air pressure, and road surface temperature distribution image;

[0062] The tire information includes tire pressure waveform information and tire ground pressure distribution information;

[0063] The preprocessing includes performing gamma correction, CLAHE enhancement, and sliding window mean filtering (window size = 4) on the road surface image information and the road surface temperature distribution image in sequence, and performing time series data enhancement processing on the tire information, vehicle driving information, temperature and humidity, rainfall, and air pressure. By enhancing the road surface image information and the road surface temperature distribution image, it is convenient for the model to extract features and further improve the response speed of the model.

[0064] S2. Construct and train a multi-modal fusion model based on historical multi-modal data. The multi-modal fusion model includes a road surface type recognition sub-model, a temperature change sub-model, and a tire status sub-model. Through the cross-regional fusion of multiple sub-models, multiple factors that have a greater impact on the tires act on the model simultaneously, and based on the different weights of each model, the appropriate tire pressure is automatically adjusted to ensure less tire wear.

[0065] In step S2, the road surface type recognition sub-model is an EfficientNet-B5 model trained with labeled road surface image information, which can recognize multiple road surface types.

[0066] During the training process of the EfficientNet-B5 model, the first 10 layers of the EfficientNet-B5 network framework are frozen, and the learning rate is set to 1e-5 to update the training parameters of the top 3 layers.

[0067] The temperature change sub-model is a bidirectional LSTM model trained with historical temperature and humidity data.

[0068] The tire status sub-model is a three-dimensional convolutional network trained with historical tire information.

[0069] In this embodiment, 100,000 groups of full life cycle data (tire pressure + wear + mileage) can also be input, and a temporal convolutional network (TCN) is used to capture long-term dependencies to predict the tire life.

[0070] S3. Input the real-time collected multi-modal data into the multi-modal fusion model to obtain the road surface type, the prediction result of the tire thermal expansion effect (predicting the thermal expansion effect in the next 15 minutes), and the evaluation result of the tire wear grade. Then, combined with the gated attention mechanism to weight multi-source features, dynamically generate tire pressure regulation suggestions. Step S3 specifically includes the following steps:

[0071] S31. Calculate the weight factors:

[0072] (1);

[0073] In the formula, , and respectively represent the weight factors of the road surface type, the tire thermal expansion effect, and the tire wear grade; represents the temperature change rate; represents the precipitation intensity; represents the friction coefficient; represents the slope; represents the tire wear grade; represents the tire pressure fluctuation amplitude; represents the uneven degree of the ground pressure distribution;

[0074] S32. Multimodal fusion:

[0075] (2);

[0076] Wherein, represents the fused feature vector; represents the road surface type feature vector; represents the tire thermal expansion effect feature vector; represents the tire wear grade feature vector;

[0077] S33. Calculate the weights using the gated attention mechanism;

[0078] S331. Calculate the attention score vector through a multi-layer perceptron :

[0079] (3);

[0080] Wherein, represents the calculation function of the multi-layer perceptron;

[0081] Normalize the attention scores using the Softmax function to obtain the attention weight vector:

[0082] (4);

[0083] Wherein, represents the th attention weight corresponding to the feature; and respectively represent the th feature and the th attention score vectors corresponding to the features; represents the fused feature vector dimension;

[0084] S34. Calculate the weighted feature vector :

[0085] (5);

[0086] Wherein, represents element-wise multiplication;

[0087] S37. Calculate the tire pressure regulation recommendation value through a linear layer :

[0088] (6);

[0089] Wherein, represents the weight vector of the linear layer, and the weight vector of the linear layer The dimension is the same as ; represents the bias term.

[0090] For example, for an asphalt road surface, maintain the standard tire pressure (2.4 bar ± 0.03 bar); for a snow-covered road surface, increase the pressure by 0.15 bar to enhance the tread rigidity and reduce the side-slip probability by 37%; for high-speed driving conditions (vehicle speed > 100 km / h), increase the pressure by 0.05 bar for every 10 km / h increase in speed to suppress deformation. When the detected tire pressure drop rate > 0.3 bar / s: Activate the corresponding wheel brake pre-pressurization; limit the vehicle speed to below 80 km / h. Additionally, when the tire temperature gradient > 2 °C / min, trigger the three-stage pressure relief valve to reduce the pressure by 0.2 bar within 0.5 seconds.

[0091] In step S3, in this embodiment, the tire pressure is updated every 1000 meters to adjust the tire pressure in real time. When the tire pressure exceeds the threshold, an alarm is triggered, and the nearest repair point is recommended based on the in-vehicle navigation.

[0092] A system for an automotive tire pressure regulation method based on multi-modal fusion and attention mechanism, comprising: a multi-modal data and preprocessing module for synchronously collecting vehicle driving information, road surface image information, environmental information, and tire information; a multi-modal fusion model construction and training module for constructing and training a multi-modal fusion model based on historical multi-modal data, and the multi-modal fusion model includes a road surface type recognition sub-model, a temperature change sub-model, and a tire state sub-model; a tire pressure regulation recommendation generation module for inputting the real-time collected multi-modal data into the multi-modal fusion model to obtain the road surface type, tire thermal expansion effect prediction result, and tire wear level evaluation result, and then dynamically generating tire pressure regulation recommendations by combining the gated attention mechanism to weight multi-source features; a trigger alarm module for triggering an alarm when the tire pressure exceeds the threshold and recommending the nearest repair point based on the in-vehicle navigation.

[0093] The multi-modal data and preprocessing module includes a vehicle driving information collection unit, a road surface image information collection unit, an environmental information collection unit, and a tire information collection unit. The vehicle driving information collection unit includes a nine-axis IMU sensor disposed on the vehicle body and a wheel speed sensor for collecting wheel speed information. The nine-axis IMU sensor is used to collect vehicle speed, acceleration, and vehicle body inclination data; the road surface image information collection unit includes an in-vehicle camera with a resolution of 1280×720 disposed on the vehicle body; the environmental information collection unit includes an in-vehicle temperature and humidity sensor, a rain sensor, a barometric pressure sensor, and a FLIR thermal imager for collecting road surface temperature distribution images; the tire information collection unit includes an implantable MEMS sensor for collecting tire pressure waveform information and a pressure sensor for collecting tire ground pressure distribution information.

[0094] In this embodiment, different observation angles can also be simulated by randomly rotating (±15°) the vehicle-mounted camera and the FLIR thermal imager.

[0095] The temperature and humidity sensor is an SHT35 sensor, the rainfall sensor is an integrated optical rain gauge, the barometric pressure sensor is a BMP-280 sensor, and the resolution of the FLIR thermal imager is 640×512.

[0096] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit them. Although the present invention has been described in detail with reference to the preferred embodiments, those of ordinary skill in the art should understand that they can still modify or equivalently replace the technical solutions of the present invention, and these modifications or equivalent replacements cannot make the modified technical solutions deviate from the spirit and scope of the technical solutions of the present invention.

Claims

1. A method for controlling automobile tire pressure based on multimodal fusion and attention mechanism, characterized in that: The following steps are involved: S1. Collect and preprocess multimodal data: synchronously collect vehicle driving information, road image information, environmental information and tire information; In step S1, the vehicle driving information includes vehicle speed, wheel speed, acceleration and vehicle body inclination angle data; Environmental information includes temperature and humidity, rainfall, air pressure, and road surface temperature distribution images; Tire information includes tire pressure waveform information and tire ground contact pressure distribution information; The preprocessing includes gamma correction, CLAHE enhancement and sliding window mean filtering of the road surface image information and road surface temperature distribution image, as well as time series data enhancement processing of tire information, vehicle driving information, temperature and humidity, rainfall and air pressure; S2. construct and train a multimodal fusion model based on historical multimodal data, where the multimodal fusion model includes a road type recognition sub-model, a temperature change sub-model, and a tire status sub-model; In step S2, the road type recognition sub-model is an EfficientNet-B5 model trained with annotated road image information; During the training of the EfficientNet-B5 model, the EfficientNet-B5 network framework with the first 10 layers frozen was used, and the learning rate was set to 1e-5 to update the training parameters of the top 3 layers; Step S3 specifically includes the following steps: S31. Calculate the weight factor: （1）； In the formula, , and The weighting factors for road surface type, tire thermal expansion effect, and tire wear level, respectively; Indicates the rate of temperature change; Indicates precipitation intensity; represents the friction coefficient; Indicates slope; Indicates tire wear grade; Indicates the tire pressure fluctuation range; Indicates the uneven distribution of ground pressure; S32, Multimodal Fusion: （2）； In the formula, represents the fused feature vector; Represents the road surface type feature vector; Represents the eigenvector of tire thermal expansion effect; represents the tire wear grade feature vector; S33, gated attention mechanism calculates weights; S331. Calculate the attention score vector through a multi-layer perceptron : （3）； In the formula, Represents the multi-layer perceptron calculation function; The attention score is normalized using the Softmax function to obtain the attention weight vector: （4）； In the formula, Indicates The attention weight corresponding to each feature; and Respectively represent Features and The attention score vector corresponding to each feature; Represents the fused feature vector Dimensions; S34. Calculate the weighted eigenvector : （5）； In the formula, Indicates the multiplication of corresponding elements; S37, calculate the recommended value of tire pressure control through linear layer : （6）； In the formula, represents the weight vector of the linear layer, and the weight vector of the linear layer Dimensions and same; represents the bias term; The temperature change sub-model is a bidirectional LSTM model trained with historical temperature and humidity data; The tire status sub-model is a three-dimensional convolutional network trained with historical tire information; S3. Input the multimodal data collected in real time into the multimodal fusion model to obtain the road type, tire thermal expansion effect prediction results and tire wear level evaluation results, and then combine the gated attention mechanism to weight multi-source features to dynamically generate tire pressure control suggestions.

2. The automobile tire pressure control method based on multimodal fusion and attention mechanism according to claim 1, characterized in that: In step S3, when the tire pressure exceeds the threshold, an alarm is triggered and the nearest maintenance point is recommended based on the vehicle navigation.

3. The system for automobile tire pressure control method based on multimodal fusion and attention mechanism as described in claim 2 above is characterized in that: include: Multimodal data and preprocessing module, used to simultaneously collect vehicle driving information, road image information, environmental information and tire information; A multimodal fusion model construction training module is used to construct and train a multimodal fusion model based on historical multimodal data, and the multimodal fusion model includes a road type recognition sub-model, a temperature change sub-model, and a tire status sub-model; The tire pressure control suggestion generation module is used to input the multimodal data collected in real time into the multimodal fusion model to obtain the road type, tire thermal expansion effect prediction results and tire wear level evaluation results, and then combine the gated attention mechanism to weight multi-source features to dynamically generate tire pressure control suggestions; The trigger alarm module is used to trigger an alarm when the tire pressure exceeds the threshold and recommend the nearest maintenance point based on the vehicle navigation.

4. The system of the automobile tire pressure control method based on multimodal fusion and attention mechanism according to claim 3 is characterized in that: The multimodal data and preprocessing module includes a vehicle driving information acquisition unit, a road image information acquisition unit, an environmental information acquisition unit and a tire information acquisition unit, wherein the vehicle driving information acquisition unit includes a nine-axis IMU sensor arranged on the vehicle body and a wheel speed sensor for collecting wheel speed information, and the nine-axis IMU sensor is used to collect vehicle speed, acceleration and vehicle body inclination angle data; The road image information acquisition unit includes a 1280×720 resolution vehicle-mounted camera installed on the vehicle body; The environmental information acquisition unit includes an on-board temperature and humidity sensor, a rainfall sensor, an air pressure sensor, and a FLIR thermal imager for acquiring road surface temperature distribution images; The tire information acquisition unit includes an implanted MEMS sensor for acquiring tire pressure waveform information and a pressure sensor for acquiring tire ground contact pressure distribution information.

5. The system of the automobile tire pressure control method based on multimodal fusion and attention mechanism according to claim 4 is characterized in that: The temperature and humidity sensor is an SHT35 sensor, the rainfall sensor is an integrated optical rain gauge, the air pressure sensor is a BMP-280 sensor, and the resolution of the FLIR thermal imager is 640×512.

Images