An adaptive road surface perception method based on smart tire system transfer learning

Abstract

The application belongs to the technical field of vehicle control, and discloses a self-adaptive road surface perception method based on intelligent tire system migration learning, which comprises the following steps: extracting features from original data; taking the original data and vehicle model labels as source domain data to train a deep adversarial learning network; obtaining original data in actual driving, and extracting features to obtain features as target domain data; jointly putting the features into the deep adversarial learning network and adversarially learning the feature extractor and the domain classifier of the deep adversarial learning network; performing secondary feature extraction on the target domain data to obtain homologous features; inputting the homologous features into the deep adversarial learning network to obtain road surface recognition results; and the application can realize precise road surface perception under a specific tire model in the case that the target domain labeled data is limited, the deep learning algorithm directly migrates the model parameters trained by the source domain labeled data, and is directly compatible with the migration property of the convolutional neural network structure, so that fast and accurate road surface recognition can be realized when the working conditions of the vehicle are different.

Description

Technical Field

[0001] This invention relates to the field of vehicle control technology, and more specifically, to an adaptive road perception method based on transfer learning of an intelligent tire system. Background Technology

[0002] With the increasing intelligence of automobiles, the maturation of advanced driver assistance systems (ADAS) and autonomous driving technologies has spurred the demand for more comprehensive and accurate perception systems. Since the forces acting on the road surface directly affect vehicle movement, the ability of intelligent vehicles to perceive the road surface is of paramount importance. Accurate identification of road surface information such as friction coefficient, terrain features, and unevenness levels; road anomalies such as cracks and potholes; and special terrain features such as wet surfaces, gravel surfaces, and sandy areas helps ADAS to adaptively optimize suspension height, four-wheel power and braking force distribution. This improves vehicle safety and comfort under adverse road conditions and reduces the likelihood of dangerous accidents caused by insufficient driver control. Therefore, intelligent vehicles need to possess the ability to recognize various road surface types and information.

[0003] Utilizing intelligent tire systems for road surface information perception and terrain recognition represents a novel approach to road awareness. Intelligent tire systems typically integrate multiple sensors, often accurately reflecting the road's impact on vehicles. Combined with intelligent algorithms, they can reliably achieve functions such as terrain recognition and road parameter perception. However, due to the need to collect large amounts of road data to train models, intelligent tire systems are often designed for specific bench tests or experimental vehicle models, which often differ from real-world vehicle application conditions. Therefore, when considering practical applications, further thought and improvement are needed on how to enhance the adaptability of intelligent tire road surface perception models to different vehicle models.

[0004] Currently, vehicle manufacturers can categorize their methods for sensing road information while a vehicle is in motion into the following two types.

[0005] (1) Non-contact methods: Using non-contact sensors such as binocular cameras and lidar to perceive road information. Mercedes-Benz's "Magic Carpet" system adopts this method: by installing binocular cameras in front of the vehicle, the vehicle is located and 3D mapped, realizing the identification of road information and features such as road type and road surface anomalies, and achieving high accuracy. However, perception schemes based on non-contact sensors cannot achieve direct and high-precision measurement of the vehicle's motion state and the road surface adhesion state. In addition, non-contact sensor measurement methods are not direct, so they are mostly affected by weather and lighting conditions, and the accuracy and robustness of their perception cannot be guaranteed in adverse weather conditions and lighting conditions.

[0006] (2) Contact-based approach: This approach uses data from vehicle response sensors (such as acceleration sensors and vehicle tilt sensors) combined with a vehicle dynamics model to estimate road surface parameters acting on the vehicle. A typical practical application is the general-purpose MRC active suspension system. The accuracy of prediction using this approach depends on the precision of the vehicle model. Using a simple vehicle model will cause a large amount of uncertainty to accumulate during the Lumana estimation process, while using a complex vehicle model will significantly increase the response time for road surface recognition.

[0007] Tires are the only part of a vehicle that comes into contact with the road surface during driving, bearing the vehicle's power and torque, making their perception of road information crucial. The tire's motion determines the vehicle's driving state, and the area of ​​contact between the tire and the ground most directly reflects the impact of road forces, adhesion, terrain features, and elevation changes on the vehicle. However, the above methods cannot fully capture information about the tire-road contact area, resulting in limited and indirect perception of the interaction and relative motion between the tire and the road. Intelligent tire systems are a new type of intelligent sensing system capable of capturing the "tactile" details of the road surface. By deploying multiple sensors (such as accelerometers and piezoelectric sensors) within the tire to collect road information and applying intelligent algorithms to obtain microscopic features under complex road surface conditions, they provide vehicles with a variety of road information. Due to their direct and significant influence from the road surface, intelligent tire systems are considered an excellent medium for providing road condition information, possessing ideal research potential and development prospects.

[0008] Currently, domestic and international companies and scholars have achieved phased results in research on intelligent tire systems. Foreign tire companies have developed intelligent tire systems that can monitor tire wear in real time by embedding multi-source sensors inside the tire, and achieve preliminary perception of road conditions through tire pressure and acceleration information. The Pirelli Cyber ​​tire system is one of the more advanced and sophisticated intelligent tire systems. The embedded sensors in the Cyber ​​tire collect data such as tire pressure and acceleration during vehicle operation. The tire also has an RFID chip that connects to software in the vehicle's computer. When potential hazards are identified or predicted (such as loss of traction or hydroplaning), the system commands the vehicle's electronic systems to intervene promptly, ensuring driving safety. The Cyber ​​tire system provides the vehicle and driver with a wealth of information, including tire type, recommended tire pressure, load index, speed rating, and in-vehicle information such as temperature and tire pressure. Because monitoring tire pressure and wear information is crucial for a vehicle's speed, driving performance, and safety under extreme tire conditions, this system is primarily used in sports cars and racing cars to improve drivers' and vehicles' understanding of road and tire information under extreme conditions, thereby enhancing racing car performance and safety. It has already been successfully applied to the McLaren Artura sports car.

[0009] Existing smart tire solutions focus primarily on monitoring relatively traditional tire information such as tire pressure and wear, with limited application of embedded sensors or sensor arrays for road perception. Therefore, most current smart tires can only identify slip ratios based on wheel speed and tire pressure information, roughly assess road conditions (such as hydroplaning), and make timely adaptive adjustments, but cannot achieve precise abnormal road surface detection and terrain recognition.

[0010] In addition, there are structural differences between vehicles of different brands and even between different models of the same brand. These structural differences are reflected in the tire's stress, ultimately causing the piezoelectric, acceleration, and pressure signals from the embedded sensors in the intelligent tire system to carry subtle features related to the vehicle's own parameters under the same road conditions and speeds. Furthermore, these features are not entirely affected by road surface excitation. Therefore, intelligent tire road surface recognition algorithms with low adaptability are prone to producing different recognition results when applied to different vehicles. Training intelligent tire road surface recognition models requires a large amount of data. Conducting experiments or simulations to collect large training sets for each vehicle model is extremely labor-intensive and impractical. Moreover, in real-world applications, there are issues such as a lack of sufficient labeled fault samples and variations in the distribution of test samples and training samples due to changing operating conditions. These limitations restrict the application of intelligent tire-based road surface recognition algorithms in practical engineering. Most existing intelligent tire solutions are only applied to certain specific vehicle models: for example, the Pirelli Cyber ​​tire system is only adapted for the McLaren Artura, and Michelin's Pilot Sport series intelligent tire system is only applicable to a small number of specific sports models from brands such as Porsche and BMW.

[0011] To address this issue, an adaptive road perception method based on transfer learning of an intelligent tire system is proposed. Summary of the Invention

[0012] The present invention aims to provide an adaptive road perception method based on transfer learning of intelligent tire systems to solve or improve at least one of the above-mentioned technical problems.

[0013] In view of this, a first aspect of the present invention is to provide an adaptive road perception method based on transfer learning of an intelligent tire system.

[0014] The first aspect of the present invention provides an adaptive road perception method based on transfer learning of an intelligent tire system. The intelligent tire system includes a tire pressure sensor, an acceleration sensor, and multiple piezoelectric sensors disposed within the tire cavity. The adaptive road perception method is implemented through the intelligent tire system and includes the following steps: Preprocessing: acquiring first raw data of an experimental vehicle during experimental driving through the intelligent tire system; extracting features from the first raw data to obtain first multi-source sensor features; using the first multi-source sensor features and the vehicle model label of the experimental vehicle as source domain data, and combining them with multi-source sensor data of unlabeled unknown vehicle models to form a hybrid feature space; and using the hybrid feature space to train a deep adversarial learning network. Training is performed to enable the system to identify and classify road surfaces under experimental vehicles and tire models. Practical calculations are then conducted: Secondary raw data of a specific vehicle model during actual driving is acquired through the tire intelligent system. Feature extraction is performed on this secondary raw data to obtain features from a second multi-source sensor. The preprocessed source domain data and the secondary raw data are fed into a deep adversarial learning network, and the feature extractor and domain classifier of the deep adversarial learning network undergo adversarial learning. The feature extractor after adversarial learning performs secondary feature extraction on the second multi-source sensor features to obtain homologous features that can be used for classification by a convolutional neural network. These homologous features are then input into the preprocessed deep adversarial learning network to obtain road surface recognition results.

[0015] This invention provides an adaptive road perception method based on transfer learning for an intelligent tire system, designing an algorithm to improve the adaptive capability of the road recognition model. Since collecting large amounts of training data for all vehicle models is impractical, a road recognition algorithm based on transfer learning is considered. Transfer learning enables accurate road perception for specific tire models by fine-tuning the trained model, even with limited labeled data in the target domain. A deep learning algorithm based on deep transfer learning is chosen. This algorithm directly transfers model parameters trained using labeled data from the source domain, aligning with the direct transferability property of convolutional neural network structures, enabling fast and accurate road recognition under varying vehicle operating conditions.

[0016] In addition, the technical solutions provided by embodiments of the present invention may also have the following additional technical features:

[0017] In any of the above technical solutions, the feature extraction step includes: filtering and denoising the original data to obtain processed data; and performing wavelet transform feature extraction on the processed data to obtain the multi-source sensor features.

[0018] In any of the above technical solutions, the wavelet change feature extraction adopts the following formula: f(t)=Aj +∑D j Among them, A j For the low-frequency approximation signal, D j is the detail signal in the high-frequency part, and the wavelet transform features include approximate signals and detail signals. A and D are the multi-scale spatial energy distribution features extracted by wavelet transform, f(t) is the data to be decomposed, and j is the number of decomposition levels.

[0019] In any of the above technical solutions, the intelligent tire system is used to acquire various tire data when the vehicle is driving on the road; the tire pressure sensor is used to acquire tire pressure data during the tire-road contact process; the acceleration sensor is used to acquire tire acceleration response data during the tire-road contact process; and the piezoelectric sensor is embedded in the inner surface of the tire to acquire strain during the tire-road contact process.

[0020] In any of the above technical solutions, the intelligent tire system further includes an additional module, which comprises: a data acquisition module for high-frequency sampling of data from a tire pressure sensor, an acceleration sensor, and a piezoelectric sensor to generate real-time one-dimensional tire data; a data transmission module for transmitting the one-dimensional tire data to a data storage unit; a data storage module for storing the one-dimensional tire data; a data processing module for performing data analysis and processing on the one-dimensional tire data; and a power supply module for supplying power to the data acquisition module, data transmission module, data storage module, data processing module, tire pressure sensor, acceleration sensor, and piezoelectric sensor; wherein the sampling frequency range of the acceleration sensor and the piezoelectric sensor is 500Hz-1000Hz.

[0021] In any of the above technical solutions, the tire is a radial tire.

[0022] In any of the above technical solutions, the road surface identification results specifically include: road surface type classification, including: cement road surface, asphalt road surface, gravel road surface and dirt road; road surface feature classification, including: intact, damaged, speed bumps and manhole covers.

[0023] In any of the above technical solutions, adversarial learning of the feature extractor and domain classifier of the deep adversarial learning network includes: using a mixed data space as input to the deep adversarial learning network, and training the domain classifier and feature extractor for adversarial learning; so that the feature extractor can extract homologous data features of the test vehicle and any other vehicle, and resolve the differences between homologous data features; so that the domain classifier can classify the multi-source sensor features after feature extraction, in order to distinguish the vehicle source of the multi-source sensor features.

[0024] The beneficial effects of this invention compared to the prior art are as follows:

[0025] This invention innovatively employs a deployment scheme that integrates a tire pressure sensor and multiple piezoelectric sensors within the tire, establishing an intelligent tire system capable of acquiring, transmitting, and processing tire pressure and strain signals. Furthermore, an accelerometer is installed inside the tire to acquire road vibration information, achieving a biomimetic design based on the principle of tactile perception. The intelligent tire recognition algorithm designed in this invention can capture information such as vibration, pressure, and strain during the tire-road contact process, accurately identifying and classifying the road surface based on the feature-extracted sensor data. Simultaneously, the designed transferable road perception algorithm maintains a fast response speed and a relatively simple structure to ensure the timeliness and robustness of transfer learning.

[0026] This invention designs a road perception model transfer technique based on DaNN. This technique enables the perception algorithm model to transfer between different vehicle speeds, tire models, and vehicle parameters, achieving high-accuracy road recognition and classification under these conditions. The designed DaNN network model adjusts its parameters according to the characteristics of intelligent tire sensor data, enabling accurate and rapid transfer of CNN road recognition networks.

[0027] The intelligent tire system and road surface perception and recognition algorithm designed in this invention have more comprehensive perception functions and exhibit significantly higher response speed. The algorithm model designed in this invention can achieve high-precision special road surface classification and road surface anomaly recognition, and it can still maintain high accuracy, strong robustness, and excellent timeliness in the DaNN transfer learning model. It can also work well with transfer learning algorithms to achieve recognition function transfer in different application scenarios.

[0028] This invention designs a generalizable road surface recognition model based on DANN, achieving high-accuracy road surface classification under different vehicle and tire parameters. The DaNN transfer learning model designed based on intelligent tire sensor data enables the road surface recognition function to be transferred under different vehicle and tire parameters, achieving superior recognition performance and stability in all cases. It enables road surface recognition in various scenarios even with only a single experimental scenario dataset.

[0029] Additional aspects and advantages of embodiments of the invention will become apparent in the following description or may be learned by practice of embodiments of the invention. Attached Figure Description

[0030] The above and / or additional aspects and advantages of the present invention will become apparent and readily understood from the description of the embodiments taken in conjunction with the following drawings, in which:

[0031] Figure 1 This is a schematic diagram of the process of the present invention;

[0032] Figure 2 This is a structural block diagram of the tire intelligent system of the present invention. Detailed Implementation

[0033] To better understand the above-mentioned objectives, features, and advantages of the present invention, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. It should be noted that, unless otherwise specified, the embodiments and features described in these embodiments can be combined with each other.

[0034] Many specific details are set forth in the following description in order to provide a full understanding of the invention. However, the invention may also be practiced in other ways different from those described herein, and therefore the scope of protection of the invention is not limited to the specific embodiments disclosed below.

[0035] Please see Figure 1-2 The following describes an adaptive road perception method based on transfer learning of an intelligent tire system, according to some embodiments of the present invention.

[0036] It consists of hardware components and algorithms.

[0037] An embodiment of the first aspect of the present invention proposes an adaptive road perception method based on transfer learning of an intelligent tire system. In some embodiments of the present invention, such as... Figure 1 As shown, an adaptive road perception method based on transfer learning of an intelligent tire system is provided. This adaptive road perception method based on transfer learning of an intelligent tire system includes:

[0038] In the algorithm section, this invention designs a road surface recognition and classification algorithm using a deep learning neural network method based on the aforementioned intelligent tire system hardware, and designs a transfer learning model based on DaNN (Deep Domain Adaptive Network) to achieve vehicle type adaptability of the road surface recognition model. The specific steps are as follows:

[0039] The training process begins with filtering and denoising the raw data collected by the intelligent tire system. Wavelet transform is then used to extract features from each frequency band of the data's time-frequency information. The binary wavelet decomposition of the signal can be expressed as:

[0040] f(t) = A j +∑D j ;

[0041] Where A j For the low-frequency approximation signal, D j The high-frequency detail signal is represented by j, which is the decomposition level.

[0042] The extracted signal features are labeled and used as source neighborhood labeled data;

[0043] This algorithm builds a neural network using deep learning algorithms and trains a multi-layer convolutional neural network (CNN) on labeled source domain data to achieve road surface recognition based on the CNN in the source domain. The algorithm can classify various special road surfaces and identify various road surface anomalies, stably outputting high-accuracy and reliable prediction results under various complex working conditions. Known vehicle parameters are used as the source domain data, with training labels; unknown vehicle parameters are used as the target domain dataset, without training labels. The source and target domain data are simultaneously input into the DANN network. The DANN's feature extractor and domain classifier perform adversarial learning, outputting a feature extractor that can extract similar features from the source and target domain data. The target domain data after feature extraction can share model data with the CNN network model trained using the source domain data, thus achieving road surface classification based on the target domain data even without target domain labels.

[0044] This invention provides an adaptive road perception method based on transfer learning of an intelligent tire system. The invention innovatively employs a deployment scheme of a tire-embedded tire pressure sensor and multiple piezoelectric sensors to build an intelligent tire system capable of acquiring, transmitting, and processing tire pressure and strain signals. An accelerometer is installed inside the tire to acquire road vibration information, achieving a biomimetic design based on the principle of tactile perception. The intelligent tire recognition algorithm designed in this invention can capture information such as vibration, pressure, and strain during the tire-road contact process, accurately identifying and classifying the road surface based on the feature-extracted sensor data. Simultaneously, the designed transferable road perception algorithm maintains a fast response speed and a relatively simple structure to ensure the timeliness and robustness of transfer learning.

[0045] An embodiment of the first aspect of the present invention proposes an adaptive road perception method based on transfer learning of an intelligent tire system. In some embodiments of the present invention, such as... Figure 2 As shown, an intelligent tire system is provided that implements an adaptive road perception method based on transfer learning for intelligent tire systems. The intelligent tire system includes:

[0046] The hardware component includes an intelligent tire sensing system and additional modules. The intelligent tire sensing system consists of an accelerometer, tire pressure sensor, and piezoelectric sensor embedded in the radial tire. The additional modules include a data acquisition module, a data transmission module, a data storage module, a data processing module, and a power supply module. The tire pressure and accelerometer sensors installed inside the tire collect the tire's response to the current road conditions, such as tire bounce and tire pressure changes. The data acquisition module performs high-frequency sampling of the sensor data, generating real-time one-dimensional tire information. This information is then transmitted to the data storage unit via the data transmission unit, and further transmitted to the data processing unit for data analysis and processing. The power supply module provides power to the modules that require it.

[0047] In summary, this technology enables the perception algorithm model designed in the hardware system to transfer between different vehicle speeds, tire models, and vehicle parameters, achieving high-accuracy road surface recognition and classification under all these conditions. The designed DaNN network model adjusts its parameters based on the characteristics of the smart tire sensor data, enabling accurate and rapid transfer of the CNN road surface recognition network.

[0048] The intelligent tire system and road surface perception and recognition algorithm designed in this invention have more comprehensive perception functions and exhibit significantly higher response speed. The algorithm model designed in this invention can achieve high-precision special road surface classification and road surface anomaly recognition, and it can still maintain high accuracy, strong robustness, and excellent timeliness in the DaNN transfer learning model. It can also work well with transfer learning algorithms to achieve recognition function transfer in different application scenarios.

[0049] This invention designs a generalizable road surface recognition model based on DANN, achieving high-accuracy road surface classification under different vehicle and tire parameters. The DaNN transfer learning model designed based on intelligent tire sensor data enables the road surface recognition function to be transferred under different vehicle and tire parameters, achieving superior recognition performance and stability in all cases. It enables road surface recognition in various scenarios even with only a single experimental scenario dataset.

[0050] In the description of this invention, it should be understood that the terms "longitudinal", "lateral", "up", "down", "front", "rear", "left", "right", "vertical", "horizontal", "top", "bottom", "inner", "outer", etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings, and are only for the convenience of describing this invention, and are not intended to indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation, and therefore should not be construed as a limitation of this invention.

[0051] The embodiments described above are merely preferred embodiments of the present invention and are not intended to limit the scope of the present invention. Various modifications and improvements made by those skilled in the art to the technical solutions of the present invention without departing from the spirit of the present invention should fall within the protection scope defined by the claims of the present invention.

Claims

1. An adaptive road perception method based on transfer learning of an intelligent tire system, characterized in that, The intelligent tire system includes a tire pressure sensor, an acceleration sensor, and multiple piezoelectric sensors disposed within the tire cavity. The adaptive road perception method is implemented through the intelligent tire system and includes the following steps: Preprocessing: The first raw data of the experimental vehicle during the experimental journey is obtained through the tire intelligent system, and feature extraction is performed on the first raw data to obtain the first multi-source sensor features; The first multi-source sensor features and the vehicle model label of the experimental vehicle are used together as source domain data, and together with the multi-source sensor data of the unknown vehicle model without label, a hybrid feature space is formed. The deep adversarial learning network is trained through the hybrid feature space so that it can identify and classify the road surface under the experimental vehicle and tire model. Practical calculation: The tire intelligent system acquires the second raw data of a certain model of vehicle in actual driving, and performs feature extraction on the second raw data to obtain the second multi-source sensor features; The source domain data and the second original data in the preprocessing are put into a deep adversarial learning network, and the feature extractor and domain classifier of the deep adversarial learning network are subjected to adversarial learning. The feature extractor after adversarial learning performs secondary feature extraction on the features of the second multi-source sensor to obtain homologous features that can be used by the convolutional neural network for classification. The homologous features are input into a deep adversarial learning network in the preprocessing stage to obtain road surface recognition results; The piezoelectric sensor is embedded in the inner surface of the tire to obtain the strain during the contact process between the tire and the road surface; The road surface recognition results specifically include: Road surface types can be categorized as follows: cement road surface, asphalt road surface, gravel road surface, and dirt road. Road surface features can be categorized as follows: intact, damaged, speed bumps, and manhole covers.

2. The adaptive road perception method based on transfer learning of an intelligent tire system according to claim 1, characterized in that, The feature extraction steps include: The original data is filtered and denoised to obtain processed data; Wavelet transform feature extraction is performed on the processed data to obtain the features of the multi-source sensor.

3. The adaptive road perception method based on transfer learning of an intelligent tire system according to claim 2, characterized in that, The wavelet transform feature extraction uses the following formula: ； in, For the approximate signal in the low-frequency part, The wavelet transform features include both approximate and detail signals, and A and D represent the multi-scale spatial energy distribution features extracted through wavelet transform. For the data to be decomposed, j Number of decomposition layers 。 4. The adaptive road perception method based on transfer learning of an intelligent tire system according to claim 1, characterized in that, The intelligent tire system is used to acquire various tire data when the vehicle is driving on the road. as well as The tire pressure sensor is used to collect tire pressure data during the tire-road contact process. The acceleration sensor is used to obtain tire acceleration response data during the tire-road contact process.

5. The adaptive road perception method based on transfer learning of an intelligent tire system according to claim 4, characterized in that, The intelligent tire system also includes an additional module, which includes: The data acquisition module is used to perform high-frequency sampling of data from tire pressure sensors, acceleration sensors, and piezoelectric sensors to generate real-time one-dimensional tire data; The data transmission module is used to transmit one-dimensional tire data to the data storage unit; A data storage module is used to store the one-dimensional tire data; The data processing module is used to perform data analysis and processing on the one-dimensional tire data; The power supply module is used to supply power to the data acquisition module, data transmission module, data storage module, data processing module, tire pressure sensor, acceleration sensor, and piezoelectric sensor. The sampling frequency range of the accelerometer and piezoelectric sensor is 500Hz-1000Hz.

6. The adaptive road perception method based on transfer learning of an intelligent tire system according to claim 1, characterized in that, The tires mentioned are radial tires.

7. The adaptive road perception method based on transfer learning of an intelligent tire system according to claim 1, characterized in that, Adversarial learning of the feature extractor and domain classifier of the deep adversarial learning network includes: Using a hybrid data space as input to a deep adversarial learning network, adversarial learning is performed to train the domain classifier and feature extractor. This enables the feature extractor to extract source data features from the test vehicle model and any other vehicle model, and to resolve differences between source data features; This enables the domain classifier to classify the multi-source sensor features after feature extraction, thereby distinguishing the vehicle source of the multi-source sensor features.

Images