Road surface type recognition method and device based on bidirectional timing feature adaptive fusion

Abstract

The application provides a road surface type recognition method and device based on bidirectional time sequence feature adaptive fusion, and relates to the technical field of data processing.The method determines dynamic temperature by acquiring longitudinal acceleration, transverse acceleration and vertical acceleration collected in the process of vehicle driving, and obtains features for representing road surface types; then, time sequence features are constructed based on longitudinal acceleration, transverse acceleration, vertical acceleration and dynamic temperature, and time stamps when longitudinal acceleration, transverse acceleration and vertical acceleration are collected; finally, a neural network model is used to process the time sequence features, and the classification result of the road surface type is output. The road surface type recognition method based on bidirectional time sequence feature adaptive fusion provided by the application improves the accuracy and reliability of road surface type recognition, and realizes accurate recognition and accurate prediction of the road surface type where the automobile tire is located.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of data processing, and particularly relates to a road surface type identification method and device based on bidirectional time sequence feature adaptive fusion. BACKGROUND

[0002] In recent years, with the rapid development of sensor technology, data processing capability and machine learning algorithm, sensor-based road surface type monitoring has gradually become a research hotspot. For example, by installing various sensors on vehicles to collect road surface type data in real time, and then extracting road surface features (such as potholes, slippery, etc.) through image recognition technology, road surface type identification is performed.

[0003] However, the existing road surface type identification method still has some deficiencies. The existing algorithm cannot fully utilize the collected time sequence data when identifying the road surface type, and the model can only be used in specific environments and conditions, which limits the identification accuracy and robustness of the model, and thus leads to inaccurate identification of the road surface type by the model. SUMMARY

[0004] The present application provides a road surface type identification method and device based on bidirectional time sequence feature adaptive fusion, to solve the problem of inaccurate identification of road surface type in the prior art.

[0005] The present application provides a road surface type identification method based on bidirectional time sequence feature adaptive fusion, comprising the following steps:

[0006] Obtaining longitudinal acceleration, lateral acceleration and vertical acceleration collected during vehicle driving;

[0007] Based on the longitudinal acceleration, lateral acceleration and vertical acceleration, determining a dynamic temperature; the dynamic temperature is a feature used to represent the road surface type, which is obtained by analyzing the relationship between the tire internal gas pressure, temperature and tire acceleration using classical mechanics and aerodynamics model;

[0008] Based on the longitudinal acceleration, lateral acceleration, vertical acceleration and dynamic temperature, and the time stamp when the longitudinal acceleration, lateral acceleration and vertical acceleration are collected, a time sequence feature is constructed;

[0009] Inputting the time sequence feature into a trained neural network model, and outputting the road surface type classification result output by the neural network model;

[0010] The neural network model is composed of a bidirectional time sequence feature adaptive fusion network and an LSTM network; the bidirectional time sequence feature adaptive fusion network is used for bidirectional processing of time sequence features to capture information of the correlation between the front and back of the time sequence features, and obtain fusion attention features; and the LSTM network is used for classifying road surface types based on the fusion attention features to obtain a classification result.

[0011] According to the road surface type identification method based on bidirectional time sequence feature adaptive fusion provided by the application, the bidirectional time sequence feature adaptive fusion network comprises a convolution layer, a forward attention module, a reverse attention module and a fusion attention output module.

[0012] The convolution layer is used for generating queries, keys and values from the time sequence features.

[0013] The forward attention module is used for determining forward attention features based on the queries, the keys and the values.

[0014] The reverse attention module is used for determining reverse attention features based on the queries, the keys and the values.

[0015] The fusion attention output module is used for determining fusion attention features based on the forward attention features and the reverse attention features.

[0016] According to the road surface type identification method based on bidirectional time sequence feature adaptive fusion provided by the application, the fusion attention output module comprises a splicing module, a gating signal generation module and a fusion module.

[0017] The splicing module is used for splicing the forward attention features and the reverse attention features to obtain splicing features.

[0018] The gating signal generation module is used for generating gating signals based on the splicing features.

[0019] The fusion module is used for fusing the forward attention features and the reverse attention features based on the gating signals to obtain fusion attention features.

[0020] According to the road surface type identification method based on bidirectional time sequence feature adaptive fusion provided by the application, the loss function of the LSTM network is determined based on a dynamic weight factor.

[0021] The dynamic weight factor is related to the prediction confidence of the neural network model and is used for dynamically adjusting the loss function of the LSTM network.

[0022] According to the present invention, a road surface type identification method based on bidirectional temporal feature adaptive fusion is provided, wherein determining dynamic temperature based on the longitudinal acceleration, the lateral acceleration, and the vertical acceleration includes:

[0023] The total magnitude of acceleration is determined based on the longitudinal acceleration, the lateral acceleration, and the vertical acceleration.

[0024] The dynamic temperature is determined based on the total magnitude of the acceleration.

[0025] According to the road surface type recognition method based on bidirectional temporal feature adaptive fusion provided by the present invention, the step of acquiring longitudinal acceleration, lateral acceleration, and vertical acceleration collected during vehicle travel includes:

[0026] The longitudinal acceleration, lateral acceleration, and vertical acceleration are acquired by acceleration sensors installed on the inner wall of the tire contact surface during vehicle operation.

[0027] The longitudinal acceleration is directed along the direction of the vehicle's movement; the lateral acceleration is perpendicular to the direction of the vehicle's movement; and the vertical acceleration is perpendicular to the ground.

[0028] The present invention also provides a road surface type identification device based on bidirectional temporal feature adaptive fusion, comprising the following modules:

[0029] The acquisition module is used to acquire the longitudinal acceleration, lateral acceleration, and vertical acceleration collected during the vehicle's movement.

[0030] The temperature module is used to determine the dynamic temperature based on the longitudinal acceleration, the lateral acceleration, and the vertical acceleration. The dynamic temperature is obtained by analyzing the relationship between the internal gas pressure and temperature of the tire and the tire acceleration using classical mechanics and aerodynamics models, and is used to characterize the road surface type.

[0031] The time series module is used to construct time series features based on the longitudinal acceleration, the lateral acceleration, the vertical acceleration, and the dynamic temperature, as well as the timestamps when the longitudinal acceleration, lateral acceleration, and vertical acceleration are collected;

[0032] The classification module is used to input the time series features into the trained neural network model and output the road surface type classification result output by the neural network model;

[0033] The neural network model is composed of a bidirectional time sequence feature adaptive fusion network and an LSTM network; the bidirectional time sequence feature adaptive fusion network is used for bidirectional processing of time sequence features to capture information of the correlation between before and after in the time sequence features, and obtain fusion attention features; and the LSTM network is used for classifying road surface types based on the fusion attention features to obtain a classification result.

[0034] The application further provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, and the processor implements the road surface type identification method based on bidirectional time sequence feature adaptive fusion according to any one of the above when executing the computer program.

[0035] The application further provides a non-transitory computer-readable storage medium having a computer program stored thereon, and the computer program is executed by a processor to implement the road surface type identification method based on bidirectional time sequence feature adaptive fusion according to any one of the above.

[0036] The application further provides a computer program product, including a computer program, and the computer program is executed by a processor to implement the road surface type identification method based on bidirectional time sequence feature adaptive fusion according to any one of the above.

[0037] The application provides a road surface type identification method based on bidirectional time sequence feature adaptive fusion, which acquires longitudinal acceleration, transverse acceleration and vertical acceleration collected in the process of vehicle driving, and determines dynamic temperature by using a classical mechanics and aerodynamics model to obtain features for representing road surface types; then, time sequence features are constructed based on longitudinal acceleration, transverse acceleration, vertical acceleration and dynamic temperature, and time stamps when longitudinal acceleration, transverse acceleration and vertical acceleration are collected; then, a neural network model is constructed by using a bidirectional time sequence feature adaptive fusion network and an LSTM network, and the neural network model is used to process time sequence features to fully capture information of the correlation between before and after in the time sequence features, so that the neural network model outputs a classification result of road surface types on the basis of comprehensively understanding dynamic changes of road surface types, and the accuracy and reliability of road surface type identification are improved, and accurate identification and accurate prediction of road surface types for automobile tires are realized. BRIEF DESCRIPTION OF DRAWINGS

[0038] In order to more clearly illustrate the technical solutions of the present application or the prior art, the following will briefly introduce the drawings needed to be used in the embodiments or prior art description. Obviously, the drawings in the following description are some embodiments of the present application, and for those skilled in the art, other drawings can also be obtained without creative labor on the basis of these drawings.

[0039] Figure 1 is a flowchart of a road surface type recognition method based on bidirectional time sequence feature adaptive fusion provided by the present application.

[0040] Figure 2 is a car tire running schematic diagram in a vehicle driving process in the road surface type recognition method based on bidirectional time sequence feature adaptive fusion provided by the present application.

[0041] Figure 3 is a multi-feature chi-square distribution, F distribution and mutual information test result schematic diagram in the road surface type recognition method based on bidirectional time sequence feature adaptive fusion provided by the present application.

[0042] Figure 4 is a raw data preprocessing flowchart in the road surface type recognition method based on bidirectional time sequence feature adaptive fusion provided by the present application.

[0043] Figure 5 is a framework schematic diagram of a fusion attention output module combined with an LSTM network in the bidirectional time sequence feature adaptive fusion network provided by the present application.

[0044] Figure 6 is a single feature longitudinal acceleration provided by the present application and a comparison diagram of category prediction and data fitting results of combined features.

[0045] Figure 7 is a single feature lateral acceleration provided by the present application and a comparison diagram of category prediction and data fitting results of combined features.

[0046] Figure 8 is a single feature vertical acceleration provided by the present application and a comparison diagram of category prediction and data fitting results of combined features.

[0047] Figure 9 is a comparison diagram of model training loss results using cross-entropy loss and Focal loss provided by the present application.

[0048] Figure 10 is a confusion matrix schematic diagram of classification results provided by the present application.

[0049] Figure 11 is a structure schematic diagram of a road surface type recognition device based on bidirectional time sequence feature adaptive fusion provided by the present application.

[0050] Figure 12 is a structure schematic diagram of an electronic device provided by the present application. DETAILED DESCRIPTION

[0051] In recent years, with the rapid development of sensor technology, data processing capability and machine learning algorithms, sensor-based road surface type monitoring has gradually become a research hotspot. For example, by installing various sensors (such as acceleration sensors, gyroscopes and cameras, etc.) on vehicles, real-time road surface type data can be collected. These data not only include dynamic information such as vehicle speed, acceleration, steering angle, etc., but also can extract road surface features (such as potholes, slippery, etc.) through image recognition technology.

[0052] However, although these emerging technologies bring rich road surface state information, existing road surface type identification methods still have the following shortcomings:

[0053] (1) Single feature dependence: Many existing algorithms only rely on a single type of feature (such as acceleration, speed, etc.), failing to fully utilize the complementarity of multiple sensor data. In this way, the identification accuracy and robustness of the model are limited, especially in complex road conditions, where a single feature often cannot fully reflect the overall picture of the road surface type.

[0054] (2) Insufficient processing of time series information: Road surface types have obvious time series characteristics, and traditional methods have difficulty in capturing important dynamic changes in time series when processing time series data. For example, the driving characteristics of a vehicle on different road surfaces may change over time, and traditional models cannot adapt to such changes in a timely manner.

[0055] (3) Lack of adaptive ability: The feature performance under different road surface types varies greatly, and existing algorithms lack adaptive fusion capability, making it difficult to maintain good identification performance in different environments and conditions. This lack of flexibility results in insufficient adaptability of the model under certain conditions, reducing the reliability and accuracy of the model in actual road surface type identification applications.

[0056] To address the above shortcomings, the present application proposes a road surface type identification method based on bidirectional time series feature adaptive fusion, aiming to fully exploit the potential information in the data by adaptively fusing multiple time series features, thereby improving the accuracy and reliability of road surface type identification and creating conditions for the safety and efficiency of intelligent transportation.

[0057] To make the purpose, technical scheme and advantages of the present application clearer, the technical scheme in the present application will be described clearly and completely below in combination with the drawings in the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative labor are within the scope of protection of the present application.

[0058] The following will be described in combination with Figures 1 to 12The application discloses a road surface type recognition method and device based on bidirectional time sequence feature adaptive fusion.

[0059] The application provides a road surface type recognition method based on bidirectional time sequence feature adaptive fusion, which aims to improve road safety and driving comfort through real-time monitoring and analysis of road surface conditions. By adaptively fusing time sequence feature data obtained from multiple sensors, advanced machine learning and signal processing techniques are used to generate a data format containing multiple features, enabling accurate recognition of road surface types. This method can be applied to autonomous driving, intelligent vehicles, and road maintenance scenarios.

[0060] Figure 1 is a flowchart of the road surface type recognition method based on bidirectional time sequence feature adaptive fusion provided by the application, as Figure 1 shown, the method comprises the following steps:

[0061] Step 101, acquiring longitudinal acceleration, lateral acceleration and vertical acceleration collected during vehicle driving.

[0062] The embodiment of the application acquires longitudinal acceleration, lateral acceleration and vertical acceleration during vehicle driving through data acquisition sensors.

[0063] The data acquisition sensor (hereinafter referred to as sensor) includes acceleration sensors, pressure sensors, temperature sensors and other sensors, which can collect acceleration, pressure and temperature parameters during vehicle driving.

[0064] Figure 2 is a schematic diagram of the vehicle driving process in the road surface type recognition method based on bidirectional time sequence feature adaptive fusion provided by the application, as Figure 2 shown, the sensor is connected to the inner wall of the automobile tire, and the three-axis direction during the rotation of the sensor is the acceleration direction, i.e. during vehicle driving, the longitudinal acceleration is , the lateral acceleration is , and the vertical acceleration is .

[0065] Optionally, the longitudinal acceleration, lateral acceleration and vertical acceleration collected during vehicle driving include:

[0066] Acquiring longitudinal acceleration, lateral acceleration and vertical acceleration collected during vehicle driving through acceleration sensors arranged on the inner wall of the tire ground surface;

[0067] The direction of the longitudinal acceleration is along the direction of the automobile advancing; the direction of the lateral acceleration is perpendicular to the direction of the automobile advancing; and the direction of the vertical acceleration is perpendicular to the ground.

[0068] Specifically, longitudinal acceleration, lateral acceleration and vertical acceleration collected by sensors arranged on the inner wall of the tire ground contact surface during vehicle driving are acquired (see Figure 2 ) for further reference.

[0069] The longitudinal acceleration is in the direction of the vehicle advancing, the lateral acceleration is in the direction perpendicular to the vehicle advancing direction, and the vertical acceleration is in the direction perpendicular to the ground.

[0070] Step 102, determining a dynamic temperature based on the longitudinal acceleration, the lateral acceleration and the vertical acceleration.

[0071] The dynamic temperature is a feature for characterizing the road surface type obtained by analyzing the relationship between the tire internal gas pressure, temperature and tire acceleration by using classical mechanics and aerodynamics model.

[0072] The feature for characterizing the road surface type is obtained by analyzing the relationship between the tire internal gas pressure, temperature and tire acceleration by using classical mechanics and aerodynamics model. The embodiment of the present application takes a certain tire (see Figure 2 ) as an example for illustration.

[0073] In the embodiment of the present application, the sensor is used as the data acquisition source, and the data set detailed composition is designed. The sensor is connected to the inner wall of the vehicle tire, the three-axis directions during the sensor rotation are taken as the acceleration directions, and the data is collected through the signal receiving chip in the sensor.

[0074] Table 1 is the data set division composition table provided by the embodiment of the present application. As shown in Table 1, the data of the tire driving in different road conditions (in other cases, other road conditions can also be collected according to actual needs) is collected by using the sensor in the embodiment of the present application, and the different road conditions include cement road, cobblestone road, Belgium road, rough asphalt, cement undulating road, smooth asphalt and epoxy floor. The collected data is divided into training set, verification set and test set according to the proportion of 0.7:0.15:0.15.

[0075] Table 1 Data set division composition table

[0076]

[0077] ​​​Table 2 is an initial setting table of data acquisition parameters provided by the embodiment of the application. As shown in Table 2, the sensor parameters and environment related parameters in the data acquisition process are exemplarily set. The acceleration data sampling rate is 5000 Hz, and the sampling interval for temperature, pressure and other short-time constant data is 30 seconds.

[0078] Table 2 Initial setting table of data acquisition parameters

[0079]

[0080] In combination with Table 2, the tire internal gas pressure P, the gas volume V, the gas mole number n, the gas constant R = 287 J / (kg·K), the absolute temperature T, the tire friction F, the effective mass M of the automobile, the energy released by the friction , the heat generated by the tire friction , other energy conversion and loss , the mass m of the automobile tire and the automobile engine voltage .

[0081] Optionally, the dynamic temperature is determined based on the longitudinal acceleration, the lateral acceleration and the vertical acceleration, comprising:

[0082] determining the total amplitude of acceleration based on the longitudinal acceleration, the lateral acceleration and the vertical acceleration;

[0083] determining the dynamic temperature based on the total amplitude of acceleration.

[0084] Specifically, the total amplitude of acceleration is first determined based on the longitudinal acceleration , the lateral acceleration and the vertical acceleration , and then the dynamic temperature is determined based on the total amplitude of acceleration.

[0085] Based on the above embodiment, according to the gas state equation, the expressions of the pressure change and the temperature change at a certain time t are as follows:

[0086]

[0087] It can be seen that, under the condition of fixed volume (the tire volume changes little), the pressure change of the gas is proportional to the temperature change , that is:

[0088]

[0089] Therefore, when the temperature increases, the pressure will also increase accordingly. Combining Newton's second law, when the car tire rotates, the friction between the tire and the ground will generate heat, causing the tire temperature to rise. The tire friction force F generated by acceleration , the effective mass M is:

[0090]

[0091] In the process of tire rotation, the energy released by the friction force Can be expressed as:

[0092]

[0093] Where d is the distance of tire rotation. Combining the first law of thermodynamics, the heat generated will cause a change in temperature, and the relationship between the heat generated And the temperature change Can be approximated as:

[0094]

[0095] Where, Is the specific heat capacity. According to the law of conservation of energy, energy is always transformed and lost in various forms during transmission, so the relationship given here is not applicable under all conditions, but is used as a basis for feature selection in embodiments of the present invention. Under other conditions, different relationships can be derived as the basis for feature selection. The above relationship can be arranged to obtain the expression for the change in tire internal gas pressure :

[0096]

[0097] When the gas volume V and the mass m of the car tire are constant, the relationship between the change in pressure And the change in acceleration Is:

[0098]

[0099] According to the longitudinal acceleration (along the direction of the car's forward progress), the lateral acceleration (perpendicular to the direction of the car's forward progress, usually used to represent the acceleration when turning), and the vertical acceleration (perpendicular to the ground, usually related to the force acting on the tire when it is in contact with the ground), the overall acceleration Can be represented as a three-dimensional vector:

[0100]

[0101] The overall acceleration is calculated using the modulus of the vector The expression is:

[0102]

[0103] If the definition The expression of the change of the tire internal gas pressure can be approximated as:

[0104]

[0105] Based on the above longitudinal acceleration , lateral acceleration and vertical acceleration , the total amplitude of acceleration is determined as:

[0106]

[0107] Based on the total amplitude of acceleration , the new characteristic dynamic pressure is determined as:

[0108]

[0109] Based on the total amplitude of acceleration , the new characteristic dynamic temperature is determined as:

[0110]

[0111] Figure 3 is a multi-feature adoption chi-square distribution, F distribution and mutual information test result schematic diagram in a road surface type recognition method based on bidirectional timing feature adaptive fusion provided by the application, as shown in Figure 3 The correlation analysis of the above features using chi-square distribution, F distribution and mutual information can be seen from Figure 3 that the correlation of single features such as x , y , z， , i.e. single three-axis acceleration , , and label is not high, while the correlation of absolute temperature Temperature, centripetal acceleration Z_acc, tire internal gas pressure Pressure and automobile engine voltage Voltage is higher, which is due to the stability of these values in a long time, so the above single features cannot be used as feature indicators for road condition classification, and considering the influence of the existence of certain errors in the collected data due to the reality, for example, when collecting feature data through sensors, equipment or user operation, it will be affected by many factors, mainly including:

[0112] (1) Sensor device itself error, subject to environmental interference (such as temperature change, humidity, vibration, etc.) and improper human operation, resulting in the accuracy and reliability of the feature data decline;

[0113] (2) Missing values or incomplete data records, affecting the training and prediction accuracy of the model;

[0114] (3) Traffic conditions, weather conditions and external factors such as road maintenance affect the driving state of the vehicle and the road type, thereby affecting the stability and correlation of the feature data.

[0115] Therefore, in order to make the data contain multi-angle and multi-dimensional features, so as to more accurately describe the physical performance of the automobile tire under different operating conditions, and then improve the accuracy of the subsequent neural network model in identifying the road type, the embodiment of the present application constructs a combination feature in the form of [t, x , y , z , Accel_Temp_Product] as the input of the subsequent neural network model. The embodiment of the present application collects data of the tire driving in different road conditions through the sensor, and obtains the characteristics of the changes of various parameters through physical derivation analysis with the sensor parameters, and then performs feature analysis selection and feature combination, so that the data features contain multi-angle and multi-dimensional features, so as to more comprehensively mine data information and fully reflect the performance of the automobile tire under different road conditions.

[0116] In addition, the embodiment of the present application also deduces a dynamic model of the automobile tire in the ideal state during the movement process based on classical mechanics and aerodynamics. By considering the ideal gas equation and Newton's law inside the tire, the change relationship between the gas pressure, temperature and tire acceleration inside the automobile tire is established, which provides a new feature dynamic temperature with more abundant and strong representation ability for the subsequent neural network. The new feature is combined with the existing features, so as to more accurately describe the physical performance of the automobile tire under different operating conditions, and provide a basis for the subsequent accurate prediction of the road condition of the automobile tire.

[0117] Step 103, based on the longitudinal acceleration, the lateral acceleration, the vertical acceleration and the dynamic temperature, and the time stamp when the longitudinal acceleration, the lateral acceleration and the vertical acceleration are collected, a time series feature is constructed.

[0118] Based on longitudinal acceleration , lateral acceleration and vertical acceleration and dynamic temperature , and the longitudinal acceleration , lateral acceleration and vertical acceleration timestamp when collecting longitudinal acceleration

[0119] Figure 4 is a raw data preprocessing flowchart in a road surface type recognition method based on bidirectional time sequence feature adaptive fusion provided by the present application, as shown in Figure 4 The process of constructing time sequence features mainly includes:

[0120] Obtain raw data, that is, collect data in real time through sensors when the tire is running on different road conditions, including longitudinal acceleration , lateral acceleration and vertical acceleration of the tire during vehicle driving, and calculate the total amplitude of acceleration, and then calculate the dynamic temperature based on the total amplitude of acceleration;

[0121] Data cleaning, removing outliers and missing value filling of the above real-time collected data to ensure data quality;

[0122] Data standardization, that is, normalization processing, to ensure data consistency;

[0123] Timestamp processing, collecting the timestamp when collecting longitudinal acceleration , lateral acceleration and vertical acceleration ;

[0124] Non-numerical data processing, converting a series of category data into unique integer encoding. Specifically, it converts categorical data (such as strings) in DataFrame into an integer array, while returning a category label array. For the first return value, the integer encoding will correspond to each different category in the original data;

[0125] Create a sliding window to create an input (feature) and output (target) dataset for time series prediction. By giving a time step, the past time step data points are taken as features to predict the target value of the next data point;

[0126] Generate time sequence, based on longitudinal acceleration , lateral acceleration and vertical acceleration and dynamic temperature, and the timestamp when collecting longitudinal acceleration , lateral acceleration and vertical acceleration , construct time sequence features;

[0127] Data preparation, after data preparation is completed, generate target dataset.

[0128] The signal transmitted by the automobile tire in operation has strong time sequence characteristics, but the existing method often only considers the sequence correlation in the positive direction of the time axis, and it is difficult to fully and accurately express the forward and backward correlation of the time sequence, thereby affecting the subsequent accurate modeling data fitting and sample prediction.

[0129] In the embodiment of the application, the time sequence characteristics are constructed based on longitudinal acceleration , lateral acceleration and vertical acceleration and dynamic temperature, and the time stamps of longitudinal acceleration , lateral acceleration and vertical acceleration , so that the past and future context information can be considered at each time step, thereby improving the credibility of the data source and providing a basis for subsequent accurate modeling data fitting and sample prediction.

[0130] Step 104, inputting the time sequence characteristics into the trained neural network model, and outputting the road surface type classification result output by the neural network model.

[0131] The neural network model is composed of a bidirectional time sequence feature adaptive fusion network and an LSTM network; the bidirectional time sequence feature adaptive fusion network is used for bidirectional processing of the time sequence characteristics to capture the information of the forward and backward correlation in the time sequence characteristics, and obtain the fusion attention feature; the LSTM network is used for classifying the road surface type based on the fusion attention feature, and obtain the classification result.

[0132] First, the bidirectional time sequence feature adaptive fusion network and the LSTM network are used to construct the neural network model in the embodiment of the application; then, the data in Table 1 is used, and the training set, the validation set and the test set are divided according to the ratio of 0.7:0.15:0.15, and the neural network model is supervised trained; finally, the time sequence characteristics are input into the trained neural network model, and the road surface type classification result is output.

[0133] Optionally, the bidirectional time sequence feature adaptive fusion network comprises a convolution layer, a forward attention module, a reverse attention module and a fusion attention output module.

[0134] The convolution layer is used for generating queries, keys and values from the time sequence characteristics.

[0135] The forward attention module is used for determining the forward attention feature based on the query, the key and the value.

[0136] The reverse attention module is used for determining the reverse attention feature based on the query, the key and the value.

[0137] The fusion attention output module is configured to determine a fusion attention feature based on the forward attention feature and the reverse attention feature.

[0138] Specifically, the bidirectional time series feature adaptive fusion network in the neural network model includes a convolutional layer, a forward attention module, a reverse attention module, and a fusion attention output module.

[0139] The bidirectional time series feature adaptive fusion network is configured to process time series features bidirectionally, and the core mechanism is a bidirectional attention mechanism (Cross Attention Block) configured to capture information about the correlation between the features before and after the time series, and obtain a fusion attention feature. This process mainly includes the following steps:

[0140] Feature extraction: different convolutional layers are used to generate queries (query), keys (key), and values (value) respectively to ensure the diversity and richness of the features;

[0141] Forward attention calculation: the forward attention from the query to the key is calculated by the forward attention module, i.e., the query and the key are multiplied by the forward attention module to obtain the attention weight, determine the forward attention feature, and normalize it by the Softmax function to obtain the attention degree of different features;

[0142] Reverse attention calculation: the reverse attention from the key to the query is calculated by the reverse attention module, i.e., the key and the query are multiplied by the reverse attention module to obtain the attention weight, determine the reverse attention feature, and obtain the attention degree of different features, thereby further mining the relationship between different features;

[0143] This bidirectional attention mechanism enables the neural network model to fully capture time series features from different directions, more comprehensively understand the dynamic changes of the road surface type, and thus improve the accuracy and robustness of the neural network model recognition;

[0144] Adaptive fusion: through the fusion attention output module, the importance of different features (including forward attention features and reverse attention features) is flexibly adjusted, so that the neural network model can adaptively select the optimal features for fusion, thereby improving the classification performance of the neural network model. At the same time, the adaptive characteristics of the neural network model enable it to effectively handle noise and interference in complex environments, thereby further improving the recognition accuracy.

[0145] The embodiment of the application constructs a neural network model through a bidirectional time sequence feature adaptive fusion network and an LSTM network, and introduces a bidirectional attention mechanism in the bidirectional time sequence feature adaptive fusion network, so that information of the correlation between the front and back in the time sequence feature is fully captured, the neural network model can more comprehensively understand the dynamic change of the road surface type, and the fusion attention feature is obtained, then the LSTM network classifies the road surface type based on the fusion attention feature, and the road surface type classification result is obtained, the accuracy and robustness of the neural network model identification are improved, and accurate identification of the road surface type is realized.

[0146] The application provides a road surface type identification method based on bidirectional time sequence feature adaptive fusion, longitudinal acceleration, transverse acceleration and vertical acceleration collected in the vehicle driving process are acquired, dynamic temperature is determined by using a classical mechanics and aerodynamics model, and thus the feature for representing the road surface type is obtained, then time sequence features are constructed based on the longitudinal acceleration, the transverse acceleration, the vertical acceleration and the dynamic temperature, and the time stamp when the longitudinal acceleration, the transverse acceleration and the vertical acceleration are collected, a neural network model is constructed through a bidirectional time sequence feature adaptive fusion network and an LSTM network, and the neural network model is used to process the time sequence features, information of the correlation between the front and back in the time sequence features is fully captured, the neural network model can comprehensively understand the dynamic change of the road surface type, and the classification result of the road surface type is output, the accuracy and reliability of the road surface type identification are improved, and accurate prediction of the road surface type where the automobile tire is located is realized.

[0147] Optionally, the fusion attention output module comprises a splicing module, a gate signal generation module and a fusion module.

[0148] The splicing module is used for splicing the forward attention feature and the reverse attention feature to obtain a spliced feature.

[0149] The gate signal generation module is used for generating a gate signal based on the spliced feature.

[0150] The fusion module is used for fusing the forward attention feature and the reverse attention feature based on the gate signal to obtain a fusion attention feature.

[0151] Specifically, the fusion attention output module comprises a splicing module, a gate signal generation module and a fusion module, the splicing module is used for splicing the forward attention feature and the reverse attention feature to obtain a spliced feature, the gate signal generation module is used for generating a gate signal based on the spliced feature, and the fusion module is used for fusing the forward attention feature and the reverse attention feature based on the gate signal to obtain a fusion attention feature.

[0152] Figure 5is a framework diagram of a fusion attention output module combined with an LSTM network in a bidirectional time sequence feature adaptive fusion network provided by the present application, as shown in Figure 5

[0153] In the embodiment of the present application, the fusion attention output module in the bidirectional time sequence feature adaptive fusion network is combined with the LSTM network, so that the LSTM network can more effectively capture the context information of the input data through the bidirectional attention mechanism in the fusion attention output module, that is, fully capture the information of the correlation between the front and back in the time sequence features, and dynamically adjust the attention weight of different features. The fusion attention output module includes a splicing module, a gate signal generation module and a fusion module.

[0154] In Figure 5 , if the input time sequence is composed of T time steps of input , the hidden state and the cell state can be initialized as:

[0155]

[0156]

[0157] Based on each time step t, the following time step cycle is performed.

[0158] a. Convolution mapping, generating the query , key and value of the time step corresponding to the time step through 1x1 convolution:

[0159]

[0160] b. Forward attention calculation, calculating the forward attention from the query to the key through the forward attention module, and outputting the forward attention feature out_forward:

[0161]

[0162] c. Reverse attention calculation, calculating the reverse attention from the key to the query through the reverse attention module, and outputting the reverse attention feature out_backward:

[0163]

[0164] d. Splicing output, splicing the forward attention feature out_forward and the reverse attention feature out_backward through the splicing module to obtain the splicing feature : ​

[0165]

[0166] e. Gating signal generation, generating a gating signal through a gating signal generation module and applying a sigmoid function:

[0167]

[0168] f. Fusion attention output, fusing the forward attention feature out_forward and the backward attention feature out_backward based on the gating signal through a fusion module to obtain a fused attention feature :

[0169]

[0170] g. LSTM update, calculating in the LSTM network in combination with the attention output, defining as the input of the input gate, as the sigmoid activation function, as the hidden state of the time step t, , , , as the bias vector, , , , as the corresponding weight matrix, as the cell state of the time step t, as the forget gate, as the output gate, as the candidate cell state, then the complete calculation process of the LSTM is as follows:

[0171]

[0172] The state input of the time step is:

[0173]

[0174] The final output of the LSTM contains attention information optimized through the gating mechanism, making the LSTM more flexible and effective when processing sequence data.

[0175] The embodiment of the application first processes and calculates the data collected by the sensor to obtain time series data, then obtains combined features with stronger representation ability through specific derivation, and inputs the combined features into a network, and then captures information of correlation between the past and the future in the time series through a fusion attention output module in the bidirectional time sequence feature adaptive network, so as to effectively integrate signals from the past and the future, fit the underlying manifold of the input data, feed back dynamic changes of multi-dimensional features to the neural network model, thereby improving the accuracy of the neural network model in understanding the characteristics of the time series, and realizing real-time monitoring and accurate prediction of the road surface type where the automobile tire is located.

[0176] Optionally, the loss function of the LSTM network is determined based on a dynamic weight factor;

[0177] The dynamic weight factor is related to the prediction confidence of the neural network model, and is used for dynamically adjusting the loss function of the LSTM network.

[0178] Specifically, in terms of loss function design, considering the imbalance of the self-made data set, the embodiment of the application adopts an extended focal loss as the loss function, which is defined as:

[0179]

[0180] Among them, is the probability of model prediction, is the weight of each class, is the focal parameter.

[0181] In order to introduce the context information of the time series and improve the accuracy of the neural network model in understanding the characteristics of the time series, the embodiment of the application designs a dynamic weight factor based on the above loss function, which is used for weighting the samples of the time steps, that is, adjusting the Focal Loss of each time according to the correlation of the time series, and the expression of the modified loss function is:

[0182]

[0183] Considering that is introduced to enhance the prediction accuracy of the neural network model at the time step, the dynamic weight factor is defined based on the prediction confidence of the neural network model, and the expression of is:

[0184]

[0185] Among them, is the prediction probability of the model at the time step t, and T is the total number of time steps.

[0186] The embodiment of the application introduces a dynamic weight factor into the calculation process of the LSTM by integrating a bidirectional attention mechanism into the calculation process of the LSTM, and defines and introduces the dynamic weight factor in the loss function , dynamically adjusts the attention weight of different features, so that the neural network model can obtain the context information of the input time sequence at each time step, improves the accuracy of the neural network model in understanding the characteristics of the time sequence, enhances the performance of the model, especially in complex tasks that require understanding of context information and feature dependence, and further enhances the accuracy of the neural network model in identifying road types.

[0187] In order to verify the performance of the neural network model in the embodiment of the application, a plurality of experiments are performed based on the data in Table 1, and the experimental results are shown in Figures 6 to 10 .

[0188] Figure 6 is the single-feature longitudinal acceleration provided by the application , and the comparison chart of the category prediction and data fitting results of the combined features is shown in Figure 6 .

[0189] Figure 7 is the single-feature lateral acceleration provided by the application , and the comparison chart of the category prediction and data fitting results of the combined features is shown in Figure 7 .

[0190] Figure 8 is the single-feature vertical acceleration provided by the application , and the comparison chart of the category prediction and data fitting results of the combined features is shown in Figure 8 .

[0191] In the actual road type classification experiment, the single-feature longitudinal acceleration (as shown in Figure 6 ), the single-feature lateral acceleration (as shown in Figure 7 ) or the single-feature vertical acceleration (as shown in Figure 8 ) is used, and the number of error times generated in the road condition classification prediction is significantly higher than that of the combined features. Therefore, the neural network model provided by the application can effectively improve the discrimination ability of the neural network model and enhance the adaptability of the neural network model to complex road conditions, thereby reducing errors.

[0192] Figure 9 is the comparison chart of the model training loss results using cross-entropy loss and Focal loss provided by the application, as shown in Figure 9As shown, the improved loss function exhibits better adaptability to uneven sample distribution and can fit the original sample distribution more quickly. Compared with the cross-entropy loss function, the improved Focal loss function significantly reduces the loss during training, improves the model's convergence speed, and thus enhances the model's robustness.

[0193] Figure 10 This is a schematic diagram of the confusion matrix of the classification results provided by the present invention, as shown below. Figure 10 As shown, this reflects the model's performance in the classification task. The confusion matrix provides a clear view of the model's classification effectiveness across various categories, the specific instances of misclassification, and the overall accuracy of the model. This verifies the classification performance of the neural network model provided by this invention, demonstrating its ability to achieve accurate classification of road surface types under supervised conditions.

[0194] Based on the above embodiments and experiments, this invention provides a road surface type identification method based on bidirectional temporal feature adaptive fusion. Through in-depth analysis of sensor-collected data and in conjunction with classical mechanics and aerodynamics, a new feature dynamic temperature with strong characterization ability is generated, thereby more accurately describing the physical performance of car tires under different operating conditions. The neural network model provided by this invention can adaptively fuse temporal feature data acquired by sensors, enhancing the comprehensiveness and accuracy of the data. By utilizing the fusion attention output module in the bidirectional temporal feature adaptive network, it captures the correlation information between preceding and following events in the time series, thereby effectively integrating signals from the past and future, improving the accuracy of the neural network model's understanding of time series characteristics, and thus improving prediction accuracy. This enables real-time monitoring and accurate prediction of the road surface type of car tire operation, providing timely information and decision support for drivers.

[0195] The following describes a road surface type recognition device based on adaptive fusion of bidirectional temporal features provided by the present invention. The road surface type recognition device based on adaptive fusion of bidirectional temporal features described below can be referred to in correspondence with the road surface type recognition method based on adaptive fusion of bidirectional temporal features described above.

[0196] Based on any of the above embodiments Figure 11 This is a schematic diagram of a road surface type recognition device based on bidirectional temporal feature adaptive fusion, as presented in this paper. Figure 11 As shown. This embodiment of the invention provides a road surface type identification device based on bidirectional temporal feature adaptive fusion, including an acquisition module 1101, a temperature module 1102, a temporal module 1103, and a classification module 1104, wherein:

[0197] The acquisition module 1101 is configured to acquire longitudinal acceleration, lateral acceleration and vertical acceleration collected during vehicle driving; the temperature module 1102 is configured to determine a dynamic temperature based on the longitudinal acceleration, the lateral acceleration and the vertical acceleration; the dynamic temperature is a feature for representing a road surface type obtained by analyzing the relationship between the tire internal gas pressure, temperature and tire acceleration by using a classical mechanics and aerodynamics model; the timing module 1103 is configured to construct a time sequence feature based on the longitudinal acceleration, the lateral acceleration, the vertical acceleration and the dynamic temperature, and a timestamp when the longitudinal acceleration, the lateral acceleration and the vertical acceleration are collected; the classification module 1104 is configured to input the time sequence feature into a trained neural network model, and output a road surface type classification result output by the neural network model; the neural network model is composed of a bidirectional time sequence feature adaptive fusion network and an LSTM network; the bidirectional time sequence feature adaptive fusion network is configured to perform bidirectional processing on the time sequence feature to capture information of the correlation between the front and rear in the time sequence feature, and obtain a fusion attention feature; the LSTM network is configured to classify the road surface type based on the fusion attention feature, and obtain a classification result.

[0198] Figure 12 An example of a schematic diagram of a physical structure of an electronic device is shown in Figure 12 The electronic device can include a processor 1210, a communications interface 1220, a memory 1230 and a communications bus 1240, wherein the processor 1210, the communications interface 1220 and the memory 1230 complete mutual communication through the communications bus 1240. The processor 1210 can invoke a logical instruction in the memory 1230 to execute a road surface type identification method based on bidirectional time sequence feature adaptive fusion, and the method includes:

[0199] acquiring longitudinal acceleration, lateral acceleration and vertical acceleration collected during vehicle driving;

[0200] determining a dynamic temperature based on the longitudinal acceleration, the lateral acceleration and the vertical acceleration; the dynamic temperature is a feature for representing a road surface type obtained by analyzing the relationship between the tire internal gas pressure, temperature and tire acceleration by using a classical mechanics and aerodynamics model;

[0201] constructing a time sequence feature based on the longitudinal acceleration, the lateral acceleration, the vertical acceleration and the dynamic temperature, and a timestamp when the longitudinal acceleration, the lateral acceleration and the vertical acceleration are collected;

[0202] input the time sequence features into the trained neural network model, and output a road surface type classification result output by the neural network model;

[0203] The neural network model is composed of a bidirectional time sequence feature adaptive fusion network and an LSTM network; the bidirectional time sequence feature adaptive fusion network is used for bidirectional processing of time sequence features to capture information of the correlation between the front and rear of the time sequence features, and obtain fusion attention features; and the LSTM network is used for classifying road surface types based on the fusion attention features to obtain a classification result.

[0204] In addition, the logical instructions in the memory 1230 described above can be implemented in the form of a software function unit and sold or used as an independent product, and can be stored in a computer readable storage medium. Based on such understanding, the technical solutions of the present application essentially or the parts that contribute to the prior art or parts of the technical solutions can be embodied in the form of a software product. The computer software product is stored in a storage medium, and includes a plurality of instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present application. The aforementioned storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM, Read-Only Memory), a random access memory (RAM, Random Access Memory), a magnetic disk or an optical disk, and various media that can store program codes.

[0205] On the other hand, the present application also provides a computer program product, which comprises a computer program, the computer program can be stored on a non-transitory computer readable storage medium, and the computer program can be executed by a processor, so that the computer can execute the road surface type identification method based on bidirectional time sequence feature adaptive fusion provided by the above-mentioned methods, the method comprises:

[0206] obtaining longitudinal acceleration, transverse acceleration and vertical acceleration collected during vehicle driving;

[0207] determining a dynamic temperature based on the longitudinal acceleration, the transverse acceleration and the vertical acceleration; the dynamic temperature is obtained by analyzing the relationship between the tire internal gas pressure, temperature and tire acceleration by using classical mechanics and aerodynamics model, and is a feature for representing the road surface type;

[0208] constructing time sequence features based on the longitudinal acceleration, the transverse acceleration, the vertical acceleration and the dynamic temperature, and a timestamp when the longitudinal acceleration, the transverse acceleration and the vertical acceleration are collected;

[0209] input the time sequence feature into the trained neural network model, and output a road surface type classification result output by the neural network model;

[0210] The neural network model is composed of a bidirectional time sequence feature adaptive fusion network and an LSTM network; the bidirectional time sequence feature adaptive fusion network is used for bidirectional processing of the time sequence feature to capture information of the correlation between the front and the rear in the time sequence feature, and obtain a fusion attention feature; and the LSTM network is used for classifying the road surface type based on the fusion attention feature, and obtain a classification result.

[0211] In another aspect, the application further provides a non-transitory computer readable storage medium having a computer program stored thereon, the computer program being executed by a processor to implement the road surface type identification method based on bidirectional time sequence feature adaptive fusion provided by the above method, and the method comprises:

[0212] obtaining longitudinal acceleration, lateral acceleration and vertical acceleration collected during vehicle driving;

[0213] determining a dynamic temperature based on the longitudinal acceleration, the lateral acceleration and the vertical acceleration; the dynamic temperature is a feature for representing a road surface type obtained by analyzing the relationship between the tire internal gas pressure, temperature and tire acceleration by using a classical mechanics and aerodynamics model;

[0214] constructing a time sequence feature based on the longitudinal acceleration, the lateral acceleration, the vertical acceleration and the dynamic temperature, and a timestamp when the longitudinal acceleration, the lateral acceleration and the vertical acceleration are collected;

[0215] inputting the time sequence feature into the trained neural network model, and outputting a road surface type classification result output by the neural network model;

[0216] The neural network model is composed of a bidirectional time sequence feature adaptive fusion network and an LSTM network; the bidirectional time sequence feature adaptive fusion network is used for bidirectional processing of the time sequence feature to capture information of the correlation between the front and the rear in the time sequence feature, and obtain a fusion attention feature; and the LSTM network is used for classifying the road surface type based on the fusion attention feature, and obtain a classification result.

[0217] The apparatus embodiments described above are only illustrative, and the units described as separate components can or can not be physically separated, and the components shown as units can or can not be physical units, i.e., can be located in one place or distributed on multiple network units. Part or all of the modules can be selected to achieve the purposes of the embodiments according to actual needs. Those skilled in the art can understand and implement without creative labor.

[0218] Through the description of the above embodiments, those skilled in the art can clearly understand that the embodiments can be realized by means of software and necessary universal hardware platforms, and of course can also be realized by hardware. Based on such understanding, the above technical solutions can be embodied in the form of software products, and the computer software products can be stored in a computer readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and include a plurality of instructions to make a computer device (which can be a personal computer, a server, or a network device, etc.) execute the methods described in each embodiment or some parts of the embodiments.

[0219] It should be noted that in this document, the terms "comprising", "including", or any other variant thereof are intended to cover non-exclusive inclusion, so that processes, methods, articles, or devices including a series of elements not only include those elements, but also include other elements not explicitly listed, or further include elements inherent to such processes, methods, articles, or devices. Without more limitations, the element defined by the statement "comprising a" does not exclude the presence of additional identical elements in the process, method, article, or device including the element. In addition, it should be pointed out that the scope of the methods and devices in the embodiments of the present application is not limited to the order of performing the functions as shown or discussed, but can also include performing the functions in a substantially simultaneous manner or in a reverse order, for example, the described method can be performed in an order different from the described order, and various steps can also be added, omitted, or combined. In addition, the features described with reference to some examples can be combined in other examples.

[0220] The "determining B based on A" in the embodiments of the present application means that A is considered as a factor when determining B. It is not limited to "determining B based on A only", but also includes "determining B based on A and C", "determining B based on A, C and E", "determining C based on A, and determining B based on C further", and the like. In addition, it can also include A as a condition for determining B, for example, "when A meets the first condition, determining B using the first method"; for example, "when A meets the second condition, determining B"; for example, "when A meets the third condition, determining B based on the first parameter"; and the like. Of course, A can also be a condition for determining B, for example, "when A meets the first condition, determining C using the first method, and determining B further based on C"; and the like.

[0221] The term "plurality" in the present application refers to two or more, and other quantifiers are similar.

[0222] Finally, it should be noted that: the above embodiments are only used to illustrate the technical solutions of the present application, and not to limit them; although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that: it can still modify the technical solutions recorded in the foregoing embodiments, or make equivalent replacement for part of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A road surface type recognition method based on bidirectional timing feature adaptive fusion, characterized in that, The method comprises: acquiring longitudinal acceleration, lateral acceleration and vertical acceleration collected during vehicle driving; determining a dynamic temperature based on the longitudinal acceleration, the lateral acceleration and the vertical acceleration; the dynamic temperature is obtained by analyzing the relationship between the tire internal gas pressure, temperature and tire acceleration using classical mechanics and aerodynamics model, and is a feature for representing road types; constructing a time sequence feature based on the longitudinal acceleration, the lateral acceleration, the vertical acceleration and the dynamic temperature, and a timestamp when the longitudinal acceleration, the lateral acceleration and the vertical acceleration are collected; inputting the time sequence feature into a trained neural network model, and outputting a road type classification result output by the neural network model; the neural network model is composed of a bidirectional time sequence feature adaptive fusion network and an LSTM network; the bidirectional time sequence feature adaptive fusion network is used for bidirectional processing of the time sequence feature to capture information of the correlation between the front and rear of the time sequence feature, and obtain a fusion attention feature; the LSTM network is used for classifying the road type based on the fusion attention feature to obtain a classification result; the bidirectional time sequence feature adaptive fusion network comprises a convolution layer, a forward attention module, a reverse attention module and a fusion attention output module; the convolution layer is used for generating a query, a key and a value from the time sequence feature; the forward attention module is used for determining a forward attention feature based on the query, the key and the value; the reverse attention module is used for determining a reverse attention feature based on the query, the key and the value; the fusion attention output module is used for determining a fusion attention feature based on the forward attention feature and the reverse attention feature.

2. The road surface type recognition method based on bidirectional timing feature adaptive fusion according to claim 1, characterized in that, the fusion attention output module comprises a splicing module, a gate signal generation module and a fusion module; the splicing module is used for splicing the forward attention feature and the reverse attention feature to obtain a splicing feature; the gate signal generation module is used for generating a gate signal based on the splicing feature; the fusion module is used for fusing the forward attention feature and the reverse attention feature based on the gate signal to obtain a fusion attention feature. 3.The road surface type recognition method based on bidirectional timing feature adaptive fusion according to claim 1, characterized in that, the loss function of the LSTM network is determined based on a dynamic weight factor; the dynamic weight factor is related to the prediction confidence of the neural network model, and is used for dynamically adjusting the loss function of the LSTM network.

4. The road surface type recognition method based on bidirectional timing feature adaptive fusion according to claim 1, characterized in that, The method comprises: determining a total amplitude of acceleration based on the longitudinal acceleration, the lateral acceleration and the vertical acceleration; determining a dynamic temperature based on the total amplitude of acceleration.

5. The road surface type recognition method based on bidirectional timing feature adaptive fusion according to claim 1, characterized in that, The method comprises: acquiring longitudinal acceleration, lateral acceleration and vertical acceleration collected during vehicle driving through an acceleration sensor arranged on the inner wall of the tire ground contact surface; The direction of the longitudinal acceleration is along the direction of the car advancing; the direction of the lateral acceleration is perpendicular to the direction of the car advancing; and the direction of the vertical acceleration is perpendicular to the ground.

6. A road surface type recognition device based on adaptive fusion of bidirectional timing characteristics, characterized by, The method comprises the following steps: An acquisition module is configured to acquire longitudinal acceleration, lateral acceleration, and vertical acceleration collected during vehicle driving; A temperature module is configured to determine a dynamic temperature based on the longitudinal acceleration, the lateral acceleration, and the vertical acceleration; the dynamic temperature is a feature for representing road types obtained by analyzing the relationship between tire internal gas pressure, temperature, and tire acceleration using classical mechanics and aerodynamics models; A time sequence module is configured to construct time sequence features based on the longitudinal acceleration, the lateral acceleration, the vertical acceleration, and the dynamic temperature, and time stamps when the longitudinal acceleration, the lateral acceleration, and the vertical acceleration are collected; A classification module is configured to input the time sequence features into a trained neural network model, and output a road type classification result output by the neural network model; The neural network model is composed of a bidirectional time sequence feature adaptive fusion network and an LSTM network; the bidirectional time sequence feature adaptive fusion network is configured to perform bidirectional processing on the time sequence features to capture information about the correlation between the time sequence features, and obtain fusion attention features; the LSTM network is configured to classify road types based on the fusion attention features, and obtain classification results; The bidirectional time sequence feature adaptive fusion network comprises a convolution layer, a forward attention module, a reverse attention module, and a fusion attention output module; The convolution layer is configured to generate queries, keys, and values from the time sequence features; The forward attention module is configured to determine forward attention features based on the queries, the keys, and the values; The reverse attention module is configured to determine reverse attention features based on the queries, the keys, and the values; The fusion attention output module is configured to determine fusion attention features based on the forward attention features and the reverse attention features.

7. An electronic device comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that, The processor executes the computer program to implement the road type identification method based on bidirectional time sequence feature adaptive fusion as claimed in any one of claims 1 to 5.

8. A non-transitory computer-readable storage medium having stored thereon a computer program, characterized in that, The computer program is executed by the processor to implement the road type identification method based on bidirectional time sequence feature adaptive fusion as claimed in any one of claims 1 to 5.

9. A computer program product comprising a computer program, characterized in that, The computer program is executed by the processor to implement the road type identification method based on bidirectional time sequence feature adaptive fusion as claimed in any one of claims 1 to 5.

Images