Pavement type identification method and device based on bidirectional time sequence feature adaptive fusion

Abstract

The invention provides a road surface type identification method and device based on bidirectional time sequence characteristic adaptive fusion, and relates to the technical field of data processing, and the method comprises the steps: obtaining a longitudinal acceleration, a transverse acceleration and a vertical acceleration collected in a vehicle driving process, determining a dynamic temperature, and obtaining a characteristic used for representing a road surface type; then, based on the longitudinal acceleration, the transverse acceleration, the vertical acceleration and the dynamic temperature, time stamps of the longitudinal acceleration, the transverse acceleration and the vertical acceleration are collected, and time sequence features are constructed; and finally, processing the time sequence features by using a neural network model, and outputting a classification result of pavement types. According to the road surface type identification method based on bidirectional time sequence feature adaptive fusion provided by the invention, the accuracy and reliability of road surface type identification are improved, and accurate identification and accurate prediction of the type of the road surface where the automobile tire is located are realized.

Description

Technical Field

[0001] The present invention relates to the field of data processing technology, and in particular to a road surface type recognition method and device based on bidirectional time series feature adaptive fusion. Background Art

[0002] In recent years, with the rapid development of sensor technology, data processing capabilities and machine learning algorithms, sensor-based road type monitoring has gradually become a research hotspot. For example, by installing various sensors on vehicles to collect road type data in real time, and then using image recognition technology to extract road features (such as potholes, wetness, etc.), road type identification can be performed.

[0003] However, the existing road surface type recognition methods still have some shortcomings. When performing road surface type recognition, the existing algorithms not only fail to make full use of the collected time series data, but also the models can only be used in specific environments and conditions, which limits the recognition accuracy and robustness of the models, and thus leads to inaccurate recognition of road surface types by the models. Summary of the invention

[0004] The present invention provides a road surface type recognition method and device based on bidirectional time series feature adaptive fusion, which are used to solve the problem of inaccurate road surface type recognition in the prior art.

[0005] The present invention provides a road surface type recognition method based on bidirectional temporal feature adaptive fusion, comprising the following steps: Obtain the longitudinal acceleration, lateral acceleration and vertical acceleration collected during vehicle driving; Determining a dynamic temperature based on the longitudinal acceleration, the lateral acceleration and the vertical acceleration; the dynamic temperature is a characteristic used to characterize the road surface type obtained by analyzing the relationship between the internal gas pressure, temperature and tire acceleration of the tire using a classical mechanics and aerodynamics model; constructing a time series feature based on the longitudinal acceleration, the lateral acceleration, the vertical acceleration and the dynamic temperature, and a timestamp when the longitudinal acceleration, the lateral acceleration and the vertical acceleration are collected; Inputting the time series features into a trained neural network model, and outputting a road surface type classification result output by the neural network model; The neural network model consists of a bidirectional time series feature adaptive fusion network and an LSTM network; the bidirectional time series feature adaptive fusion network is used to perform bidirectional processing on time series features to capture the information of the front and back correlation in the time series features to obtain fused attention features; the LSTM network is used to classify the road surface type based on the fused attention features to obtain a classification result.

[0006] According to a road type recognition method based on bidirectional temporal feature adaptive fusion provided by the present invention, the bidirectional temporal feature adaptive fusion network includes a convolution layer, a forward attention module, a reverse attention module and a fusion attention output module; The convolutional layer is used to generate queries, keys and values ​​from the time series features; The forward attention module is used to determine a forward attention feature based on the query, the key and the value; The reverse attention module is used to determine a reverse attention feature based on the query, the key and the value; The fused attention output module is used to determine the fused attention feature based on the forward attention feature and the reverse attention feature.

[0007] According to a road type recognition method based on bidirectional temporal feature adaptive fusion provided by the present invention, the fusion attention output module includes a splicing module, a gated signal generation module and a fusion module; The splicing module is used to splice the forward attention feature and the reverse attention feature to obtain a spliced ​​feature; The gating signal generating module is used to generate a gating signal based on the splicing feature; The fusion module is used to fuse the forward attention feature and the reverse attention feature based on the gating signal to obtain a fused attention feature.

[0008] According to a road type identification method based on bidirectional time series feature adaptive fusion provided by the present invention, the loss function of the LSTM network is determined based on a dynamic weight factor; The dynamic weight factor is related to the prediction confidence of the neural network model and is used to dynamically adjust the loss function of the LSTM network.

[0009] According to a road type identification method based on bidirectional time series feature adaptive fusion provided by the present invention, the dynamic temperature is determined based on the longitudinal acceleration, the lateral acceleration and the vertical acceleration, including: determining a total magnitude of acceleration based on the longitudinal acceleration, the lateral acceleration, and the vertical acceleration; A dynamic temperature is determined based on the total magnitude of the acceleration.

[0010] According to a road type recognition method based on bidirectional time series feature adaptive fusion provided by the present invention, the longitudinal acceleration, lateral acceleration and vertical acceleration collected during vehicle driving are obtained, including: Acquire the longitudinal acceleration, lateral acceleration and vertical acceleration collected by the acceleration sensor arranged on the inner wall of the tire contact surface during the vehicle driving process; The direction of the longitudinal acceleration is along the forward direction of the vehicle; the direction of the lateral acceleration is perpendicular to the forward direction of the vehicle; and the direction of the vertical acceleration is perpendicular to the ground.

[0011] The present invention also provides a road type recognition device based on bidirectional temporal feature adaptive fusion, comprising the following modules: An acquisition module is used to acquire the longitudinal acceleration, lateral acceleration and vertical acceleration collected during the vehicle's driving process; a temperature module, for determining a dynamic temperature based on the longitudinal acceleration, the lateral acceleration and the vertical acceleration; the dynamic temperature is a characteristic used to characterize a road surface type obtained by analyzing the relationship between tire internal gas pressure, temperature and tire acceleration using a classical mechanics and aerodynamics model; A time series module, for constructing a time series feature based on the longitudinal acceleration, the lateral acceleration, the vertical acceleration and the dynamic temperature, and a timestamp when the longitudinal acceleration, the lateral acceleration and the vertical acceleration are collected; A classification module, used for inputting the time series features into a trained neural network model and outputting a road surface type classification result output by the neural network model; The neural network model consists of a bidirectional time series feature adaptive fusion network and an LSTM network; the bidirectional time series feature adaptive fusion network is used to perform bidirectional processing on time series features to capture the information of the front and back correlation in the time series features to obtain fused attention features; the LSTM network is used to classify the road surface type based on the fused attention features to obtain a classification result.

[0012] The present invention also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the method for identifying road type based on adaptive fusion of bidirectional temporal features as described in any one of the above is implemented.

[0013] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the method for identifying road types based on adaptive fusion of bidirectional temporal features as described in any one of the above is implemented.

[0014] The present invention also provides a computer program product, including a computer program, which, when executed by a processor, implements any of the above-mentioned methods for identifying road types based on adaptive fusion of bidirectional temporal features.

[0015] The present invention provides a road type recognition method based on two-way time series feature adaptive fusion. The method obtains longitudinal acceleration, lateral acceleration and vertical acceleration collected during vehicle driving, and uses classical mechanics and aerodynamic models to determine dynamic temperature to obtain features for characterizing road type; then, based on the longitudinal acceleration, lateral acceleration, vertical acceleration and dynamic temperature, and timestamps when the longitudinal acceleration, lateral acceleration and vertical acceleration are collected, time series features are constructed; then, a neural network model is constructed through a two-way time series feature adaptive fusion network and an LSTM network, and the neural network model is used to process the time series features, fully capturing information on the front and back correlations in the time series features, so that the neural network model outputs a classification result of the road type on the basis of a comprehensive understanding of the dynamic changes of the road type, thereby improving the accuracy and reliability of road type recognition, and realizing accurate recognition and accurate prediction of the road type on which the tire of the vehicle is located. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0017] Figure 1 It is a flow chart of a road surface type recognition method based on adaptive fusion of bidirectional temporal features provided by the present invention.

[0018] Figure 2 It is a schematic diagram of the movement of automobile tires during vehicle driving in a road type recognition method based on adaptive fusion of bidirectional temporal features provided by the present invention.

[0019] Figure 3 It is a schematic diagram of the results of using chi-square distribution, F distribution and mutual information test for multiple features in a road type recognition method based on adaptive fusion of bidirectional time series features provided by the present invention.

[0020] Figure 4 It is a flow chart of raw data preprocessing in a road surface type recognition method based on bidirectional temporal feature adaptive fusion provided by the present invention.

[0021] Figure 5 It is a framework schematic diagram of the combination of a fusion attention output module and an LSTM network in a bidirectional temporal feature adaptive fusion network provided by the present invention.

[0022] Figure 6 is the single characteristic longitudinal acceleration provided by the present invention Comparison chart of category prediction and data fitting results using combined features.

[0023] Figure 7 is the single characteristic lateral acceleration provided by the present invention Comparison chart of category prediction and data fitting results using combined features.

[0024] Figure 8 It is the single characteristic vertical acceleration provided by the present invention Comparison chart of category prediction and data fitting results using combined features.

[0025] Fig. 9 This is a comparison chart of the model training loss results using cross entropy loss and focal loss provided by the present invention.

[0026] Fig.10 It is a schematic diagram of the confusion matrix of the classification result provided by the present invention.

[0027] Fig.11 This is a structural schematic diagram of a road type recognition device based on adaptive fusion of bidirectional temporal features provided in this article.

[0028] Fig.12 It is a structural schematic diagram of the electronic device provided by the present invention. DETAILED DESCRIPTION

[0029] In recent years, with the rapid development of sensor technology, data processing capabilities and machine learning algorithms, sensor-based road surface type monitoring has gradually become a research hotspot. For example, by installing various sensors (such as accelerometers, gyroscopes and cameras) on vehicles, road surface type data can be collected in real time. These data not only include dynamic information such as vehicle speed, acceleration, steering angle, etc., but also can extract road surface features (such as potholes, wetness, etc.) through image recognition technology.

[0030] However, although these emerging technologies have brought rich road condition information, the existing road type recognition methods still have shortcomings in the following aspects: (1) Single feature reliance: Many existing algorithms rely on only a single type of feature (such as acceleration, speed, etc.), failing to fully utilize the complementarity of multiple sensor data. As a result, the recognition accuracy and robustness of the model are limited, especially in complex road conditions, where a single feature often cannot fully reflect the full picture of the road type.

[0031] (2) Inadequate processing of time series information: Road surface types have obvious time series characteristics. Traditional methods have difficulty capturing important dynamic changes in time series when processing time series data. For example, the driving characteristics of a vehicle on different road surfaces may change over time, and traditional models cannot adapt to such changes in a timely manner.

[0032] (3) Lack of adaptive ability: There are large differences in the characteristics of different road types, and the existing algorithms lack adaptive fusion capabilities, making it difficult to maintain good recognition performance under different environments and conditions. This lack of flexibility leads to insufficient adaptability of the model under specific conditions, reducing the reliability and accuracy of the model in actual road type recognition applications.

[0033] In view of the above shortcomings, the present invention proposes a road surface type recognition method based on adaptive fusion of bidirectional time series features, which aims to improve the accuracy and reliability of road surface type recognition by adaptively fusing multiple time series features and fully mining the potential information in the data, thereby creating conditions for improving the safety and efficiency of intelligent transportation.

[0034] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the drawings of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0035] Combine the following Figures 1 to 12 The present invention describes a road surface type recognition method and device based on bidirectional time series feature adaptive fusion.

[0036] The present invention provides a road type recognition method based on bidirectional time series feature adaptive fusion, which aims to improve road safety and driving comfort through real-time monitoring and analysis of road conditions. By adaptively fusing time series feature data obtained by multiple sensors, using advanced machine learning and signal processing technology, a data format containing multiple features is generated to achieve accurate recognition of road types, which can be used in application scenarios such as autonomous driving, intelligent vehicles and road maintenance.

[0037] Figure 1 is a flow chart of a road surface type recognition method based on bidirectional temporal feature adaptive fusion provided by the present invention, such as Figure 1 As shown, the method comprises the following steps: Step 101: Obtain the longitudinal acceleration, lateral acceleration and vertical acceleration collected during the vehicle's driving process.

[0038] The embodiment of the present invention acquires the longitudinal acceleration, lateral acceleration and vertical acceleration of the vehicle during its travel by means of a data acquisition sensor.

[0039] Data acquisition sensors (hereinafter referred to as sensors) include acceleration sensors, pressure sensors, temperature sensors and other sensors, which can collect parameters such as acceleration, pressure and temperature during vehicle driving.

[0040] Figure 2 is a schematic diagram of the movement of automobile tires during vehicle driving in a road type recognition method based on two-way time series feature adaptive fusion provided by the present invention, such as Figure 2 As shown, the sensor is connected to the inner wall of the car tire, and the three-axis direction of the sensor during rotation is taken as the acceleration direction, that is, during the vehicle driving process, the longitudinal acceleration is , the lateral acceleration is , the vertical acceleration is .

[0041] Optionally, the acquiring of the longitudinal acceleration, lateral acceleration and vertical acceleration collected during the vehicle's travel includes: Acquire the longitudinal acceleration, lateral acceleration and vertical acceleration collected by the acceleration sensor arranged on the inner wall of the tire contact surface during the vehicle driving process; The direction of the longitudinal acceleration is along the forward direction of the vehicle; the direction of the lateral acceleration is perpendicular to the forward direction of the vehicle; and the direction of the vertical acceleration is perpendicular to the ground.

[0042] Specifically, the sensor disposed on the inner wall of the tire contact surface during the vehicle driving process (see further) Figure 2 )The longitudinal acceleration, lateral acceleration and vertical acceleration collected by the sensor. The sensor has a signal receiving chip for receiving the data collected by the sensor.

[0043] Among them, the longitudinal acceleration The direction is along the direction of the car's forward movement, and the lateral acceleration The direction is perpendicular to the direction of the car's forward movement, and the vertical acceleration The direction is perpendicular to the ground.

[0044] Step 102: Determine a dynamic temperature based on the longitudinal acceleration, the lateral acceleration, and the vertical acceleration.

[0045] The dynamic temperature is a characteristic used to characterize the road surface type obtained by analyzing the relationship between the internal gas pressure, temperature and tire acceleration of the tire using classical mechanics and aerodynamic models.

[0046] Using classical mechanics and aerodynamic models, the relationship between the internal gas pressure, temperature and tire acceleration of the tire is analyzed to obtain the characteristics used to characterize the road surface type. Figure 2 ) as an example to illustrate.

[0047] In the embodiment of the present invention, a sensor is used as a data acquisition source, and a detailed composition of a data set is designed. The sensor is connected to the inner wall of a car tire, and the three-axis directions of the sensor during rotation are respectively taken as acceleration directions, and data is collected through a signal receiving chip in the sensor.

[0048] Table 1 is a data set division composition table provided by an embodiment of the present invention. As shown in Table 1, the embodiment of the present invention uses sensors to collect data of tires driving under different road conditions (in other cases, data under other road conditions can also be collected according to actual needs). The different road conditions include: cement road, cobblestone road, Belgian road, rough asphalt, cement undulating road, smooth asphalt and epoxy floor. The collected data is divided into training set, verification set and test set according to the ratio of 0.7:0.15:0.15.

[0049] Table 1 Dataset division and composition table Table 2 is an initial setting table of data acquisition parameters provided by an embodiment of the present invention. As shown in Table 2, exemplary settings are made for sensor parameters and environment-related parameters during data acquisition. The acceleration data sampling rate is 5000 Hz, and the sampling interval for short-term constant data such as temperature and pressure is 30 seconds.

[0050] Table 2 Data acquisition parameter initial setting table Combined with Table 2, define the tire internal gas pressure P, gas volume V, gas mole number n, gas constant R =287J / (kg·K), absolute temperature T, tire friction F, effective mass of the car M, energy released by friction , Heat generated by tire friction , other energy conversion and loss , the mass m of the car tire and the voltage of the car engine .

[0051] Optionally, determining the dynamic temperature based on the longitudinal acceleration, the lateral acceleration and the vertical acceleration comprises: determining a total magnitude of acceleration based on the longitudinal acceleration, the lateral acceleration, and the vertical acceleration; A dynamic temperature is determined based on the total magnitude of the acceleration.

[0052] Specifically, based on the longitudinal acceleration , lateral acceleration and vertical acceleration , determine the total magnitude of the acceleration, and then determine the dynamic temperature based on the total magnitude of the acceleration.

[0053] Based on the above embodiment, the pressure change under a certain time t can be given according to the gas state equation: and temperature changes The expression is: It can be seen that under the condition of fixed volume (small change in tire volume), the pressure change of gas With temperature changes is proportional to: Therefore, when the temperature increases, the pressure will also increase accordingly. Combined with Newton's second law, when the car tires rotate, the friction between the tires and the ground will generate heat, causing the tire temperature to rise. The tire friction force F generated by acceleration is related to the acceleration , the relationship between the effective mass M is: During the rotation of the tire, the energy released by friction It can be expressed as: Where d is the distance the tire rotates. Combined with the first law of thermodynamics, the heat generated will cause a change in temperature. With temperature changes The relationship can be approximated as: in, is the specific heat capacity. According to the law of conservation of energy, energy will always be converted and lost in various forms during the transmission process. Therefore, the relationship given here is not applicable to all conditions, but is used as a basis for feature selection in the embodiment of the present invention. Under other conditions, different relationship can be derived as the basis for feature selection. The above relationship can be sorted out to obtain the change of gas pressure inside the tire The expression is: When the gas volume V and the mass m of the car tire are constant, the pressure change can be obtained With acceleration The changing relationship is: According to the longitudinal acceleration (in the direction of the car's movement), lateral acceleration (perpendicular to the direction of the car's travel, usually used to indicate acceleration when turning) and vertical acceleration (perpendicular to the ground, usually involving the force of the tire contacting the ground), the overall acceleration Represented as a three-dimensional vector: Calculate the overall acceleration using the magnitude of the vector , the expression is: If defined , the tire internal gas pressure changes The expression can be approximated as: Based on the above longitudinal acceleration , lateral acceleration and vertical acceleration , determine the total magnitude of the acceleration for: Based on the total magnitude of acceleration , determine the new characteristic dynamic pressure for: Based on the total magnitude of acceleration , determine the new characteristic dynamic temperature for: Figure 3 FIG. 1 is a schematic diagram of the results of using chi-square distribution, F distribution and mutual information test for multiple features in a road type recognition method based on adaptive fusion of bidirectional time series features provided by the present invention, such as Figure 3 The chi-square distribution, F distribution and mutual information are used to analyze the correlation of the above features. Figure 3 It can be seen that a single feature such as x , y , z， That is, a single three-axis acceleration , , The correlation with the label is not high, while the absolute temperature Temperature, centripetal acceleration Z_acc, tire internal gas pressure Pressure, and vehicle engine voltage Voltage are highly correlated. This is because these values ​​are stable for a long time. Therefore, none of the above single features can be used as feature indicators for road condition classification. In addition, considering the influence of certain errors in the actual data collection, for example, when collecting feature data through sensors, equipment or user operations, it will be affected by many factors, mainly including: (1) Errors in the sensor equipment itself, environmental interference (such as temperature changes, humidity, vibration, etc.) and improper human operation lead to a decrease in the accuracy and reliability of feature data; (2) Missing values ​​or incomplete data records affect model training and prediction accuracy; (3) External factors such as traffic conditions, weather conditions, and road maintenance have an impact on the vehicle's driving status and road type, thereby affecting the stability and relevance of feature data.

[0054] Therefore, in order to make the data contain multi-angle and multi-dimensional features, so as to more accurately describe the physical performance of automobile tires under different operating conditions, and thus improve the accuracy of the subsequent neural network model in identifying the road type, the embodiment of the present invention uses [t, x , y , z , Accel_Temp_Product] feature form to construct combined features as the input of the subsequent neural network model. The embodiment of the present invention collects data of tires driving under different road conditions through sensors, and obtains the characteristics of changes in various parameters through physical deduction and analysis with sensor parameters, and then performs feature analysis selection and feature combination to make the data features contain multi-angle and multi-dimensional features, thereby more comprehensively mining data information and fully reflecting the performance of automobile tires under different road conditions.

[0055] In addition, the embodiment of the present invention also derives the dynamic model of the automobile tire in the process of movement under ideal conditions based on classical mechanics and aerodynamics. By considering the ideal gas equation inside the tire and Newton's law, the changing relationship between the gas pressure and temperature inside the automobile tire and the tire acceleration is established, providing a new feature dynamic temperature with richer and stronger characterization ability for the subsequent neural network, combining the new feature with the existing feature, thereby more accurately describing the physical performance of the automobile tire under different operating conditions, and providing a basis for the subsequent accurate prediction of the road conditions of the automobile tire.

[0056] Step 103: construct a time series feature based on the longitudinal acceleration, the lateral acceleration, the vertical acceleration and the dynamic temperature, and the timestamps when the longitudinal acceleration, the lateral acceleration and the vertical acceleration are collected.

[0057] Based on longitudinal acceleration , lateral acceleration and vertical acceleration and dynamic temperature , and collect longitudinal acceleration , lateral acceleration and vertical acceleration The timestamp at the time is used to construct time series features.

[0058] Figure 4 is a raw data preprocessing flow chart of a road surface type recognition method based on bidirectional time series feature adaptive fusion provided by the present invention, such as Figure 4 As shown. The process of constructing time series features mainly includes: Obtaining raw data, that is, collecting data about tires driving under different road conditions in real time through sensors, including the longitudinal acceleration of the tires while the car is driving , lateral acceleration and vertical acceleration The total acceleration amplitude is calculated, and then the dynamic temperature is calculated based on the total acceleration amplitude. ; Data cleaning: remove outliers and fill in missing values ​​from the above real-time collected data to ensure data quality; Data standardization, i.e. normalization processing, to ensure data consistency; Timestamp processing, collecting longitudinal acceleration , lateral acceleration and vertical acceleration The timestamp of the time; Non-numeric data processing, converting a series of categorical data into unique integer codes. Specifically, it converts categorical data (such as strings) in the DataFrame into integer arrays and returns an array of category labels. For the first return value, the integer code will correspond to each different category in the original data; Create a sliding window to create an input (feature) and output (target) dataset for time series forecasting. Use the past time step data points as features to predict the target value of the next data point at a given time step. Generate time series based on longitudinal acceleration , lateral acceleration and vertical acceleration and dynamic temperature, as well as collecting longitudinal acceleration , lateral acceleration and vertical acceleration The timestamp of the time is used to construct time series features; Data preparation,After data preparation is completed, the target data set is generated.

[0059] The signals transmitted by automobile tires during operation have strong time series characteristics, but existing methods often only consider the positive serial correlation along the time axis, which makes it difficult to fully and accurately express the forward and backward correlation of the time series, thus affecting the subsequent precise modeling data fitting and sample prediction.

[0060] In the embodiment of the present invention, based on the longitudinal acceleration , lateral acceleration and vertical acceleration and dynamic temperature, as well as collecting longitudinal acceleration , lateral acceleration and vertical acceleration The timestamp at that time is used to construct time series features, which can consider the past and future contextual information at each time step, thereby improving the credibility of the data source and providing a basis for subsequent precise modeling data fitting and sample prediction.

[0061] Step 104: input the time series features into a trained neural network model, and output a road surface type classification result output by the neural network model.

[0062] The neural network model consists of a bidirectional time series feature adaptive fusion network and an LSTM network; the bidirectional time series feature adaptive fusion network is used to perform bidirectional processing on time series features to capture the information of the front and back correlation in the time series features to obtain fused attention features; the LSTM network is used to classify the road surface type based on the fused attention features to obtain a classification result.

[0063] Firstly, a bidirectional time series feature adaptive fusion network and an LSTM network are used to construct the neural network model in the embodiment of the present invention; then, the data in Table 1 is used to divide the data into a training set, a validation set, and a test set in a ratio of 0.7:0.15:0.15, and supervised training is performed on the neural network model; finally, the time series features are input into the trained neural network model, and the road type classification result is output.

[0064] Optionally, the bidirectional temporal feature adaptive fusion network includes a convolutional layer, a forward attention module, a reverse attention module and a fusion attention output module; The convolutional layer is used to generate queries, keys and values ​​from the time series features; The forward attention module is used to determine a forward attention feature based on the query, the key and the value; The reverse attention module is used to determine a reverse attention feature based on the query, the key and the value; The fused attention output module is used to determine the fused attention feature based on the forward attention feature and the reverse attention feature.

[0065] Specifically, the bidirectional temporal feature adaptive fusion network in the neural network model includes a convolutional layer, a forward attention module, a backward attention module and a fusion attention output module.

[0066] The bidirectional time series feature adaptive fusion network is used to process time series features in both directions. The core mechanism is the bidirectional attention mechanism (Cross Attention Block), which is used to capture the information of the previous and next correlations in the time series features and obtain the fused attention features. This process mainly includes the following steps: Feature extraction: Generate queries, keys, and values ​​through different convolutional layers to ensure the diversity and richness of features; Forward attention calculation: The forward attention from query to key is calculated through the forward attention module. That is, the query and key are matrix multiplied through the forward attention module to obtain the attention weight, determine the forward attention feature, and normalize it through the Softmax function to obtain the degree of attention to different features; Reverse attention calculation: The reverse attention from key to query is calculated through the reverse attention module. That is, the key and query are matrix multiplied through the reverse attention module to obtain the attention weight, determine the reverse attention feature, and obtain the degree of attention to different features, so as to further explore the relationship between different features; This bidirectional attention mechanism enables the neural network model to fully capture the temporal features from different directions and more comprehensively understand the dynamic changes of road types, thereby improving the accuracy and robustness of the neural network model recognition. Adaptive fusion: By fusing the attention output module, the emphasis on different source features (including forward attention features and reverse attention features) is flexibly adjusted according to the importance of different features, so that the neural network model can adaptively select the optimal features for fusion, thereby improving the classification performance of the neural network model. At the same time, the adaptive characteristics of the neural network model enable it to effectively handle noise and interference in complex environments, thereby further improving the recognition accuracy.

[0067] The embodiment of the present invention constructs a neural network model through a bidirectional temporal feature adaptive fusion network and an LSTM network, and introduces a bidirectional attention mechanism in the bidirectional temporal feature adaptive fusion network, so as to fully capture the information of the front-to-back correlation in the time series features, so that the neural network model can more comprehensively understand the dynamic changes of the road surface type and obtain the fused attention features. Then, the road surface type is classified based on the fused attention features through the LSTM network to obtain the road surface type classification result, which improves the accuracy and robustness of the neural network model recognition and realizes the accurate recognition of the road surface type.

[0068] The present invention provides a road type identification method based on two-way time series feature adaptive fusion. The method obtains longitudinal acceleration, lateral acceleration and vertical acceleration collected during vehicle driving, uses classical mechanics and aerodynamic models to determine dynamic temperature, thereby obtaining features for characterizing road type; then, based on the longitudinal acceleration, lateral acceleration, vertical acceleration and dynamic temperature, as well as timestamps when the longitudinal acceleration, lateral acceleration and vertical acceleration are collected, time series features are constructed; then, a neural network model is constructed through a two-way time series feature adaptive fusion network and an LSTM network, and the neural network model is used to process the time series features, fully capturing the information of the front-back correlation in the time series features, so that the neural network model outputs the classification result of the road type on the basis of a comprehensive understanding of the dynamic changes of the road type, thereby improving the accuracy and reliability of road type identification and realizing accurate prediction of the road type on which the automobile tire is located.

[0069] Optionally, the fused attention output module includes a splicing module, a gating signal generation module and a fusion module; The splicing module is used to splice the forward attention feature and the reverse attention feature to obtain a spliced ​​feature; The gating signal generating module is used to generate a gating signal based on the splicing feature; The fusion module is used to fuse the forward attention feature and the reverse attention feature based on the gating signal to obtain a fused attention feature.

[0070] Specifically, the fused attention output module includes a splicing module, a gating signal generating module and a fusion module; the splicing module is used to splice the forward attention feature and the reverse attention feature to obtain a spliced ​​feature; the gating signal generating module is used to generate a gating signal based on the splicing feature; the fusion module is used to fuse the forward attention feature and the reverse attention feature based on the gating signal to obtain a fused attention feature.

[0071] Figure 5 : is a schematic diagram of a framework combining a fusion attention output module and an LSTM network in a bidirectional temporal feature adaptive fusion network provided by the present invention, such as Figure 5 shown.

[0072] In an embodiment of the present invention, the fusion attention output module in the bidirectional temporal feature adaptive fusion network is combined with the LSTM network, so that the LSTM network can more effectively capture the context information of the input data through the bidirectional attention mechanism in the fusion attention output module, that is, fully capture the information of the before and after correlation in the time series features, and dynamically adjust the attention weights of different features. Among them, the fusion attention output module includes a splicing module, a gating signal generation module and a fusion module.

[0073] exist Figure 5 If the input time series , with T time steps of input The hidden state can be initialized and cell status for: At each time step t, the following time step loop is performed.

[0074] a. Convolutional mapping, generating queries corresponding to time steps through 1x1 convolution ,key Sum : b. Forward attention calculation, calculate the forward attention from query to key through the forward attention module, and output the forward attention feature out_forward: c. Reverse attention calculation, calculate the reverse attention from key to query through the reverse attention module, and output the reverse attention feature out_backward: d. Splicing output, through the splicing module, splicing the forward attention feature out_forward and the reverse attention feature out_backward to obtain the splicing feature : e. Gating signal generation: Generate the gating signal through the gating signal generation module and apply the sigmoid function: f. Fusion attention output, through the fusion module, based on the gated signal, the forward attention feature out_forward and the reverse attention feature out_backward are fused to obtain the fused attention feature : g.LSTM update, in the LSTM network, combined with the attention output for calculation, definition is the input gate, is the sigmoid activation function, is the hidden state at time step t, , , , is the bias vector, , , , is the corresponding weight matrix, is the cell state at time step t, For the Forgotten Gate, For the output gate, is the candidate cell state, the complete calculation process of LSTM is as follows: The state input for a time step is: The final output of LSTM It includes attention information optimized by the gating mechanism, making LSTM more flexible and effective in processing sequence data.

[0075] The embodiment of the present invention first processes and calculates the data collected by the sensor to obtain time series data, then obtains combined features with stronger representation capabilities through specific deduction and inputs them into the network, and then captures the information of the previous and next correlations in the time series through the fusion attention output module in the bidirectional temporal feature adaptive network, thereby effectively integrating signals from the past and the future, fitting the underlying manifold of the input data, and feeding back the dynamic changes of the multi-dimensional features to the neural network model, thereby improving the accuracy of the neural network model's understanding of the time series characteristics, and realizing real-time monitoring and accurate prediction of the road type on which the automobile tires are located.

[0076] Optionally, the loss function of the LSTM network is determined based on a dynamic weight factor; The dynamic weight factor is related to the prediction confidence of the neural network model and is used to dynamically adjust the loss function of the LSTM network.

[0077] Specifically, in terms of loss function design, considering the imbalance of the self-made data set, the embodiment of the present invention adopts the extended focal loss as the loss function, which is defined as: in, is the probability predicted by the model, is the weight of each category, is the focus parameter.

[0078] In order to introduce the context information of the time series and improve the accuracy of the neural network model's understanding of the characteristics of the time series, the embodiment of the present invention designs a dynamic weight factor based on the above loss function. , which is used to weight the samples of the time step, that is, to adjust the Focal Loss at each moment according to the correlation of the time series. The modified loss function expression is: Considering The introduction of is to enhance the prediction accuracy of the neural network model in time steps. Therefore, a dynamic weight factor is defined based on the prediction confidence of the neural network model. , The expression is: in, is the predicted probability of the model at time step t, and T is the total number of time steps.

[0079] The embodiment of the present invention integrates the bidirectional attention mechanism into the calculation process of LSTM, defines and introduces a dynamic weight factor in the loss function , dynamically adjust the attention weights of different features, so that the neural network model can obtain the contextual information of the input time series at each time step, improve the accuracy of the neural network model's understanding of the time series characteristics, enhance the performance of the model, especially in complex tasks that require understanding of contextual information and feature dependencies, and further enhance the accuracy of the neural network model in identifying road types.

[0080] In order to verify the performance of the neural network model in the embodiment of the present invention, the embodiment of the present invention conducted multiple groups of experiments based on the data in Table 1. The experimental results are as follows: Figures 6 to 10 shown.

[0081] Figure 6 is the single characteristic longitudinal acceleration provided by the present invention Comparison chart of category prediction and data fitting results with combined features, such as Figure 6 shown.

[0082] Figure 7 is the single characteristic lateral acceleration provided by the present invention Comparison chart of category prediction and data fitting results with combined features, such as Figure 7 shown.

[0083] Figure 8 It is the single characteristic vertical acceleration provided by the present invention Comparison chart of category prediction and data fitting results with combined features, such as Figure 8 shown.

[0084] In the experiment of actual road type classification, a single characteristic longitudinal acceleration is used (like Figure 6 As shown), single characteristic lateral acceleration (like Figure 7 as shown) or a single characteristic vertical acceleration (like Figure 8 As shown in the figure, the number of errors in road condition classification prediction is significantly higher than that in the case of using combined features. Therefore, the neural network model provided by the present invention can effectively improve the discrimination ability of the neural network model and enhance the adaptability of the neural network model to complex road conditions, thereby reducing errors by classifying road types based on combined features.

[0085] Fig. 9 This is a comparison chart of the model training loss results using cross entropy loss and focal loss provided by the present invention, such as Fig. 9 As shown in the figure, the improved loss function shows better adaptability to the situation of uneven sample distribution and can fit the original sample distribution more quickly. Compared with the cross entropy loss function, the improved loss function Focal loss significantly reduces the loss during training, improves the convergence speed of the model, and thus enhances the robustness of the model.

[0086] Fig.10 It is a schematic diagram of the confusion matrix of the classification result provided by the present invention, such as Fig.10 As shown, it reflects the performance of the model in the classification task. Through the confusion matrix, we can intuitively see the classification effect of the model in each category, the specific situation of misclassification and the overall accuracy of the model, which verifies the classification performance of the neural network model provided by the present invention, that is, it can achieve accurate classification of road surface types under supervised conditions.

[0087] Based on the above embodiments and experiments, the present invention provides a road type identification method based on two-way time series feature adaptive fusion, which generates a new feature dynamic temperature with strong characterization ability through in-depth analysis of sensor collected data and combines classical mechanics and aerodynamics, thereby more accurately describing the physical performance of automobile tires under different operating conditions; the neural network model provided by the present invention can adaptively fuse the time series feature data acquired by the sensor, enhance the comprehensiveness and accuracy of the data, and utilize the fusion attention output module in the two-way time series feature adaptive network to capture the information of the front and back correlations in the time series, thereby effectively integrating signals from the past and the future, improving the accuracy of the neural network model's understanding of the time series characteristics, and thus improving the prediction accuracy, realizing real-time monitoring and accurate prediction of the road type on which the automobile tires are running, and providing timely information and decision support for the driver.

[0088] The following describes a road surface type recognition device based on adaptive fusion of bidirectional temporal features provided by the present invention. The road surface type recognition device based on adaptive fusion of bidirectional temporal features described below and the road surface type recognition method based on adaptive fusion of bidirectional temporal features described above can refer to each other.

[0089] Based on any of the above embodiments, Fig.11 This paper provides a schematic diagram of the structure of a road type recognition device based on adaptive fusion of bidirectional temporal features. Fig.11 The embodiment of the present invention provides a road type recognition device based on bidirectional time series feature adaptive fusion, including an acquisition module 1101, a temperature module 1102, a time series module 1103 and a classification module 1104, wherein: The acquisition module 1101 is used to acquire the longitudinal acceleration, lateral acceleration and vertical acceleration collected during the vehicle driving process; the temperature module 1102 is used to determine the dynamic temperature based on the longitudinal acceleration, the lateral acceleration and the vertical acceleration; the dynamic temperature is a feature used to characterize the road surface type by analyzing the relationship between the internal gas pressure, temperature and tire acceleration of the tire using classical mechanics and aerodynamic models; the timing module 1103 is used to construct time series features based on the longitudinal acceleration, the lateral acceleration, the vertical acceleration and the dynamic temperature, as well as the timestamp when the longitudinal acceleration, the lateral acceleration and the vertical acceleration are collected; the classification module 1104 is used to input the time series features into the trained neural network model and output the road surface type classification result output by the neural network model; the neural network model is composed of a bidirectional time series feature adaptive fusion network and an LSTM network; the bidirectional time series feature adaptive fusion network is used to perform bidirectional processing on the time series features to capture the information of the front-back correlation in the time series features to obtain the fused attention features; the LSTM network is used to classify the road surface type based on the fused attention features to obtain the classification results.

[0090] Fig.12 An example of a physical structure diagram of an electronic device is shown in FIG. Fig.12 As shown, the electronic device may include: a processor 1210, a communications interface 1220, a memory 1230 and a communication bus 1240, wherein the processor 1210, the communications interface 1220 and the memory 1230 communicate with each other through the communication bus 1240. The processor 1210 may call the logic instructions in the memory 1230 to execute the road type recognition method based on the adaptive fusion of bidirectional temporal features, and the method includes: Obtain the longitudinal acceleration, lateral acceleration and vertical acceleration collected during vehicle driving; Determining a dynamic temperature based on the longitudinal acceleration, the lateral acceleration and the vertical acceleration; the dynamic temperature is a characteristic used to characterize the road surface type obtained by analyzing the relationship between the internal gas pressure, temperature and tire acceleration of the tire using a classical mechanics and aerodynamics model; constructing a time series feature based on the longitudinal acceleration, the lateral acceleration, the vertical acceleration and the dynamic temperature, and a timestamp when the longitudinal acceleration, the lateral acceleration and the vertical acceleration are collected; Inputting the time series features into a trained neural network model, and outputting a road surface type classification result output by the neural network model; The neural network model consists of a bidirectional time series feature adaptive fusion network and an LSTM network; the bidirectional time series feature adaptive fusion network is used to perform bidirectional processing on time series features to capture the information of the front and back correlation in the time series features to obtain fused attention features; the LSTM network is used to classify the road surface type based on the fused attention features to obtain a classification result.

[0091] In addition, the logic instructions in the above-mentioned memory 1230 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when it is sold or used as an independent product. Based on such an understanding, the technical solution of the present invention can be essentially or partly embodied in the form of a software product that contributes to the prior art. The computer software product is stored in a storage medium, including several instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0092] On the other hand, the present invention further provides a computer program product, the computer program product includes a computer program, the computer program can be stored in a non-transitory computer-readable storage medium, when the computer program is executed by a processor, the computer can execute the road type recognition method based on the adaptive fusion of bidirectional temporal features provided by the above methods, the method includes: Obtain the longitudinal acceleration, lateral acceleration and vertical acceleration collected during vehicle driving; Determining a dynamic temperature based on the longitudinal acceleration, the lateral acceleration and the vertical acceleration; the dynamic temperature is a characteristic used to characterize the road surface type obtained by analyzing the relationship between the internal gas pressure, temperature and tire acceleration of the tire using a classical mechanics and aerodynamics model; constructing a time series feature based on the longitudinal acceleration, the lateral acceleration, the vertical acceleration and the dynamic temperature, and a timestamp when the longitudinal acceleration, the lateral acceleration and the vertical acceleration are collected; Inputting the time series features into a trained neural network model, and outputting a road surface type classification result output by the neural network model; The neural network model consists of a bidirectional time series feature adaptive fusion network and an LSTM network; the bidirectional time series feature adaptive fusion network is used to perform bidirectional processing on time series features to capture the information of the front and back correlation in the time series features to obtain fused attention features; the LSTM network is used to classify the road surface type based on the fused attention features to obtain a classification result.

[0093] In another aspect, the present invention further provides a non-transitory computer-readable storage medium having a computer program stored thereon, which is implemented when the computer program is executed by a processor to perform the road surface type recognition method based on the adaptive fusion of bidirectional temporal features provided by the above methods, the method comprising: Obtain the longitudinal acceleration, lateral acceleration and vertical acceleration collected during vehicle driving; Determining a dynamic temperature based on the longitudinal acceleration, the lateral acceleration and the vertical acceleration; the dynamic temperature is a characteristic used to characterize the road surface type obtained by analyzing the relationship between the internal gas pressure, temperature and tire acceleration of the tire using a classical mechanics and aerodynamics model; constructing a time series feature based on the longitudinal acceleration, the lateral acceleration, the vertical acceleration and the dynamic temperature, and a timestamp when the longitudinal acceleration, the lateral acceleration and the vertical acceleration are collected; Inputting the time series features into a trained neural network model, and outputting a road surface type classification result output by the neural network model; The neural network model consists of a bidirectional time series feature adaptive fusion network and an LSTM network; the bidirectional time series feature adaptive fusion network is used to perform bidirectional processing on time series features to capture the information of the front and back correlation in the time series features to obtain fused attention features; the LSTM network is used to classify the road surface type based on the fused attention features to obtain a classification result.

[0094] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative labor.

[0095] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0096] It should be noted that, in this article, the terms "comprise", "include" or any other variant thereof are intended to cover non-exclusive inclusion, so that the process, method, article or device including a series of elements includes not only those elements, but also includes other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise one..." do not exclude the presence of other identical elements in the process, method, article or device including the element. In addition, it should be pointed out that the scope of the methods and devices in the embodiments of the present application is not limited to performing functions in the order shown or discussed, and may also include performing functions in a substantially simultaneous manner or in reverse order according to the functions involved, for example, the described method may be performed in an order different from that described, and various steps may also be added, omitted, or combined. In addition, the features described with reference to certain examples may be combined in other examples.

[0097] "Determine B based on A" in the embodiments of the present application means that the factor A should be considered when determining B. It is not limited to "B can be determined based on A alone", but should also include: "Determine B based on A and C", "Determine B based on A, C and E", "Determine C based on A, and further determine B based on C", etc. In addition, it can also include taking A as a condition for determining B, for example, "When A meets the first condition, use the first method to determine B"; for another example, "When A meets the second condition, determine B", etc.; for another example, "When A meets the third condition, determine B based on the first parameter", etc. Of course, it can also be a condition that takes A as a factor for determining B, for example, "When A meets the first condition, use the first method to determine C, and further determine B based on C", etc.

[0098] In the present invention, the term "plurality" refers to two or more than two, and other quantifiers are similar to them.

[0099] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A road type recognition method based on adaptive fusion of bidirectional temporal features, characterized in that: include: Obtain the longitudinal acceleration, lateral acceleration and vertical acceleration collected during vehicle driving; Determining a dynamic temperature based on the longitudinal acceleration, the lateral acceleration and the vertical acceleration; the dynamic temperature is a characteristic used to characterize the road surface type obtained by analyzing the relationship between the internal gas pressure, temperature and tire acceleration of the tire using a classical mechanics and aerodynamics model; constructing a time series feature based on the longitudinal acceleration, the lateral acceleration, the vertical acceleration and the dynamic temperature, and a timestamp when the longitudinal acceleration, the lateral acceleration and the vertical acceleration are collected; Inputting the time series features into a trained neural network model, and outputting a road surface type classification result output by the neural network model; The neural network model consists of a bidirectional time series feature adaptive fusion network and an LSTM network; the bidirectional time series feature adaptive fusion network is used to perform bidirectional processing on time series features to capture the information of the front and back correlation in the time series features to obtain fused attention features; the LSTM network is used to classify the road surface type based on the fused attention features to obtain a classification result.

2. The road surface type recognition method based on bidirectional temporal feature adaptive fusion according to claim 1 is characterized in that: The bidirectional temporal feature adaptive fusion network includes a convolutional layer, a forward attention module, a reverse attention module and a fusion attention output module; The convolutional layer is used to generate queries, keys and values ​​from the time series features; The forward attention module is used to determine a forward attention feature based on the query, the key and the value; The reverse attention module is used to determine a reverse attention feature based on the query, the key and the value; The fused attention output module is used to determine the fused attention feature based on the forward attention feature and the reverse attention feature.

3. The road surface type recognition method based on bidirectional temporal feature adaptive fusion according to claim 2 is characterized in that: The fusion attention output module includes a splicing module, a gating signal generation module and a fusion module; The splicing module is used to splice the forward attention feature and the reverse attention feature to obtain a spliced ​​feature; The gating signal generating module is used to generate a gating signal based on the splicing feature; The fusion module is used to fuse the forward attention feature and the reverse attention feature based on the gating signal to obtain a fused attention feature.

4. The road surface type recognition method based on bidirectional temporal feature adaptive fusion according to claim 1 is characterized in that: The loss function of the LSTM network is determined based on a dynamic weight factor; The dynamic weight factor is related to the prediction confidence of the neural network model and is used to dynamically adjust the loss function of the LSTM network.

5. The road surface type recognition method based on bidirectional temporal feature adaptive fusion according to claim 1 is characterized in that: The determining of the dynamic temperature based on the longitudinal acceleration, the lateral acceleration and the vertical acceleration comprises: determining a total magnitude of acceleration based on the longitudinal acceleration, the lateral acceleration, and the vertical acceleration; A dynamic temperature is determined based on the total magnitude of the acceleration.

6. The road surface type recognition method based on bidirectional temporal feature adaptive fusion according to claim 1 is characterized in that: The obtaining of the longitudinal acceleration, lateral acceleration and vertical acceleration collected during the vehicle driving process includes: Acquire the longitudinal acceleration, lateral acceleration and vertical acceleration collected by the acceleration sensor arranged on the inner wall of the tire contact surface during the vehicle driving process; The direction of the longitudinal acceleration is along the forward direction of the vehicle; the direction of the lateral acceleration is perpendicular to the forward direction of the vehicle; and the direction of the vertical acceleration is perpendicular to the ground.

7. A road type recognition device based on adaptive fusion of bidirectional temporal features, characterized in that: include: An acquisition module is used to acquire the longitudinal acceleration, lateral acceleration and vertical acceleration collected during the vehicle's driving process; a temperature module, for determining a dynamic temperature based on the longitudinal acceleration, the lateral acceleration and the vertical acceleration; the dynamic temperature is a characteristic used to characterize a road surface type obtained by analyzing the relationship between tire internal gas pressure, temperature and tire acceleration using a classical mechanics and aerodynamics model; A time series module, for constructing a time series feature based on the longitudinal acceleration, the lateral acceleration, the vertical acceleration and the dynamic temperature, and a timestamp when the longitudinal acceleration, the lateral acceleration and the vertical acceleration are collected; A classification module, used for inputting the time series features into a trained neural network model and outputting a road surface type classification result output by the neural network model; The neural network model consists of a bidirectional time series feature adaptive fusion network and an LSTM network; the bidirectional time series feature adaptive fusion network is used to perform bidirectional processing on time series features to capture the information of the front and back correlation in the time series features to obtain fused attention features; the LSTM network is used to classify the road surface type based on the fused attention features to obtain a classification result.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, the road surface type recognition method based on adaptive fusion of bidirectional temporal features as described in any one of claims 1 to 6 is implemented.

9. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the road surface type recognition method based on adaptive fusion of bidirectional temporal features as described in any one of claims 1 to 6 is implemented.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the road surface type recognition method based on adaptive fusion of bidirectional temporal features as described in any one of claims 1 to 6 is implemented.

Images