Pavement-type recognition method and system, and electronic apparatus and vehicle

Abstract

A pavement-type recognition method and system, and an electronic apparatus and a vehicle. The method comprises: on the basis of detection data, obtaining a target pavement type, wherein the detection data at least comprises raw pavement haptic-data, which is pavement haptic-data that is not absorbed by a damping device.

Description

Road surface type identification methods, systems, electronic devices and vehicles

[0001] This application claims priority to Chinese patent application No. 202411046132.8, filed on July 31, 2024, the entire contents of which are incorporated herein by reference. Technical Field

[0002] This disclosure belongs to the field of intelligent vehicle technology, and in particular relates to a road surface type recognition method, system, electronic device and vehicle. Background Technology

[0003] With the development of intelligent vehicle technology, road surface type recognition technology has become an important factor affecting vehicle driving safety and comfort. For example, image acquisition modules installed on vehicles can collect image information of the road surface the vehicle is traveling on, identify the road surface type based on the image information, and then select the appropriate vehicle driving mode according to the road surface type. Summary of the Invention

[0004] This disclosure provides a road surface type identification method, system, electronic device, and vehicle that can solve the problem of low accuracy in road surface type identification.

[0005] In a first aspect, a road surface type identification method is provided, the method comprising: obtaining a target road surface type based on detection data; wherein the detection data includes at least: original road surface tactile data, wherein the original road surface tactile data is road surface tactile data that has not been absorbed by the shock absorption device.

[0006] Secondly, a road surface type identification device is provided, comprising: a processing module; the processing module is used to obtain the target road surface type based on detection data. The detection data includes at least: raw road surface tactile data, wherein the raw road surface tactile data is road surface tactile data that has not been absorbed by the shock absorption device.

[0007] Thirdly, a road surface recognition system is provided, comprising: a wheel, a sensor, and a controller. The wheel is rigidly connected to the sensor, which is used to collect detection data. The controller is communicatively connected to the sensor and is used to determine the target road surface type based on the detection data. The detection data includes at least: raw road surface tactile data, which is road surface tactile data not absorbed by the shock absorption device.

[0008] Fourthly, an electronic device is provided, comprising: a memory, a processor, and a computer program stored in the memory, wherein the processor executes the computer program to implement the road surface type identification method described in the first aspect.

[0009] Fifthly, a computer-readable storage medium is provided, on which a program or instructions are stored, which, when executed by a processor, implement the road surface type identification method described in the first aspect.

[0010] In a sixth aspect, a computer program product is provided, comprising: a computer program or instructions, wherein when the computer program or instructions are executed by a processor, the road surface type identification method described in the first aspect is implemented.

[0011] In a seventh aspect, a vehicle is provided for use in a road surface recognition system. The vehicle is communicatively connected to a cloud server, and under preset conditions, the vehicle executes the road surface type recognition method described in the first aspect. The preset conditions include one of the following: determining that the cloud server is faulty, or that the vehicle is offline.

[0012] The road surface type identification method, system, electronic device, and vehicle provided in some embodiments of this disclosure obtain the target road surface type based on detection data. Since the detection data includes at least the original road surface tactile data, the original road surface tactile data is not affected by factors such as light and weather, and the original road surface tactile data is the road surface tactile data that has not been absorbed by the shock absorption device, it can more accurately reflect the road surface condition. Therefore, it can improve the accuracy of road surface type identification. Attached Figure Description

[0013] Figure 1 is a flowchart of a road surface type identification method according to some embodiments;

[0014] Figure 2 is a flowchart of another road surface type identification method according to some embodiments;

[0015] Figure 3 is a flowchart of another road surface type identification method according to some embodiments;

[0016] Figure 4 is a flowchart of another road surface type identification method according to some embodiments;

[0017] Figure 5 is a flowchart of obtaining the first identification result according to some embodiments;

[0018] Figure 6 is a flowchart of another road surface type identification method according to some embodiments;

[0019] Figure 7 is a flowchart of obtaining the second identification result according to some embodiments;

[0020] Figure 8 is a flowchart of the arbitration module performing an arbitration method according to some embodiments;

[0021] Figure 9 is a flowchart of another road surface type identification method according to some embodiments;

[0022] Figure 10 is a system architecture diagram of a vehicle-cloud collaborative road type identification method according to some embodiments;

[0023] Figure 11 is a structural diagram of a road surface recognition system according to some embodiments;

[0024] Figure 12 is a structural diagram of another road surface recognition system according to some embodiments;

[0025] Figure 13 is a block diagram of a vehicle and a cloud server according to some embodiments;

[0026] Figure 14 is a block diagram of an electronic device according to some embodiments. Detailed Implementation

[0027] The technical solutions of the embodiments of this disclosure will be clearly described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this disclosure. All other embodiments obtained by those skilled in the art based on the embodiments of this disclosure are within the scope of protection of this disclosure.

[0028] It should be noted that, in this document, the terms “comprising,” “including,” or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus.

[0029] Without further limitations, an element defined by the phrase "comprising a..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element. Furthermore, it should be noted that the scope of the methods and apparatus in this disclosure is not limited to performing functions in the order shown or discussed, but may also include performing functions substantially simultaneously or in the reverse order, depending on the functions involved. For example, the described methods may be performed in a different order than described, and various steps may be added, omitted, or combined. Additionally, features described with reference to certain examples may be combined in other examples.

[0030] In related technologies, road surface type identification is performed by collecting image information of the road surface on which the vehicle is driving, thereby determining the driving mode. However, due to the influence of factors such as lighting and weather, the quality of the collected image information is not high and cannot reflect the road surface condition well, resulting in low accuracy of road surface type identification.

[0031] Some embodiments of this disclosure obtain the target road surface type based on detection data. The detection data includes at least: raw road surface tactile data. Since the raw road surface tactile data is not affected by factors such as light and weather, and is road surface tactile data that has not been absorbed by the shock absorption device, it can more accurately reflect the road surface condition. Therefore, it can improve the accuracy of road surface type identification. It should be noted that the target road surface type is one of several road surface types.

[0032] To further improve the accuracy of road surface type identification, some embodiments of this disclosure also provide a multi-sensory fusion technical solution, which combines acoustic data with the original road surface tactile data, and determines the target road surface type based on the original road surface tactile data and acoustic data, thereby enriching the amount of information reflected in the road surface condition and further improving the accuracy of road surface type identification.

[0033] To further improve the accuracy of road surface type identification, some embodiments of this disclosure also incorporate road surface visual data. The road surface type determined based on the road surface visual data is compared with the road surface type determined based on the original road surface tactile data and acoustic data, and the final target road surface type is determined through arbitration.

[0034] Furthermore, in order to improve the accuracy of road surface type determination based on road surface visual data, the road surface visual data is also processed by frame segmentation, and road surface type identification is performed on multiple frames of images respectively. The road surface type determined based on the road surface visual data is then determined through decision fusion.

[0035] In addition to improving the accuracy of road type recognition, some embodiments of this disclosure also provide a vehicle-cloud collaborative road type recognition method. On the one hand, by executing the road type recognition method on a cloud server, the computing power requirements on the vehicle's local machine are reduced, saving local computing resources and energy consumption. On the other hand, when the cloud server malfunctions or the vehicle is offline and unable to communicate with the cloud server, the road type recognition method can be executed by the vehicle's local processor to determine the road type, thereby ensuring the reliability of road type recognition and improving driving safety.

[0036] In the above method, the cloud server and the vehicle can use the same road type recognition model to perform the road type recognition method. The road type recognition model is trained on the cloud server, and the cloud server will update the vehicle's road type recognition model based on preset conditions.

[0037] The technical solutions of this disclosure are described below through some embodiments. The road surface type identification method described in the embodiments shown in Figures 1 to 6 can be executed by a cloud server or by a vehicle.

[0038] As shown in Figure 1, the road surface type identification method includes steps S10-S20.

[0039] S10: Acquire detection data.

[0040] For example, the test data includes at least: raw road surface tactile data, which is road surface tactile data that has not been absorbed by the damping equipment.

[0041] In some embodiments, detection data can be acquired by data acquisition sensors.

[0042] For example, the data acquisition sensor includes a tactile sensor. The raw road surface tactile data is acquired based on the tactile sensor, which is used to collect the raw road surface tactile data. The tactile sensor may be an accelerometer.

[0043] The tactile sensor is located on the wheel; or, it is located at a connecting frame rigidly connected to the wheel, the connecting frame being positioned between the shock absorber and the wheel; or, it is located on the subframe, the subframe being rigidly connected to the wheel; or, it is located on the steering tie rod, the steering tie rod being rigidly connected to the wheel. The detection data includes vibration acceleration data. The wheel may be a tire.

[0044] Since the tactile sensors are located on the above components, and these components are synchronized with the vibration state of the wheel, the vibration signals collected by the tactile sensors on the above components can fully reflect the vibration state of the wheel, avoiding the influence of shock absorption equipment on vibration acceleration data. Since the vibration state of the wheel is a response to the road surface condition, it can reflect the road surface condition well.

[0045] In some embodiments, the method further includes: denoising the detection data.

[0046] For example, the vibration acceleration data corresponding to the vibration signal can be obtained by denoising the vibration signal collected by the tactile sensor as follows: The main form of the collected vibration signal is an analog voltage signal. The vibration signal is converted from analog to digital, which can be achieved by an analog-to-digital converter, to obtain the digital signal of the vibration signal. The digital signal of the vibration signal is denoised, for example, by filtering the influence of vehicle acceleration, deceleration, turning, and change of direction on the vibration signal through a high-pass filter. Then, the denoised vibration signal is analyzed and extracted to obtain the vibration acceleration data corresponding to the vibration signal.

[0047] Vibration acceleration data is a quantity used to characterize the rate of change of vibration signals. Vibration acceleration data includes three-axis acceleration data: X, Y, and Z. The X-axis is the lateral acceleration data, the Y-axis is the longitudinal acceleration data, and the Z-axis is the vertical acceleration data. The acceleration data in these three directions reflects the vibration of the vehicle in different directions. Therefore, the vehicle's response to the road surface can be analyzed from multiple dimensions, which helps to more accurately determine the road surface type.

[0048] S20: Based on the detection data, the target road surface type is obtained.

[0049] In some embodiments, an identification result is obtained based on the detection data; and a target road surface type is obtained based on the identification result. Obtaining the identification result based on the detection data includes: obtaining data features based on the detection data; and obtaining the identification result based on the data features.

[0050] Based on the detection data, the method for obtaining data features is shown in Figure 2. The road surface type identification method further includes steps S201-S203 as follows:

[0051] S201: Extract features from the original road surface tactile data to obtain the tactile data features corresponding to the original road surface tactile data.

[0052] The data features include: tactile data features corresponding to the original road surface tactile data.

[0053] In some embodiments, the raw road surface tactile data includes: vibration acceleration data, and the data features include: vibration acceleration features corresponding to the vibration acceleration data.

[0054] Feature extraction is performed on the original road surface tactile data to obtain the tactile data features corresponding to the original road surface tactile data, including: obtaining the time-domain features, frequency-domain features, and time-frequency-domain features corresponding to the triaxial acceleration data of the vibration acceleration data, and obtaining the tactile data features based on at least one of the time-domain features, frequency-domain features, or time-frequency-domain features; for example, the triaxial acceleration data includes: lateral acceleration data, vertical acceleration data, and longitudinal acceleration data.

[0055] In some embodiments, acquiring the time-domain features, frequency-domain features, and time-frequency-domain features corresponding to the triaxial acceleration data of vibration acceleration data includes: acquiring the time-domain features corresponding to the acceleration data of each of the three axes. For example, the time-domain features include at least one of the following: mean, variance, standard deviation, maximum value, minimum value, and peak-to-peak value; or, performing a Fourier transform on the time-domain features to obtain the frequency-domain features corresponding to the acceleration data of the axis; the frequency-domain features include at least one of the following: average frequency, maximum frequency, energy spectrum, and center frequency; or, extracting wavelet transform coefficients by performing a continuous wavelet transform on the time-domain features, and obtaining the time-frequency-domain features corresponding to the acceleration data of the axis based on the wavelet transform coefficients of each layer. The time-frequency-domain features involve both time information and frequency information, and include statistical features such as the mean and variance of the wavelet coefficients of each layer.

[0056] S202: Based on the data features, obtain the first identification result.

[0057] In some embodiments, the first recognition result is obtained based on the tactile data features and the first recognition model; the first recognition model includes the correspondence between the tactile data features and the road surface type.

[0058] In some embodiments, the data features are input into a first recognition model to obtain a first recognized road surface and a first confidence level; the first recognition result is obtained based on the first recognized road surface and the first confidence level; and the first recognition result is used as the recognition result.

[0059] The first recognition model is trained based on at least one of time-domain features, frequency-domain features, or time-frequency-domain features and the corresponding road surface type as input. The first recognition model is one or more of decision tree, support vector machine, random forest, and multilayer perceptron. The first recognition model can be trained and updated based on preset conditions.

[0060] In some embodiments, inputting the data features into a first recognition model to obtain a first recognized road surface and a first confidence level includes: inputting the data features into the first recognition model to obtain multiple candidate recognized road surfaces and a candidate confidence level corresponding to each candidate recognized road surface; and based on the candidate confidence level corresponding to each candidate recognized road surface, determining the candidate recognized road surface corresponding to the highest candidate confidence level as the first recognized road surface, and determining the highest confidence level as the first confidence level.

[0061] For example, suppose the first recognition model is a deep learning model. A series of data features are input into this model. It will output multiple possible road surface types as candidate road surfaces, and provide a first confidence level for each candidate road surface. The first confidence level indicates the probability or certainty that the first recognition model considers the candidate road surface to be correct.

[0062] For example, if the first identification model identifies three possible road surface types: asphalt, cement, and gravel, and provides three candidate confidence scores: 0.85 for asphalt, 0.75 for cement, and 0.60 for gravel, it's understandable that asphalt has the highest candidate confidence score. Therefore, the first identification model will determine asphalt as the first identified road surface and use 0.85 as the first confidence score.

[0063] S203: Use the first recognition result as the recognition result.

[0064] In some embodiments, by acquiring detection data, the target road surface type is obtained based on the detection data. Since the detection data includes at least the original road surface tactile data, the original road surface tactile data is not affected by factors such as light and weather, and the original road surface tactile data is the road surface tactile data that has not been absorbed by the shock absorption device, it can more accurately reflect the road surface condition. Therefore, it can improve the accuracy of road surface type identification.

[0065] In some embodiments, as shown in FIG3, the detection data further includes acoustic data. Accordingly, step S20 includes step S20'.

[0066] S20': Based on acoustic data and raw road surface tactile data, the target road surface type is obtained.

[0067] In some embodiments, since the detection data includes raw road surface tactile data and acoustic data, that is, when identifying the target road surface type, in addition to considering the raw road surface tactile data, acoustic data is also combined. The target road surface type is determined based on the raw road surface tactile data and acoustic data, which enriches the amount of information in the data reflecting the road surface condition. Therefore, the accuracy of the identified road surface type is further improved.

[0068] In some embodiments, as shown in FIG4, the detection data further includes: acoustic data, and correspondingly, the data features further include: acoustic data features corresponding to the acoustic data. Step S202 includes steps S2020-S2022.

[0069] S2020: Acoustic data features are obtained based on acoustic data.

[0070] For example, acoustic data can be collected using acoustic sensors, which can be installed on the rear fender of the wheel. The acoustic data includes tire noise frequency data, and correspondingly, the acoustic data features include the audio features corresponding to the tire noise frequency data.

[0071] S2021: The tactile data features and acoustic data features are fused to obtain fused features.

[0072] For example, by fusing tactile and acoustic data features, a fused feature is obtained, which combines two different data features to provide more comprehensive and richer road surface information.

[0073] Tactile data features reflect the physical vibration characteristics when a vehicle contacts the road surface, while acoustic data reflects the noise characteristics generated by the interaction between the tire and the road surface. By fusing these two types of data features, the ability of road type identification methods to perceive changes in road conditions can be enhanced, improving identification accuracy and robustness.

[0074] Feature fusion can also improve the robustness of road surface type identification methods, because even if one sensor is affected by noise or malfunction, data from other sensors can still provide useful supplementary information, thereby reducing the uncertainty and error of a single data source. Feature fusion can provide higher quality input for road surface type identification methods, enabling more accurate and reliable road surface type determinations.

[0075] The method for obtaining the audio features corresponding to tire noise frequency data is as follows:

[0076] Tire noise frequency data is acquired and preprocessed, such as by pre-emphasis, framing, windowing, short-time Fourier transform, and Mel filtering, to obtain the Mel spectrogram corresponding to the tire noise frequency data. Preprocessing the tire noise frequency data effectively extracts useful information for road surface type identification, reduces noise interference, and thus improves the accuracy and robustness of road surface type identification.

[0077] Feature extraction is performed on tire noise frequency data, for example, by using a deep residual network to extract features from the Mel spectrogram corresponding to the tire noise frequency data, thus obtaining the audio features corresponding to the tire noise frequency data. The deep residual network, through its multi-layered structure, can effectively learn and capture complex patterns and deep features in the Mel spectrogram.

[0078] Residual connections in a network help mitigate the vanishing gradient problem during deep network training, allowing the network to be trained deeper and thus learn and represent data better. Audio features extracted in this way can reflect the acoustic properties generated by the interaction between the road surface and tires. Feature extraction based on deep residual networks can improve the accuracy and robustness of road surface type identification.

[0079] For example, the Mel spectrogram corresponding to the audio can be input into a Residual Network 18 (ResNet-18) to output the audio features extracted from the audio data. The ResNet-18 network mainly includes four convolutional layers to extract local image features, eight residual blocks to solve the gradient vanishing problem, and a global average pooling layer to transform the feature map into a one-dimensional vector, thereby obtaining the audio features.

[0080] In some embodiments, tactile data features and acoustic data features are fused to obtain fused features. For example, the method of fusing vibration acceleration features and audio features to obtain fused features includes: utilizing the complementarity of audio features and vibration acceleration features, inputting the separately processed audio features and vibration acceleration features together into the feature fusion module for fusion to obtain fused features.

[0081] In some embodiments, the tactile data features and the acoustic data features are spliced ​​together to obtain fused features.

[0082] For example, vibration acceleration features are represented as a first vector, and audio features as a second vector. The first and second vectors are directly concatenated to form a longer vector to represent the fused features. The fused feature vector is then input into the first recognition model to establish a mapping relationship between the feature vector and the road surface type.

[0083] In some embodiments, to reduce computational load, the fusion features can also be dimensionality reduced, for example, by performing Principal Component Analysis (PCA) dimensionality reduction.

[0084] S2022: Based on the fusion features, obtain the first recognition result.

[0085] In some embodiments, the fused features are input into a multilayer perceptron (MLP) to obtain a first identification result corresponding to the road surface to be identified. The first identification result includes: a first identified road surface and a first confidence level. The MLP includes: the correspondence between the fused features and the road surface type.

[0086] For example, suppose an MLP has been trained to identify road surface types based on fused features. These fused features are obtained by fusing features from acoustic and tactile data, and these features can reflect the vehicle's response characteristics to different road surfaces.

[0087] In a vehicle driving scenario, acoustic and tactile data of the current road surface were acquired, and a fused feature was generated through feature extraction and fusion steps. This fused feature was then fed into an MLP model. The MLP model contains multiple hidden layers that process the input fused feature using non-linear activation functions, learning the mapping relationship between the fused feature and the road surface type. After a series of weight and bias calculations, the MLP's output layer provides a probability distribution representing the likelihood that the MLP considers the input feature to belong to different road surface types.

[0088] For example, if the MLP output layer has three neurons, corresponding to asphalt, cement, and gravel road types respectively, the output probability distribution might be: asphalt road with a probability of 0.70, cement road with a probability of 0.25, and gravel road with a probability of 0.05. In this example, the MLP model determines that asphalt road has the highest probability, so it uses asphalt road as the first identification result and 0.70 as the first confidence level.

[0089] In some embodiments, fused features are obtained by fusing tactile data features and acoustic data features. Based on the fused features, a first recognition result is obtained. In this process, acoustic data features are combined, which can also reflect the state information of the road surface. Moreover, tactile data features and acoustic data features are complementary. Tactile data features reflect the road surface state from a tactile perspective, while acoustic data features reflect the road surface state from an auditory perspective. Therefore, the amount of information reflected in the road surface state is enriched. Obtaining the first recognition result based on the fused features further improves the accuracy of the identified road surface type.

[0090] For ease of understanding, some embodiments of this disclosure also provide schematic diagrams of obtaining a first recognition result based on tactile data features and acoustic data features. The tactile data features take vibration acceleration features as an example, and the acoustic data features take audio features as an example, as shown in Figure 5. It includes: data processing process (step S501), feature extraction process (step S502), feature fusion process (step S503), and recognition result (step S504). The process is described in the foregoing embodiments and will not be repeated here.

[0091] In some embodiments, as shown in FIG6, the detection data further includes: road surface visual data. The road surface visual data can be acquired by a visual sensor, such as a camera. By combining the road surface type identified based on the road surface visual data, the final target road surface type is determined. The method further includes the following steps S2023 and S203'.

[0092] S2023: Obtain a second recognition result based on road surface visual data.

[0093] In some embodiments, as shown in FIG7, the road surface visual data is subjected to frame segmentation processing (step S701) to obtain multiple image frames corresponding to the road surface visual data.

[0094] For example, multiple image frames within a preset time period are acquired, such as visual image 1, visual image 2, visual image 3, ..., visual image N, and preprocessing operations are performed on each visual image. In some embodiments, to avoid interference from the surrounding environment on the video image, the visual images obtained by frame division can also be preprocessed, such as image cropping and normalization processing for each visual image.

[0095] Obtain the candidate road surface type recognition results corresponding to the multiple image frames respectively; input the multiple image frames into the ResNet-18 network structure for feature extraction (step S702), and obtain the candidate road surface type recognition results corresponding to each visual image. The road surface recognition results include: candidate road surface type.

[0096] In some embodiments, the road surface recognition result may further include: candidate confidence scores corresponding to candidate road surface types. The confidence score provides the level of trust in the recognition result. A high confidence score means that the recognition result is more reliable, while a low confidence score means that the recognition result is less reliable and may require further verification. Thus, the reliability of the decision can be improved by using the confidence scores corresponding to candidate road surface types.

[0097] As shown in Figure 7, the candidate road surface type 1 corresponding to visual image 1 is y. 2,1 With a confidence level of α 2，1 The candidate road surface type 2 corresponding to visual image 2 is y. 2,2 With a confidence level of α 2，2 Correspondingly, the candidate road surface type N corresponding to the visual image N is y. 2,N With a confidence level of α 2，N .

[0098] The candidate road surface type identification results corresponding to the multiple image frames are fused by decision (step S703) to obtain the second identification result. The second identified road surface is determined based on the candidate road surface type with the highest frequency of occurrence; the second confidence level is determined based on the candidate confidence level corresponding to the candidate road surface type with the highest frequency of occurrence; and the second identification result is obtained based on the second identified road surface and the second confidence level (step S704).

[0099] For example, after framing the road surface visual data, a total of 10 visual images are obtained, and the candidate road surface type corresponding to 8 of these visual images is cement road; therefore, cement road is determined as the second recognition result. For example, the second confidence level is determined based on the candidate confidence level corresponding to the candidate road surface type with the highest frequency of occurrence.

[0100] In some embodiments, the mean of the candidate confidence scores for the most frequently occurring candidate road surface type is determined as the second confidence score; or, the maximum of the candidate confidence scores for the most frequently occurring candidate road surface type is determined as the second confidence score; or, the minimum of the candidate confidence scores for the most frequently occurring candidate road surface type is determined as the second confidence score. The second identification result can be represented as y. 2 The corresponding confidence level can be expressed as α2.

[0101] S203': Based on the first identification result and the second identification result, the target road surface type is obtained.

[0102] In some embodiments, if the first identified road surface is the same as the second identified road surface, then at least one of the first identified road surface or the second identified road surface is determined to be the target road surface type; in some embodiments, the average of the first confidence level and the second confidence level can also be determined as the confidence level of the target road surface type.

[0103] If the first identified road surface is different from the second identified road surface, then the road surface type with the higher confidence level between the first confidence level and the second confidence level is determined as the target road surface type. In some embodiments, if the first confidence level is high, then the first confidence level is determined as the confidence level of the target road surface type; if the second confidence level is high, then the second confidence level is determined as the confidence level of the target road surface type.

[0104] Assumption: The first recognition result can be represented as y 1 The corresponding first confidence level can be represented as α1. The second recognition result can be represented as y. 2 The corresponding second confidence level can be represented as α2.

[0105] This process can be executed by the arbitration module, and the corresponding flowchart is shown in Figure 8. Input the first recognition result y. 1 And the first confidence level α1 (step S801), the second recognition result y 2 The first confidence level α2 represents the credibility of the first identification result, and the second confidence level represents the credibility of the second identification result. By inputting the first confidence level and the second confidence level, it is beneficial to make decision-making judgments based on the identification results based on different detection signals according to the confidence level.

[0106] For example, it is determined whether the first identification result is the same as the second identification result (step S803). If they are the same, the target road surface type is determined to be either the first identification result or the second identification result (step S804), and the confidence level of the target road surface type is determined to be... If they are different, determine whether the first confidence level α1 is greater than or equal to the second confidence level α2 (step S805). If it is greater than or equal to α2, determine the target road surface type as the first identification result, and the confidence level of the target road surface type is α1 (step S806). If it is not greater than or equal to α2, determine the target road surface type as the second identification result, and the second confidence level of the target road surface type is α2 (step S807). Output the target road surface type and the corresponding confidence level.

[0107] In some embodiments, by further combining the second recognition result corresponding to the road surface visual data obtained based on road surface visual data, and based on the first recognition result and the second recognition result, the target road surface type is obtained. This is equivalent to further combining visual information on the basis of tactile and auditory information, and determining the target road surface type based on the features of the three senses, thereby further improving the accuracy of the identified road surface type. In addition, by performing frame-by-frame processing on the road surface visual data, road surface type recognition is performed on multiple frames of images respectively, and the second recognition result is determined through decision fusion, thereby improving the accuracy of the second recognition result.

[0108] The embodiments disclosed above utilize the relatively low acquisition frequency of acoustic and tactile data, making it easier to achieve frequency fusion. Simultaneously, acoustic and tactile data are easy to quantize, providing stable and reliable feature information for road surface type identification.

[0109] Some embodiments of this disclosure also consider the limitations of high-frequency visual data acquisition in terms of computational load, processing efficiency, and accuracy when performing quantization processing. Therefore, acoustic and tactile data are first fused, and then comprehensively considered with the visual recognition results to obtain the final road surface recognition result, balancing computational efficiency and accuracy.

[0110] In some embodiments, as shown in FIG9, after obtaining the target road surface type, the method further includes step S30.

[0111] S30: Set the vehicle driving mode based on the target road surface type.

[0112] For example, by establishing a correspondence between road surface types and vehicle driving modes, the vehicle driving mode corresponding to the road surface type can be selected.

[0113] In some embodiments, because the road surface type is determined more accurately, the driving mode selected based on the road surface type is also more appropriate, thereby improving driving safety and comfort.

[0114] The road surface type identification methods corresponding to the embodiments in Figures 1 to 9 can be executed by cloud services or by vehicles.

[0115] Some embodiments of this disclosure also provide a road surface type recognition method for vehicle-cloud collaboration, as shown in Figure 10. The vehicle-cloud collaboration system includes a cloud server and a vehicle. The cloud server and the vehicle are communicatively connected. In this embodiment, the cloud server is used as the primary recognition terminal, and the vehicle is used as the secondary recognition terminal. Under normal circumstances, the cloud server performs road surface type recognition and sends the recognized road surface type to the vehicle.

[0116] When the cloud server malfunctions or the vehicle is offline and unable to communicate with the cloud server, the vehicle's local processor can identify the road type to determine the road type.

[0117] The cloud server includes: a data cloud storage module 1001 and a data cloud computing module 1002. The vehicle includes: a data acquisition module 1003, a data processing module 1004, a road surface type recognition module 1005, a result arbitration module 1006, and a vehicle driving mode control module 1007.

[0118] The data acquisition module 1003 is used to acquire detection data. The process of the data acquisition module acquiring detection data is described in the embodiments corresponding to Figures 1 to 9, and will not be repeated here.

[0119] It is important to note that timestamp alignment is required to ensure that the detection data collected by each sensor in the data acquisition module is synchronized in time.

[0120] The data cloud storage module 1001 is configured to store the acquired detection data.

[0121] The data cloud computing module 1002 is configured to identify road surface type based on detection data and send the identification results to the vehicle driving mode control module.

[0122] The data cloud computing module 1002 includes: a data preprocessing module, a road surface type identification module, and a result arbitration module. The data preprocessing module is used to preprocess the detection data. The road surface type identification module includes: a first identification model and a second identification model.

[0123] For example, a first recognition model is used to identify road surface types based on data collected by a tactile sensor and an acoustic sensor, while a second recognition model is used to identify road surface types based on data collected by a visual sensor. A result arbitration module arbitrates the results of the first and second recognition models to obtain the final road surface recognition result.

[0124] The road surface type recognition module is also configured to perform model training and updating. The road surface type recognition module will train the first recognition model and the second recognition model based on preset conditions, and update the first recognition model and the second recognition model based on the training results. At the same time, it will update the first recognition model and the second recognition model in the vehicle's road surface type recognition module.

[0125] Preset conditions include a fixed period, such as training once every month, or based on user feedback results, such as recognition accuracy being lower than the preset value.

[0126] In order to increase training data, prevent overfitting, and improve training accuracy during model training, this disclosure also employs image data enhancement methods such as mirror flipping, changing contrast, and changing brightness.

[0127] The data processing module 1004 is configured to preprocess the acquired detection data.

[0128] The road surface type recognition module 1005 includes a first recognition model and a second recognition model.

[0129] The first recognition model is used to identify road surface types based on data collected by tactile sensors and acoustic sensors, while the second recognition model is used to identify road surface types based on data collected by visual sensors.

[0130] As a result, the arbitration module 1006 is configured to arbitrate the results of the first recognition model and the second recognition model to obtain the final road surface recognition result.

[0131] The vehicle driving mode control module 1007 is configured to control the vehicle's driving mode based on road surface recognition results.

[0132] In some embodiments, on the one hand, by executing the road type recognition method on a cloud server, the computational power requirements on the vehicle's local area are reduced, saving local computing resources and energy consumption. On the other hand, when the cloud server fails or the vehicle is offline and unable to communicate with the cloud server, the road type recognition method can be executed by the vehicle's local processor to determine the road type, thereby ensuring the reliability of road type recognition and improving driving safety.

[0133] In some embodiments, the data cloud computing module can also determine the vehicle's driving mode based on the road surface type recognition result and directly send the vehicle's driving mode to the vehicle so that the vehicle adopts the vehicle's driving mode, which can further save the vehicle's computing resources and energy consumption.

[0134] This disclosure provides a road surface type identification device, including a processing module, according to some embodiments.

[0135] The processing module is configured to determine the target road surface type based on the detection data. The detection data includes at least: raw road surface tactile data, which is the road surface tactile data that has not been absorbed by the damping device.

[0136] In some embodiments, the processing module is configured to obtain an identification result based on the detection data; and to obtain the target road surface type based on the identification result.

[0137] In some embodiments, the processing module is configured to obtain data features based on the detection data; and to obtain an identification result based on the data features.

[0138] In some embodiments, the processing module is configured to extract features from the original road surface tactile data to obtain tactile data features corresponding to the original road surface tactile data; obtain a first recognition result based on the tactile data features; and use the first recognition result as the recognition result.

[0139] In some embodiments, the raw road surface tactile data includes vibration acceleration data.

[0140] Accordingly, the data features include: the vibration acceleration features corresponding to the vibration acceleration data.

[0141] In some embodiments, the processing module is configured to acquire time-domain features, frequency-domain features, and time-frequency-domain features corresponding to the triaxial acceleration data of the vibration acceleration data, and to obtain tactile data features based on at least one of the time-domain features, frequency-domain features, or time-frequency-domain features.

[0142] For example, the triaxial acceleration data includes: lateral acceleration data, vertical acceleration data, and longitudinal acceleration data.

[0143] In some embodiments, the processing module is configured to acquire the time-domain features corresponding to the acceleration data of each of the three axes; or, perform a Fourier transform on the time-domain features to obtain the frequency-domain features corresponding to the acceleration data of the axis; or, perform a continuous wavelet transform on the time-domain features to extract wavelet transform coefficients, and obtain the time-frequency domain features corresponding to the acceleration data of the axis based on the wavelet transform coefficients of each layer.

[0144] In some embodiments, the time-domain features include at least one of the following: mean, variance, standard deviation, maximum value, minimum value, and peak-to-peak value.

[0145] The frequency domain features include at least one of the following: average frequency, maximum frequency, energy spectrum, and center frequency.

[0146] In some embodiments, the processing module is further configured to perform noise reduction processing on the detection data.

[0147] In some embodiments, the raw road surface tactile data is acquired based on a tactile sensor configured to collect the raw road surface tactile data, the tactile sensor being disposed at a connecting frame rigidly connected to the wheel.

[0148] In some embodiments, the processing module is configured to obtain the first recognition result based on the tactile data features and the first recognition model; the first recognition model includes the correspondence between the tactile data features and the road surface type.

[0149] In some embodiments, the processing module is configured to input the data features into a first recognition model to obtain a first recognized road surface and a first confidence level; obtain a first recognition result based on the first recognized road surface and the first confidence level; and use the first recognition result as the recognition result.

[0150] In some embodiments, the first recognition model is trained based on at least one of time-domain features, frequency-domain features, or time-frequency-domain features and the corresponding road surface type as input. The first recognition model is one or more of decision tree, support vector machine, random forest, and multilayer perceptron.

[0151] In some embodiments, the first recognition model is trained and updated based on preset conditions.

[0152] In some embodiments, the processing module is configured to input the data features into a first recognition model to obtain a first recognized road surface and a first confidence level; input the data features into the first recognition model to obtain multiple candidate recognized road surfaces and a candidate confidence level corresponding to each candidate recognized road surface; and, based on the candidate confidence level corresponding to each candidate recognized road surface, determine the candidate recognized road surface corresponding to the highest candidate confidence level as the first recognized road surface, and determine the highest confidence level as the first confidence level.

[0153] In some embodiments, the detection data further includes acoustic data.

[0154] The processing module is configured to obtain the target road surface type based on the acoustic data and the original road surface tactile data.

[0155] In some embodiments, the processing module is configured to obtain acoustic data features based on the acoustic data; perform fusion processing on the tactile data features and the acoustic data features to obtain fused features; obtain the first recognition result based on the fused features; and use the first recognition result as the recognition result.

[0156] In some embodiments, the processing module is further configured to perform dimensionality reduction processing on the fused features.

[0157] In some embodiments, the acoustic data includes tire noise frequency data; correspondingly, the acoustic data features include audio features corresponding to the tire noise frequency data.

[0158] In some embodiments, the processing module is configured to: extract features from the Mel spectrogram corresponding to the tire noise frequency data using a deep residual network to obtain the audio features corresponding to the tire noise frequency data.

[0159] In some embodiments, the processing module is configured to acquire tire noise frequency data; perform pre-emphasis, framing, windowing, short-time Fourier transform, and Mel filtering on the tire noise frequency data to obtain the Mel spectrum corresponding to the tire noise frequency data.

[0160] In some embodiments, the processing module is configured to concatenate the tactile data features and the acoustic data features to obtain fused features; and input the fused features into a multilayer perceptron to obtain the first recognition result.

[0161] For example, the first identification result includes: a first identified road surface and a first confidence level; the multilayer perceptron includes: the correspondence between the fused features and the road surface type.

[0162] In some embodiments, the detection data further includes: road surface visual data.

[0163] In some embodiments, the processing module is further configured to obtain a second recognition result based on the road surface visual data; and to obtain the target road surface type based on the first recognition result and the second recognition result.

[0164] In some embodiments, the processing module is configured to perform frame-segmentation processing on the road surface visual data to obtain multiple image frames corresponding to the road surface visual data; obtain candidate road surface type recognition results corresponding to the multiple image frames respectively; and perform decision fusion on the candidate road surface type recognition results corresponding to the multiple image frames respectively to obtain the second recognition result.

[0165] In some embodiments, the processing module is configured to perform a voting decision on the candidate road surface type recognition results corresponding to the plurality of image frames respectively, and determine the candidate road surface type with the highest frequency of occurrence as the second recognition result.

[0166] In some embodiments, the second identification result includes a second identified road surface and a second confidence level.

[0167] The processing module is configured to determine at least one of the first identified road surface or the second identified road surface as the target road surface type if the first identified road surface is the same as the second identified road surface; and to determine the road surface type with higher confidence between the first confidence level and the second confidence level as the target road surface type if the first identified road surface is different from the second identified road surface.

[0168] In some embodiments, the candidate road surface type identification result includes: the candidate road surface type and the candidate confidence level corresponding to the candidate road surface type.

[0169] The processing module is configured to determine a second identified road surface based on the candidate road surface type with the highest frequency of occurrence; determine a second confidence level based on the candidate confidence level corresponding to the candidate road surface type with the highest frequency of occurrence; and obtain a second identification result based on the second identified road surface and the second confidence level.

[0170] In some embodiments, the processing module is configured to determine the mean of the candidate confidence scores corresponding to the candidate road surface type with the highest frequency of occurrence as the second confidence score; or, determine the maximum value of the candidate confidence scores corresponding to the candidate road surface type with the highest frequency of occurrence as the second confidence score; or, determine the minimum value of the candidate confidence scores corresponding to the candidate road surface type with the highest frequency of occurrence as the second confidence score.

[0171] In some embodiments, the apparatus can be used to execute the technical solutions of the above-described method embodiments, and its implementation principle and technical effect are similar, so they will not be repeated here.

[0172] This disclosure also provides a road surface recognition system, as shown in FIG11. The road surface recognition system 1100 includes: a wheel 1103, a sensor, and a controller.

[0173] The wheel is rigidly connected to the sensor, which is configured to collect detection data. The controller is communicatively connected to the sensor and configured to obtain the target road surface type based on the detection data. The detection data includes at least: raw road surface tactile data, which is road surface tactile data that has not been absorbed by the shock absorption device.

[0174] In some embodiments, the road surface recognition system 1100 may further include: a connecting frame and a shock-absorbing device, the connecting frame being located between the shock-absorbing device and the wheel, the connecting frame being rigidly connected to the wheel, and the sensor being disposed on the connecting frame.

[0175] As shown in Figure 11, the sensor can be installed on the lower control arm 1101 of the connecting frame, which is located between the shock absorption device 1102 and the wheel 1103.

[0176] In some embodiments, referring to FIG11, the sensor may also be directly mounted on the wheel 1103.

[0177] In some embodiments, the road surface recognition system 1100 may further include: a subframe, the subframe being rigidly connected to the wheel, and the sensor being disposed on the subframe. As shown in FIG12, the sensor may be disposed on the subframe 1201.

[0178] In some embodiments, the road surface recognition system 1100 may further include: a steering tie rod rigidly connected to the wheel, and the sensor disposed on the steering tie rod. Referring to FIG12, the sensor may also be disposed on the steering tie rod 1202.

[0179] In some embodiments, the sensor includes a tactile sensor configured to acquire raw road surface tactile data.

[0180] In some embodiments, the sensor further includes at least one of an acoustic sensor or a camera.

[0181] In some embodiments, the sensors and controllers of the road surface recognition system are used to execute the technical solutions of the above-described method embodiments. Their implementation principles and technical effects are similar, and will not be repeated here.

[0182] This disclosure also provides a vehicle for use in a road surface recognition system, as shown in Figure 13. The vehicle 1000 is communicatively connected to a cloud server 2000. Under preset conditions, the vehicle 1000 executes the road surface type recognition methods described above.

[0183] For example, the preset conditions include: determining that the cloud server is faulty, or that the vehicle is offline.

[0184] This disclosure also provides a cloud server in some embodiments, which is applied to a road surface recognition system and is used to implement the above-described road surface type recognition methods.

[0185] This disclosure also provides an electronic device, as shown in FIG14. The electronic device 1400 includes: a memory 1401, a processor 1402, and a computer program stored in the memory. The processor 1402 executes the computer program to implement the above-described road surface type identification methods.

[0186] For example, the electronic device may be deployed in a vehicle or a cloud server, or the electronic device may be deployed in both a vehicle and a server.

[0187] Some embodiments of this disclosure also provide a computer-readable storage medium storing a program or instructions that, when executed by a processor, implement the above-described methods for identifying road surface types.

[0188] Some embodiments of this disclosure also provide a computer program product, which includes a computer program or instructions, wherein when the computer program or instructions are executed by a processor, the above-described methods for identifying road surface types are implemented.

[0189] It should be noted that the aforementioned computer-readable medium may be a computer-readable signal medium or a computer-readable storage medium or any combination thereof.

[0190] Computer-readable storage media may be, for example,—but not limited to—an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof.

[0191] More examples of computer-readable storage media may include, but are not limited to: electrical connections having one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0192] In some embodiments of this disclosure, the computer-readable storage medium can be any tangible medium that contains or stores a program that can be used or combined with an instruction execution system, apparatus, or device. In some embodiments of this disclosure, the computer-readable signal medium can include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof.

[0193] A computer-readable signal medium may be any computer-readable medium other than a computer-readable storage medium, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to: wires, optical fibers, radio frequency (RF), etc., or any suitable combination thereof.

[0194] The aforementioned computer-readable medium may be included in the aforementioned electronic device; or it may exist independently and not assembled into the electronic device.

[0195] The aforementioned computer-readable medium carries one or more programs that, when executed by the electronic device, cause the electronic device to: obtain the target road surface type based on the detection data.

[0196] The detection data includes at least: raw road surface tactile data, which is road surface tactile data that has not been absorbed by the shock absorption device.

[0197] Computer program code for performing the operations of some embodiments of this disclosure can be written in one or more programming languages ​​or a combination thereof, including but not limited to: object-oriented programming languages ​​such as Java, Smalltalk, and C++, as well as conventional procedural programming languages ​​such as the "C" language or similar programming languages.

[0198] The program code can execute entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or it can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0199] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function.

[0200] It should also be noted that in some alternative implementations, the functions marked in the box may occur in a different order than those marked in the attached figures.

[0201] For example, two consecutively represented blocks can actually be executed in substantially parallel order, and sometimes they can be executed in reverse order, depending on the functions involved. It should also be noted that each block in a block diagram and / or flowchart, as well as combinations of blocks in block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified functions or operations, or using a combination of dedicated hardware and computer instructions.

[0202] The units described in some embodiments of this disclosure can be implemented in software or hardware. The names of the units do not necessarily limit the unit itself.

[0203] The functions described above in this document can be performed, at least in part, by one or more hardware logic components. For example, exemplary types of hardware logic components that can be used, without limitation, include: Field Programmable Gate Arrays (FPGAs), Application-Specific Integrated Circuits (ASICs), Application Standard Products (ASSPs), System-on-Chip (SoCs), Complex Programmable Logic Devices (CPLDs), and so on.

[0204] The embodiments of this disclosure have been described above with reference to the accompanying drawings. However, this disclosure is not limited to the specific embodiments described above. The specific embodiments described above are merely illustrative and not restrictive. Those skilled in the art can make many other implementations under the guidance of this disclosure without departing from the spirit and scope of the claims. All of these implementations are within the protection scope of this disclosure.

Claims

1. A method for identifying a road surface type, comprising: obtaining a target road surface type based on detection data; wherein the detection data at least comprises original road surface haptic data, the original road surface haptic data being road surface haptic data not absorbed by a shock absorption device.

2. The method of claim 1, wherein, The obtaining of the target road surface type based on the detection data comprises: obtaining an identification result based on the detection data; and obtaining the target road surface type based on the identification result.

3. The method of claim 2, wherein, The obtaining of the identification result based on the detection data comprises: obtaining data features based on the detection data; and obtaining the identification result based on the data features.

4. The method of claim 3, wherein, The obtaining of the data features based on the detection data comprises: performing feature extraction on the original road surface haptic data to obtain haptic data features corresponding to the original road surface haptic data; The obtaining of the identification result based on the data features comprises: obtaining a first identification result based on the haptic data features; and taking the first identification result as the identification result.

5. The method of claim 4, wherein, The original road surface haptic data comprises vibration acceleration data. The data features comprise vibration acceleration features corresponding to the vibration acceleration data.

6. The method of claim 5, wherein, The performing of feature extraction on the original road surface haptic data to obtain haptic data features corresponding to the original road surface haptic data comprises: obtaining time domain features, frequency domain features and time-frequency domain features corresponding to three-axis acceleration data of the vibration acceleration data respectively, and obtaining the haptic data features based on at least one of the time domain features, the frequency domain features or the time-frequency domain features; wherein the three-axis acceleration data comprises lateral acceleration data, vertical acceleration data and longitudinal acceleration data.

7. The method of claim 6, wherein, The obtaining of the time domain features, the frequency domain features and the time-frequency domain features corresponding to the three-axis acceleration data of the vibration acceleration data comprises at least one of: for each axis of the three axes, obtaining time domain features corresponding to acceleration data of the each axis; performing Fourier transform on the time domain features to obtain frequency domain features corresponding to acceleration data of any one axis of the three axes; or performing continuous wavelet transform on the time domain features to extract wavelet transform coefficients, and obtaining time-frequency domain features corresponding to acceleration data of the any one axis based on the wavelet transform coefficients of each layer.

8. The method of claim 7, wherein, The time domain features comprise at least one of: mean value, variance, standard deviation, maximum value, minimum value and peak-to-peak value. The frequency domain features comprise at least one of: average frequency, maximum frequency, energy spectrum and center frequency.

9. The method of any one of claims 1-8, wherein, Before the obtaining of the target road surface type based on the detection data, the method further comprises denoising the detection data.

10. The method of any one of claims 1-8, wherein, The original road surface haptic data is obtained based on a haptic sensor configured to collect the original road surface haptic data, and the haptic sensor is arranged at a connecting frame rigidly connected with a wheel.

11. The method of any one of claims 7-10, wherein, The obtaining of the first identification result based on the haptic data features comprises: obtaining the first identification result based on the haptic data features and a first identification model, wherein the first identification model comprises a corresponding relationship between the haptic data features and road surface types.

12. The method of any one of claims 3-7, wherein, The obtaining the identification result based on the data feature comprises: inputting the data feature into a first identification model to obtain a first identified road surface and a first confidence level; obtaining the first identification result based on the first identified road surface and the first confidence level; and taking the first identification result as the identification result.

13. The method of claim 12, wherein, The first identification model is trained based on at least one of time domain features, frequency domain features or time-frequency domain features, and corresponding road surface types as input, and the first identification model is at least one of a decision tree, a support vector machine, a random forest or a multi-layer perceptron.

14. The method of claim 12, further comprising: The first identification model is trained and updated based on a preset condition.

15. The method of claim 12, wherein, The inputting the data feature into a first identification model to obtain a first identified road surface and a first confidence level comprises: inputting the data feature into the first identification model to obtain a plurality of candidate identified road surfaces and a candidate confidence level corresponding to each candidate identified road surface in the plurality of candidate identified road surfaces; and based on the candidate confidence level corresponding to each candidate identified road surface, determining a candidate identified road surface corresponding to a highest candidate confidence level as the first identified road surface, and determining the highest candidate confidence level as the first confidence level.

16. The method of any one of claims 4-7, wherein, The detection data further comprises acoustic data. The obtaining the target road surface type based on the detection data further comprises: obtaining the target road surface type based on the acoustic data and the original road surface haptic data.

17. The method of claim 16, wherein, The obtaining the target road surface type based on the acoustic data and the original road surface haptic data comprises: obtaining acoustic data features based on the acoustic data; performing fusion processing on the haptic data features and the acoustic data features to obtain fusion features; obtaining the first identification result based on the fusion features; and taking the first identification result as the identification result.

18. The method of claim 17, wherein, After the fusion processing on the haptic data features and the acoustic data features to obtain fusion features, the method further comprises: performing dimension reduction processing on the fusion features.

19. The method of claim 17, wherein, The acoustic data comprises tire noise audio data, and the acoustic data features comprise audio features corresponding to the tire noise audio data.

20. The method of claim 19, wherein, The obtaining acoustic data features based on the acoustic data comprises: extracting features of a mel-frequency spectrogram corresponding to the tire noise audio data by a deep residual network to obtain the audio features corresponding to the tire noise audio data.

21. The method of claim 19, wherein, The method of obtaining the mel-frequency spectrogram corresponding to the tire noise audio data comprises: obtaining the tire noise audio data; and performing pre-emphasis, framing, windowing, short-time Fourier transform and mel-filtering processing on the tire noise audio data to obtain the mel-frequency spectrogram corresponding to the tire noise audio data.

22. The method of any one of claims 17-21, wherein, The obtaining the first identification result based on the fusion features comprises: performing splicing processing on the haptic data features and the acoustic data features to obtain the fusion features; and inputting the fusion features into a multi-layer perceptron to obtain the first identification result; wherein the first identification result comprises a first identified road surface and a first confidence level; and the multi-layer perceptron comprises a corresponding relationship between the fusion features and road surface types.

23. The method of any one of claims 16-22, wherein, The detection data further includes road surface visual data; The method further includes: obtaining a second identification result based on the road surface visual data; and obtaining the target road surface type based on the first identification result and the second identification result.

24. The method of claim 23, wherein, The obtaining of the second identification result based on the road surface visual data includes: performing frame processing on the road surface visual data to obtain a plurality of image frames corresponding to the road surface visual data; obtaining candidate road surface type identification results corresponding to the plurality of image frames respectively; and performing decision fusion on the candidate road surface type identification results corresponding to the plurality of image frames respectively to obtain the second identification result.

25. The method of claim 24, wherein, The performing of the decision fusion on the candidate road surface type identification results corresponding to the plurality of image frames respectively to obtain the second identification result includes: performing voting decision on the candidate road surface type identification results corresponding to the plurality of image frames respectively, and determining a candidate road surface type with the highest frequency of occurrence as the second identification result.

26. The method of claim 24, wherein, The second identification result includes a second identified road surface and a second confidence degree. The obtaining of the target road surface type based on the first identification result and the second identification result includes: if the first identified road surface is the same as the second identified road surface, determining at least one of the first identified road surface or the second identified road surface as the target road surface type; and if the first identified road surface is different from the second identified road surface, determining a road surface type with a higher confidence degree between the first confidence degree and the second confidence degree as the target road surface type. The candidate road surface type identification result includes the candidate road surface type and a candidate confidence degree corresponding to the candidate road surface type.

27. The method of claim 26, wherein, The performing of the decision fusion on the candidate road surface type identification results corresponding to the plurality of image frames respectively to obtain the second identification result includes: determining the second identified road surface based on the candidate road surface type with the highest frequency of occurrence; and determining the second confidence degree based on a candidate confidence degree corresponding to the candidate road surface type with the highest frequency of occurrence, and obtaining the second identification result according to the second identified road surface and the second confidence degree. The determining of the second confidence degree based on the candidate confidence degree corresponding to the candidate road surface type with the highest frequency of occurrence includes one of:

28. The method of claim 27, wherein, determining a mean value of the candidate confidence degree corresponding to the candidate road surface type with the highest frequency of occurrence as the second confidence degree; determining a maximum value of the candidate confidence degree corresponding to the candidate road surface type with the highest frequency of occurrence as the second confidence degree; or determining a minimum value of the candidate confidence degree corresponding to the candidate road surface type with the highest frequency of occurrence as the second confidence degree. The system includes: a wheel; 29. A road surface recognition system wherein, a sensor rigidly connected with the wheel, the sensor being configured to collect detection data; and a controller in communication connection with the sensor, the controller being configured to obtain a target road surface type based on the detection data; wherein the detection data at least includes original road surface tactile data, the original road surface tactile data being road surface tactile data not absorbed by a damping device.

30. The system of claim 29, further comprising: ​ ​ a connecting frame rigidly connected with the wheel, the sensor being arranged on the connecting frame; and a damping device, the connecting frame being between the damping device and the wheel.

31. The system of claim 29, further comprising: a subframe rigidly connected with the wheel, the sensor being arranged on the subframe.

32. The system of claim 29, further comprising: a steering drag link rigidly connected with the wheel, the sensor being arranged on the steering drag link.

33. The system of any one of claims 29-32, wherein, The sensor comprises a tactile sensor configured to collect original road surface tactile data.

34. The system of any one of claims 29-33, wherein, The sensor comprises at least one of an acoustic sensor or a camera.

35. An electronic device comprising: a memory, a processor, and a computer program stored on the memory, the processor executing the computer program to implement the road surface type identification method according to any one of claims 1-28.

36. A computer readable storage medium storing a program or instructions, the program or the instructions being executed by a processor to implement the road surface type identification method according to any one of claims 1-28.

37. A computer program product, comprising: A computer program or instructions, the computer program or the instructions being executed by a processor to implement the road surface type identification method according to any one of claims 1-28. 38.A vehicle applied to a road surface identification system, wherein the vehicle is in communication connection with a cloud server, and the vehicle executes the road surface type identification method according to any one of claims 1-28 under a preset condition. The preset condition comprises one of determining that the cloud server is faulty or the vehicle is in an off-network state.

Images