Method for estimating road friction coefficient based on intelligent tire and related device

Abstract

The present disclosure provides a kind of based on intelligent tire road surface friction coefficient estimation method and related equipment, it is related to intelligent automobile state estimation and control technical field, especially in the field of road surface friction coefficient estimation, the method comprises: with the rotation of intelligent tire as a cycle, according to the preset angle as sampling interval is sampled, the acceleration signal of multiple cycles of intelligent tire is obtained, including longitudinal, lateral, vertical acceleration signal, acceleration signal is carried out spatial short-time Fourier transform, the transformation result of acceleration signal in angular space dimension in three directions is obtained, the transformation result is input to convolutional neural network, the road surface type and corresponding confidence are obtained, according to road surface type, corresponding confidence and road surface type friction coefficient relationship table, the first friction coefficient of road surface is determined.Not susceptible to external environmental factors, not dependent on speed information and larger excitation can be completed estimation, there is no state coupling problem.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to the technical field of intelligent vehicle state estimation and control, in particular to the field of road surface friction coefficient estimation, and especially to a road surface friction coefficient estimation method based on intelligent tires and related equipment. BACKGROUND

[0002] With the continuous development of vehicle automatic driving technology to higher level automation and the gradual expansion of vehicle performance boundaries, in various driving scenarios, especially in emergency working conditions, it has become an important challenge to achieve robust vehicle safety control. In the field of vehicle motion control, accurate and timely estimation of the road surface friction coefficient is crucial, and high-level chassis control systems need this information to improve vehicle handling stability control performance.

[0003] In related technologies, the way to estimate the friction coefficient is to first obtain the input of the tire (such as the normal load and the longitudinal and lateral slip state) and the response of the tire (tire force and torque), and then identify the friction coefficient parameters in the tire model through a certain method. Since the road surface friction coefficient mainly affects the nonlinear region of the tire force, such methods usually require the tire to be subjected to a large longitudinal or lateral excitation to estimate the friction coefficient. However, sufficient excitation of the tire usually means that the vehicle is approaching the edge of stability, and in comparison, it is more ideal and safe to obtain the friction coefficient under the condition that the tire is only subjected to small or near-zero excitation.

[0004] In addition, the calculation of the tire slip state in the input of such a method requires vehicle speed information, and generally, model-based vehicle speed estimation also requires friction coefficient information, so the above-mentioned method of estimating the friction coefficient also has a state coupling problem.

[0005] In addition to the above mainstream dynamics method, there is also a road surface classification method based on vehicle-mounted cameras, which does not require a large degree of tire excitation, but is easily affected by environmental factors such as rain, snow, and light conditions.

[0006] Therefore, in related technologies, the estimation of the friction coefficient has the following problems: not only does it require vehicle speed information and a large longitudinal or lateral excitation, but it also has a state coupling problem and is easily affected by environmental factors.

[0007] It should be noted that the information disclosed in the above background section is only used to strengthen the understanding of the background of the present disclosure, and therefore can include information that does not constitute prior art known to those of ordinary skill in the art. SUMMARY

[0008] The present disclosure provides a method for estimating road surface friction coefficient based on intelligent tire and related equipment, which at least partially overcomes the problems in the prior art that vehicle speed information and large longitudinal or lateral excitation are required when estimating the friction coefficient, the state coupling problem exists, and the method is easily affected by environmental factors.

[0009] Other characteristics and advantages of the present disclosure will become apparent from the following detailed description, or will be learned by practice of the present disclosure.

[0010] In a first aspect, the embodiments in the present disclosure provide a method for estimating road surface friction coefficient based on intelligent tire, the method comprising:

[0011] Taking one rotation of the intelligent tire as a period, sampling according to a preset angle as a sampling interval to obtain acceleration signals of the intelligent tire in multiple periods; the acceleration signals include longitudinal acceleration signals, lateral acceleration signals and vertical acceleration signals;

[0012] Performing spatial short-time Fourier transform on the acceleration signals to obtain transformation results of the acceleration signals in three directions in the angular space dimension;

[0013] Inputting the transformation results into a convolutional neural network to obtain a road surface type and a corresponding confidence;

[0014] Determining a first friction coefficient of the road surface according to the road surface type, the corresponding confidence and a road surface type friction coefficient relationship table.

[0015] In a possible embodiment, the method further comprises:

[0016] Processing the first friction coefficient and a first confidence corresponding to the first friction coefficient by adaptive Kalman filtering to determine a second friction coefficient.

[0017] In a possible embodiment, the adaptive Kalman filtering comprises a prediction update process and a measurement update process;

[0018] Processing the first friction coefficient and a first confidence corresponding to the first friction coefficient by adaptive Kalman filtering to determine a second friction coefficient, comprising:

[0019] In the prediction update process, a friction coefficient prediction value of a current time and an error covariance prediction value of the current time are determined;

[0020] In the measurement update process, a Kalman gain is determined according to the error covariance prediction value of the current time and the first confidence;

[0021] A second friction coefficient of the current time is determined according to the first friction coefficient, the Kalman gain and the friction coefficient prediction value of the current time.

[0022] In a possible embodiment, in the measurement update process, the Kalman gain is determined according to the error covariance prediction value of the current time and the first confidence level, comprising:

[0023] The measurement covariance is determined according to the first confidence level; the larger the first confidence level is, the smaller the measurement covariance is;

[0024] The Kalman gain is determined according to the measurement covariance and the error covariance prediction value of the current time.

[0025] In a possible embodiment, the error covariance prediction value is determined, comprising:

[0026] The error covariance update value of the measurement update process of the last adaptive Kalman filtering is obtained; the error covariance update value is determined according to the Kalman gain of the last time and the error covariance prediction value in the prediction update process of the last adaptive Kalman filtering;

[0027] The error covariance prediction value of the current time is determined according to the error covariance update value.

[0028] In a possible embodiment, the method further comprises:

[0029] In the process of processing the first friction coefficient and the first confidence level corresponding to the first friction coefficient by the adaptive Kalman filtering, it is judged whether there is an effective first friction coefficient in the current period;

[0030] If there is, the measurement update process is performed;

[0031] If there is not, the measurement update process is skipped, the friction coefficient prediction value of the current time is taken as the second friction coefficient of the current time, and the error covariance prediction value of the current time is taken as the error covariance update value of the current time.

[0032] In a second aspect, the embodiments in the present disclosure provide a road surface friction coefficient estimation device based on an intelligent tire, comprising:

[0033] A sampling unit is configured to sample according to a preset angle as a sampling interval to obtain acceleration signals of the intelligent tire in multiple periods, with one rotation of the intelligent tire as one period; the acceleration signals comprise longitudinal acceleration signals, lateral acceleration signals and vertical acceleration signals;

[0034] A Fourier transform unit is configured to perform spatial short-time Fourier transform on the acceleration signals to obtain transformation results of the acceleration signals in three directions in the angular space dimension;

[0035] An estimation unit is configured to input the transformation results into a convolutional neural network to obtain a road surface type and a corresponding confidence level;

[0036] The determination unit is configured to determine a first friction coefficient of the road surface according to the road surface type, the corresponding confidence degree, and a road surface type friction coefficient relationship table.

[0037] In a possible embodiment, the method further includes filtering the first friction coefficient and the first confidence degree corresponding to the first friction coefficient by adaptive Kalman filtering to determine a second friction coefficient.

[0038] In a third aspect, an electronic device is provided, including a processor and a memory storing executable instructions of the processor, wherein the processor is configured to execute the method of the first aspect by executing the executable instructions.

[0039] In a fourth aspect, a computer readable storage medium is provided, which stores a computer program. The computer program is executed by a processor to implement the method of the first aspect.

[0040] In a fifth aspect, according to another aspect of the present disclosure, a computer program product or a computer program is also provided, which includes computer instructions stored in a computer readable storage medium. A processor of a computer device reads the computer instructions from the computer readable storage medium, and the processor executes the computer instructions to enable the computer device to execute the method of any one of the aspects.

[0041] The method and related device for estimating road surface friction coefficient based on intelligent tire provided by the embodiments of the present disclosure relate to the technical field of intelligent vehicle state estimation and control, and in particular to the field of road surface friction coefficient estimation. The method includes: taking one rotation of the intelligent tire as a period, sampling at a preset angle as a sampling interval to obtain acceleration signals of the intelligent tire in multiple periods, the acceleration signals including longitudinal acceleration signals, lateral acceleration signals, and vertical acceleration signals; performing spatial short-time Fourier transform on the acceleration signals to obtain transformation results of the acceleration signals in three directions in the angle space dimension; inputting the transformation results into a convolutional neural network to obtain a road surface type and a corresponding confidence degree; and determining a first friction coefficient of the road surface according to the road surface type, the corresponding confidence degree, and a road surface type friction coefficient relationship table. The method is not easily affected by external environmental factors, does not depend on vehicle speed information and a large excitation to complete the estimation, and does not have the problem of state coupling. The three-direction acceleration information is fused in the model to provide rich features, so that the model realizes feature complementation between different road surface types. The acceleration signals are directly sampled at the preset angle to adapt to the requirement of consistency of the size of the input of the convolutional neural network, and the spatial short-time Fourier transform processing is performed to obtain more intuitive transformation results in the time-space dimension, thereby improving the prediction accuracy.

[0042] It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the disclosure. BRIEF DESCRIPTION OF DRAWINGS

[0043] The accompanying drawings, which are incorporated in and form a part of the specification, illustrate embodiments consistent with the present disclosure and, together with the description, serve to explain the principles of the disclosure. It is to be understood that the drawings are only schematic, and that they do not necessarily represent a limiting

[0044] Figure 1 A flow chart of a method for estimating road surface friction coefficient based on intelligent tire is shown in the embodiments of the present disclosure;

[0045] Figure 2 A schematic diagram of a transformed result is shown in the embodiments of the present disclosure;

[0046] Figure 3 A flow chart of another method for estimating road surface friction coefficient based on intelligent tire is shown in the embodiments of the present disclosure;

[0047] Figure 4 A schematic diagram of a process for estimating road surface friction coefficient is shown in the embodiments of the present disclosure,

[0048] Figure 5 A flow chart of a process for adaptive Kalman filtering is shown in the embodiments of the present disclosure;

[0049] Figure 6 A structural schematic diagram of a convolutional neural network is shown in the embodiments of the present disclosure;

[0050] Figure 7 A schematic diagram of cross-validation is shown in the embodiments of the present disclosure;

[0051] Figure 8 A flow chart of a process for adaptive Kalman filtering is shown in the embodiments of the present disclosure;

[0052] Figure 9 A schematic diagram of an evaluation result is shown in the embodiments of the present disclosure;

[0053] Figure 10 A schematic diagram of another evaluation result is shown in the embodiments of the present disclosure;

[0054] Figure 11 A structural schematic diagram of a device for estimating road surface friction coefficient based on intelligent tire is shown in the embodiments of the present disclosure;

[0055] Figure 12 A structural schematic diagram of an electronic device is shown in the embodiments of the present disclosure. DETAILED DESCRIPTION

[0056] Example implementations are now described in greater detail with reference to the figures. Like reference numerals can be used to refer to like elements throughout. The example implementations can be implemented in numerous ways, and are not limited to the examples described herein; instead, examples are provided so that this disclosure will be thorough and complete, and will fully convey the concept of the example implementations to those skilled in the art. The described features, structures, or characteristics can be combined in any suitable manner in one or more implementations.

[0057] In addition, the drawings are only schematic and are non-limiting. Identical elements are denoted by the same reference numerals throughout the figures. Particular features, structures, or characteristics can be universal in nature, or specific to a particular example implementation. Each of the references numerals can represent a single or plurality feature or combination of features.

[0058] The example implementations will now be described in detail with reference to the accompanying drawings and examples.

[0059] First, a road surface friction coefficient estimation method is provided in the embodiments of the present disclosure, which can be executed by any electronic device with computing processing capability. The following process takes the electronic device as an example of a terminal device.

[0060] Figure 1 A flowchart of a road surface friction coefficient estimation method based on intelligent tires is shown in the embodiments of the present disclosure, as shown in FIG. 1, the road surface friction coefficient estimation method based on intelligent tires provided in the embodiments of the present disclosure includes the following steps: Figure 1

[0061] S102: Take one rotation of the intelligent tire as a period, and sample at a preset angle as a sampling interval to obtain acceleration signals of the intelligent tire in multiple periods, the acceleration signals including longitudinal acceleration signals, lateral acceleration signals, and vertical acceleration signals.

[0062] In a possible embodiment, an acceleration sensor can be arranged in the intelligent tire, and the acceleration sensor samples at a preset angle as a sampling interval to obtain acceleration signals of the intelligent tire in multiple periods, taking one rotation period (360°) of each intelligent tire as one processing period of the acceleration signals. The acceleration signals obtained by the intelligent tire system sampling are data information in angular space dimensions.

[0063] The preset angle can be 1°.

[0064] ​S104: Perform spatial short-time Fourier transform on the acceleration signal to obtain a transform result of the acceleration signal in three directions in the angular space dimension.

[0065] In a possible embodiment, the acceleration signal sampled in a processing period is subjected to spatial short-time Fourier transform.

[0066] In a possible embodiment, in order to improve the accuracy of the friction coefficient output by the convolutional neural network (CNN) and improve the prediction speed of the convolutional neural network, the input features can be better used when the convolutional neural network is estimating. In the embodiment of the present disclosure, the acceleration signal in the angular space dimension is processed by using spatial short-time Fourier transform to obtain a transform result of the acceleration signal in three directions in the angular space dimension.

[0067] It should be noted that the spatial short-time Fourier transform in the embodiment of the present disclosure is based on the input in the angular space dimension and performs short-time Fourier transform (STFT) in the angular space dimension.

[0068] The three directions are respectively the longitudinal direction, the lateral direction and the vertical direction in the tire inner sensor coordinate system.

[0069] The spatial short-time Fourier transform can overcome the problem that the conventional Fourier transform cannot locate the frequency component changing in time. In the embodiment of the present disclosure, the input signal is the acceleration signal in the angular space dimension, and based on the spatial short-time Fourier transform, the transform result can represent the signal intensity under different angular space positions and angular space frequencies.

[0070] Based on this, when the transform result is input to the convolutional neural network, the convolutional neural network can be provided with more abundant information, and the classification accuracy of the convolutional neural network can be further improved.

[0071] The angular space frequency represents the number of spatial variation periods contained in a unit angle range.

[0072] In the embodiment of the present disclosure, the parameters of the spatial short-time Fourier transform for processing the acceleration signal can be set in multiple ways, and only two kinds are exemplarily given, as follows.

[0073] The first kind: the acceleration sampling interval can be set to 1 degree, the window length used by the spatial short-time Fourier transform can be 64, the window function used can be a Hamming window, the overlap rate is 75% of the window length, and the number of points used by the FFT is twice the window length.

[0074] The second kind: the acceleration sampling interval can be set to 0.1 degrees, the Hamming window is used as the window function, the window length is 512, the overlap rate is 75%, and the number of points of the FFT in the spatial short-time Fourier transform is twice the window length.

[0075] Both of the above two methods can be used, the second kind presents clearer graphs than the first kind, and the first kind can reduce the computational power consumption. In this scheme, the first kind of spatial short-time Fourier transform is selected to process the acceleration signal, and the transform result is input into the convolutional neural network to complete the prediction.

[0076] Figure 2 A schematic diagram of a transform result in the embodiment of the present disclosure is shown, as shown in Figure 2 The coordinates are: angular position (°) and angular spatial frequency (cycle / °), and 1080° is taken as an example of three cycles of sampling results. Taking ice as an example of road surface type, Figure 2 200-1400 represent intensity, including: 200, 400, 600, 800, 1000, 1200 and 1400, respectively, and the intensity is represented by a gray-based color gradient.

[0077] The horizontal axis represents the angular position, and the horizontal coordinates respectively show 200, 400, 600, 800 and 1000. The vertical axis represents the angular spatial frequency, and the vertical coordinates show 10 -2 , 10 -1 , 10 0 .

[0078] S106: input the transform result into the convolutional neural network to obtain the road surface type and the corresponding confidence.

[0079] In one possible embodiment, the output of the convolutional neural network is different road surface types and the confidence corresponding to different road surface types.

[0080] The road surface type can include many types, such as fine asphalt, coarse asphalt, paving bricks, cement, wet ceramic tiles, wet basalt, snow, and ice, etc.

[0081] Through the maximum confidence, the final road surface type output by the convolutional neural network in this estimation process can be determined.

[0082] S108: determine the first friction coefficient of the road surface according to the road surface type, the corresponding confidence, and the road surface type friction coefficient relationship table.

[0083] In one possible embodiment, the road surface type friction coefficient relationship table includes the association between the road surface type and the friction coefficient, for example: fine asphalt 1.0, coarse asphalt 1.0, paving bricks 0.6, cement 0.7, wet ceramic tiles 0.1, wet basalt 0.4, compacted snow 0.3, and ice road surface 0.1.

[0084] In a possible embodiment, a final road surface type of the convolutional neural network output is determined by the maximum confidence, and a first friction coefficient is determined based on the final road surface type and a road surface type friction coefficient relationship table.

[0085] The above method in the embodiments of the present disclosure can achieve the following technical effects:

[0086] (1) Not susceptible to external environmental factors, and can be completed without relying on vehicle speed information and large excitation, without state coupling problems, and the problems existing in the related art are solved.

[0087] (2) A variety of machine learning methods are used for experiments, and the convolutional neural network is most suitable for the three-direction acceleration signals collected in the embodiments of the present disclosure. The three-direction transformation results can be spliced in the spatial dimension after being input into the convolutional neural network, forming an input tensor, so that the model can perceive more rich features and have higher precision.

[0088] (3) Based on the convolutional neural network, because it needs to input consistent size, when sampling, if sampling is performed according to time, for example, sampling is performed according to 1 second or several seconds, in the case that the vehicle speed is unstable, effective sampling features cannot be obtained, and the size consistency requirement of input cannot be met. The convolutional neural network cannot use the features sampled according to time. Based on this consideration, the inventors think of sampling according to a fixed preset angle, so that acceleration signals meeting the network input requirement can be obtained, and the estimation process of the convolutional neural network is adapted.

[0089] (4) After directly sampling the features in the angular space dimension, the conventional Fourier transform cannot meet the specific data analysis process. Based on this consideration, the inventors use the angular space dimension acceleration signal to perform a spatial short-time Fourier transform, so that the transformation result can be obtained, which is used to represent the distribution intensity of the sampling acceleration signal in the angular space frequency as the angular position changes.

[0090] Figure 3 Another flowchart of a road surface friction coefficient estimation method based on an intelligent tire in the embodiments of the present disclosure is shown, as shown in Figure 3 , the method comprises:

[0091] S302: Take one rotation of the intelligent tire as a period, sample according to a preset angle as a sampling interval, and obtain acceleration signals of the intelligent tire in multiple periods. The acceleration signals include a longitudinal acceleration signal, a lateral acceleration signal, and a vertical acceleration signal.

[0092] S304: Perform spatial short-time Fourier transform on the acceleration signal to obtain a transform result of the acceleration signal in angular space dimension in three directions.

[0093] S306: Input the transform result into a convolutional neural network to obtain a road surface type and a corresponding confidence.

[0094] S308: Determine a first friction coefficient of the road surface and a corresponding first confidence according to the road surface type, the corresponding confidence, and a road surface type friction coefficient relationship table.

[0095] S310: Process the first friction coefficient and the first confidence corresponding to the first friction coefficient by adaptive Kalman filtering to determine a second friction coefficient.

[0096] In a possible embodiment, the first friction coefficient is further subjected to friction coefficient estimation by adaptive Kalman filtering (AKF), and the process update of the adaptive Kalman filtering can adopt a random walk model, and the first friction coefficient is adaptively adjusted based on the first confidence and the adaptive Kalman filtering.

[0097] Figure 4 A schematic diagram of a road surface friction coefficient estimation process in an embodiment of the present disclosure is shown, as shown in Figure 4 The transform result of the acceleration signal is input into a convolutional neural network, a road surface type and a corresponding first confidence are output, a first friction coefficient is determined through a road surface type friction coefficient relationship table, and the first friction coefficient and the corresponding first confidence are input into adaptive Kalman filtering, and a second friction coefficient is output.

[0098] In a possible embodiment, the first friction coefficient and the corresponding first confidence are used in a measurement update process of the adaptive Kalman filtering.

[0099] The adaptive Kalman filtering includes a prediction update process and a measurement update process.

[0100] Figure 5 A flowchart of an adaptive Kalman filtering process in an embodiment of the present disclosure is shown, as shown in Figure 5 The flowchart includes the following steps:

[0101] S502: In the prediction update process, a friction coefficient prediction value of a current time and an error covariance prediction value of the current time are determined.

[0102] In a possible embodiment, the manner of obtaining the friction coefficient prediction value of the current time can include determining the friction coefficient prediction value of the current time according to the second friction coefficient obtained in the measurement update process of the last adaptive Kalman filtering.

[0103] In a possible embodiment, the manner of obtaining the error covariance prediction value of the current time can include: obtaining an error covariance update value of a measurement update process of the last adaptive Kalman filtering; wherein the error covariance update value is determined according to the Kalman gain of the last time and the error covariance prediction value in a prediction update process of the last adaptive Kalman filtering; and determining the error covariance prediction value of the current time according to the error covariance update value.

[0104] It should be noted that the current time, the last time and the like in the embodiments of the present disclosure refer to the number of internal loops in the adaptive Kalman filtering process.

[0105] S504: In the measurement update process, the Kalman gain is determined according to the error covariance prediction value of the current time and the first confidence.

[0106] In a possible embodiment, the Kalman gain can be determined in the following manner: the measurement covariance is determined according to the first confidence; wherein the greater the first confidence, the smaller the measurement covariance; and the Kalman gain is determined according to the measurement covariance and the error covariance prediction value of the current time.

[0107] S506: The second friction coefficient of the current time is determined according to the first friction coefficient, the Kalman gain and the friction coefficient prediction value of the current time.

[0108] Through the above process, the first friction coefficient and the first confidence output by the convolutional neural network are used in the filtering process to perform re-estimation processing on the friction coefficient, and the estimation process of the convolutional neural network is combined with the adaptive Kalman filtering process to jointly estimate the friction coefficient, thereby improving the accuracy of the estimated friction coefficient. The above process is not simply combined, but the friction coefficient is adaptively adjusted according to the confidence, the gain is determined by using the confidence, and different gains are dynamically determined based on the different confidence, so as to adjust the first friction coefficient in real time and output the second friction coefficient.

[0109] In a possible embodiment, the measurement update process of the adaptive Kalman filtering process is mainly a process of processing the first friction coefficient and the first confidence. The results sampled by the intelligent tire system are not generated at a fixed frequency, but are sampled based on a preset angle and depend on the speed of the tire. The faster the speed, the more results are obtained in the same time. However, the results obtained by sampling are usually lower than the sampling rate of the adaptive Kalman filtering. Since the AKF runs at a fixed sampling interval, the sampling results of the intelligent tire system are not available at every moment. Therefore, an update flag is introduced to indicate whether there is a valid first friction coefficient at the current moment.

[0110] Based on this, it can be determined whether there is an effective first friction coefficient between the prediction update process and the measurement update process of the adaptive Kalman filtering process. The method can include the following steps: in the process of processing the first friction coefficient and the first confidence corresponding to the first friction coefficient by the adaptive Kalman filtering, it is determined whether there is an effective first friction coefficient in the current period; if there is, the measurement update process is performed; if there is not, the measurement update process is skipped, the friction coefficient prediction value of the current time is taken as the second friction coefficient of the current time, and the error covariance prediction value of the current time is taken as the error covariance update value of the current time.

[0111] The convolutional neural network in the embodiment of the present disclosure is a trained neural network, Figure 6 A structural schematic diagram of a convolutional neural network in the embodiment of the present disclosure is shown, as shown in the figure, including a first convolutional layer, a maximum pooling layer, a second convolutional layer, a first full connection layer, a second full connection layer, and a normalization layer. Figure 6

[0112] Exemplarily, the transformation results of the acceleration signals in the three directions in the angular space dimension can be spliced in the spatial dimension, respectively including a transformation result of a longitudinal acceleration signal, a transformation result of a lateral acceleration signal, and a transformation result of a vertical acceleration signal, to form an input tensor with a size of 65x57x1 as an input of the first convolutional layer.

[0113] The first convolutional layer can include 16 5x5 convolutional kernels with a step of 1. The output is a first convolutional result. After batch normalization (BN) and ReLU activation of the first convolutional result, the first processing result is input into a maximum pooling layer with a size of 2x2. Figure 6 The two processes are represented by BN and ReLU, respectively.

[0114] The step of the maximum pooling layer can be 2, which can reduce the spatial dimension, and the output is a pooling result.

[0115] The second convolutional layer can include 32 3x3 convolutional kernels. The input is the pooling result, and the output is a second convolutional result. Batch normalization and ReLU activation are performed on the second convolutional result to extract features and obtain a second processing result.

[0116] The first full connection layer can include 128 neurons, the input is the second processing result, and the output is a first full connection result. ReLU activation is performed on the first full connection result to obtain a third processing result.

[0117] ​The second full connection layer can include 8 neurons. The input thereof is the third processing result, and the second full connection layer can map the features to 8 output categories, and then classify through a normalization (softmax) layer to obtain the corresponding road surface type and confidence.

[0118] In the training process, a stochastic gradient descent (SGDM) optimizer with momentum can be used, the learning rate is set to 0.005, and the mini-batch size is 128.

[0119] In a possible embodiment, part of the collected data is used for training of the convolutional neural network, and another part is left for subsequent estimation algorithm verification, so as to better reflect the generalization ability of the model on unfamiliar data.

[0120] Samples of vehicles on different road surface types under the conditions of regular uniform speed and double lane change (DLC) can be collected, and the samples are divided in proportion. The entire data set is divided into a training set, a verification set, and a test set for training of the CNN model.

[0121] Table 1 shows a schematic diagram of a confusion matrix when a test set is tested in an embodiment of the present disclosure. As shown in Table 1, the confusion matrix includes various road surface types, and the following eight types are taken as examples: fine asphalt, coarse asphalt, brick, cement, wet ceramic tile, wet basalt, snow, and ice.

[0122] Specific data information is shown in Table 1.

[0123] Table 1

[0124]

[0125] The horizontal direction of Table 1 represents different road surface types output by the convolutional neural network, that is, the predicted categories. The vertical direction represents the real road surface types during sampling, that is, the real categories. The data in the table represents the number.

[0126] For example, for the cement, the number of real categories in the vertical direction is 82, that is, the total number of samples on the cement road surface is 82. The number of predicted categories in the horizontal direction is 82, that is, the number of final road surface types output by the convolutional neural network is 82.

[0127] Table 2 represents the precision and negative predictive value respectively. Specific data information is shown in Table 2, and the unit is %.

[0128] Table 2

[0129]

[0130] Exemplarily, taking fine asphalt as an example, in an item in which the predicted category is fine asphalt in the transverse direction, the number of items in which the predicted category is fine asphalt is 255, the number of items in which the true category is fine asphalt is 251, and the number of items in which the true category is coarse asphalt is 4. Then, the probability of predicting positive, i.e., the precision, is 98.4% and the probability of predicting negative, i.e., the negative predictive value, is 1.6% when the final road type is estimated by the convolutional neural network.

[0131] Table 3 shows the recall and the specificity. The specific data information is shown in Table 3, and the unit is %.

[0132] Table 3

[0133]

[0134] Exemplarily, taking fine asphalt as an example, in an item in which the predicted category is fine asphalt in the longitudinal direction, the number of items in which the true category is fine asphalt is 252, the number of items in which the predicted category is fine asphalt is 251, and the number of items in which the predicted category is other categories is 1. Then, the probability of predicting positive, i.e., the recall, is 99.6% and the probability of predicting negative, i.e., the specificity, is 0.4% when the final road type is estimated by the convolutional neural network.

[0135] In the table 2 and the table 3, the probability values corresponding to the transverse direction and the longitudinal direction of the table 1 are output.

[0136] As can be seen from the above table, the accuracy of the estimation result of the convolutional neural network in the embodiment of the disclosure is very high after training. In the case of not depending on the vehicle motion state information (vehicle speed information) and large excitation, the road type can be accurately obtained, and then the friction coefficient is determined.

[0137] In order to evaluate the robustness of the convolutional neural network, five-fold cross-validation (5-fold cross-validation, CV) is used to evaluate on the entire data set.

[0138] Figure 7 A schematic diagram of cross-validation in the embodiment of the disclosure is shown, as shown in Figure 7 The cross-validation results of estimating separately by the longitudinal acceleration signal, the lateral acceleration signal and the vertical acceleration signal, and estimating together by the acceleration signals in three directions.

[0139] The longitudinal coordinate represents the accuracy of the estimation, and the longitudinal coordinate includes: 0.88, 0.9, 0.92, 0.94, 0.96, 0.98 and 1. The transverse coordinate represents the acceleration signal used, and includes: the longitudinal acceleration signal, the lateral acceleration signal and the vertical acceleration signal, and the acceleration signals in three directions.

[0140] Based on Figure 7The range of accuracy using different acceleration signals can be seen that the accuracy is the highest when using the acceleration signals in three directions to estimate, because the three-axis information is fused in the convolutional neural network, providing richer features, so that the model can realize feature complementation between different road types, and improve the estimation accuracy.

[0141] In a possible embodiment, the processing manner of the prediction update process and the measurement update process of the adaptive Kalman filtering process can include the following steps, as shown in the following formulas.

[0142] The prediction update process of the kth time includes:

[0143] (1)

[0144] (2)

[0145] wherein, represents the friction coefficient prediction value of the kth time, and represents that it is calculated based on the second friction coefficient of the (k-1)th time. represents the error covariance prediction value of the kth time, and represents that it is calculated based on the error covariance update value of the (k-1)th time; represents the error covariance update value of the kth time; represents the second friction coefficient of the (k-1)th time, Q represents the process noise covariance, which can be set to 0.001, and A represents the unit matrix.

[0146] wherein, the initial second friction coefficient can be set to an extreme value far away from the friction coefficient corresponding to the real road type. This is not specifically limited in the embodiments of the present disclosure.

[0147] The measurement update process of the kth time includes:

[0148] (3)

[0149] (4)

[0150] (5)

[0151] wherein, represents the Kalman gain of the kth time. represents the second friction coefficient of the kth time, and represents that it is calculated based on the friction coefficient prediction value of the kth time, the Kalman gain of the kth time and the first friction coefficient of the kth time. represents the error covariance update value of the kth time, and represents that it is calculated based on the error covariance prediction value of the kth time and the Kalman gain of the kth time, represents the measurement covariance, denotes the first friction coefficient of the kth time, and H and I both denote unit matrices.

[0152] wherein the measurement covariance can be determined based on formula (6) as follows.

[0153] (6)

[0154] wherein, denotes the first confidence corresponding to the first friction coefficient of the kth time.

[0155] When the first confidence approaches 1, that is, the confidence is high, the measurement covariance approaches 0, and the adaptive Kalman filter gives a larger Kalman gain in the measurement update process. When the first confidence approaches 0, that is, the confidence is low, the obtained first friction coefficient is low in trustworthiness, the measurement covariance approaches infinity, and the obtained Kalman gain is very small, and the first friction coefficient of this time is actually ignored.

[0156] Figure 8 shows a flowchart of an adaptive Kalman filtering process in the embodiment of the present disclosure, as shown in Figure 8 , which includes a prediction update process, a judgment process and a measurement update process.

[0157] S802: Perform the prediction update process.

[0158] S804: Determine whether upflg is 1, if yes, perform S806; if no, perform S808.

[0159] wherein upflg is the abbreviation of update flag to determine whether there is an effective first friction coefficient at the current time.

[0160] S806: Perform the measurement update process.

[0161] S808 takes the friction coefficient prediction value of the current time as the second friction coefficient of the current time, and takes the error covariance prediction value of the current time as the error covariance update value of the current time, and returns to perform S802.

[0162] Through the above method in the embodiment of the present disclosure, when the vehicle travels at a constant speed on the coarse asphalt pavement, Figure 9 shows a schematic diagram of an evaluation result in the embodiment of the present disclosure, as shown in Figure 9 , which includes four coordinate systems, wherein the longitudinal coordinates of three coordinate systems are friction coefficients (μ), which are respectively: true friction coefficient, first friction coefficient and second friction coefficient, and the longitudinal coordinate of the other coordinate system is the confidence corresponding to the first friction coefficient, and the horizontal coordinates of the four coordinate systems are time (s).

[0163] The friction coefficient is shown in the coordinate system from 0-1.2, 0, 0.2, 0.4, 0.6, 0.8, 1 and 1.2, the confidence is from 0.5-1, and the time includes: 0, 2, 4, 6, 8, 10, 12, 14, 16 and 18, etc. The time dimension of the horizontal coordinate in the four coordinate systems is aligned.

[0164] The true friction coefficient of the crude pitch is 1, and in order to strictly evaluate, the initial state of the AKF, that is, the initial second friction coefficient, is set to 0.1, which is much lower than the true friction coefficient. During the conventional uniform driving process, the first friction coefficient directly output by the convolutional neural network misjudges once. The value of the second friction coefficient output after AKF filtering adjustment quickly converges to the correct value at the beginning when it is available.

[0165] In addition, the AKF can adjust the measurement covariance according to the probability output by the CNN, thereby adjusting the value of the second friction coefficient output. When the time is between 10s-12s, due to the low confidence (close to 0.5), if the value of the second friction coefficient is directly output at this time, it will cause the AKF to also misjudge, and the AKF effectively reduces the Kalman gain in this measurement update process, thereby maintaining the stability of the estimation result.

[0166] When the vehicle is on the snow and ice transition road surface, Figure 10 Another evaluation result in the embodiment of the present disclosure is shown in the schematic diagram as shown in Figure 10 As shown, four coordinate systems are included, in which the longitudinal coordinate of three coordinate systems is the friction coefficient (μ), which is respectively: the true friction coefficient, the first friction coefficient and the second friction coefficient, and the longitudinal coordinate of the other coordinate system is the confidence corresponding to the first friction coefficient, and the horizontal coordinate of the four coordinate systems is the time (s). The vehicle enters the ice surface from the snow at 4.7s.

[0167] The friction coefficient is shown in the coordinate system from 0.2-1, 0.2, 0.4, 0.6, 0.8 and 1, the confidence is from 0.5-1, and the time includes the horizontal coordinate corresponding to the friction coefficient: 0-9s, 1, 2, 3, 4, 5, 6, 7, 8 and 9, etc. And the horizontal coordinate corresponding to the confidence: 0-10s, 0, 2, 4, 6, 8 and 10, etc.

[0168] The initial state of the AKF is set to 1.0, and during the overall estimation process, the convolutional neural network has individual misjudgments, but the confidence corresponding to these first friction coefficients is low, therefore, the AKF can effectively reduce the Kalman gain of these unreliable first friction coefficients in the measurement update process, so that the estimation result remains stable. By Figure 10It can be seen that, when the road surface changes, further reprocessing the first friction coefficient output by the convolutional neural network using the AKF can make the estimation result quickly converge to the correct friction coefficient, thereby improving the accuracy of the friction coefficient estimation.

[0169] wherein, Figure 9 and Figure 10 The first friction coefficient is represented by the oblique lines, the second friction coefficient is represented by the dashed lines, and the true friction coefficient is represented by the solid lines. The oblique lines are connected by lines, so that the meaning of the first friction coefficient changing over time expressed in the schematic diagram is clearer.

[0170] Based on the same inventive concept, the disclosure also provides an intelligent tire-based road surface friction coefficient estimation device, as follows. Since the principle of solving the problem of the device embodiment is similar to the above-mentioned method embodiment, the implementation of the device embodiment can be referred to the implementation of the above-mentioned method embodiment, and the repeated parts will not be described here.

[0171] Figure 11 The structure of an intelligent tire-based road surface friction coefficient estimation device is shown in the disclosure embodiment, as shown in Figure 11 The intelligent tire-based road surface friction coefficient estimation device 110 includes: a sampling unit 1101 for sampling at a preset angle as a sampling interval to obtain acceleration signals of the intelligent tire in multiple cycles, taking one rotation of the intelligent tire as a cycle; the acceleration signals include: longitudinal acceleration signals, lateral acceleration signals and vertical acceleration signals; a Fourier transform unit 1102 for performing spatial short-time Fourier transform on the acceleration signals to obtain the transformation results of the acceleration signals in three directions in the angular space dimension; an estimation unit 1103 for inputting the transformation results into a convolutional neural network to obtain a road surface type and a corresponding confidence; a determination unit 1104 for determining a first friction coefficient of the road surface according to the road surface type, the corresponding confidence and a road surface type friction coefficient relationship table.

[0172] Those skilled in the art can understand that each aspect of the disclosure can be implemented as a system, a method or a program product. Therefore, each aspect of the disclosure can be embodied as a complete hardware embodiment, a complete software embodiment (including firmware, microcode, etc.), or an embodiment combining hardware and software aspects, which can be collectively referred to as "circuitry", "module" or "system" here.

[0173] The electronic device 1200 according to this embodiment of the disclosure will be described below with reference to Figure 12 Figure 12 The electronic device 1200 shown is only an example, and should not impose any limitation on the functions and use range of the embodiments of the disclosure. ​

[0174] like Figure 12 As shown, the electronic device 1200 is manifested in the form of a general-purpose computing device. The components of the electronic device 1200 may include, but are not limited to: at least one processing unit 1210, at least one storage unit 1220, and a bus 1230 connecting different system components (including storage unit 1220 and processing unit 1210).

[0175] The storage unit stores program code that can be executed by the processing unit 1210, causing the processing unit 1210 to perform the steps described in the "Exemplary Methods" section of this specification according to various exemplary embodiments of this disclosure. For example, the processing unit 1210 can perform the steps of any of the above-described method embodiments.

[0176] Storage unit 1220 may include readable media in the form of volatile storage units, such as random access memory (RAM) 12201 and / or cache memory 12202, and may further include read-only memory (ROM) 12203.

[0177] Storage unit 1220 may also include a program / utility 12204 having a set (at least one) of program modules 12205, such program modules 12205 including but not limited to: operating system, one or more application programs, other program modules and program data, each or some combination of these examples may include an implementation of a network environment.

[0178] Bus 1230 can represent one or more of several types of bus structures, including a memory cell bus or memory cell controller, a peripheral bus, a graphics acceleration port, a processing unit, or a local bus using any of the various bus structures.

[0179] The electronic device 1200 can also communicate with one or more external devices 1240 such as a keyboard or pointing device, a Bluetooth device, or a database, and / or with one or more devices that enable a user to interact with the electronic device 1200 and / or one or more devices (e.g., a router, a modem, and so forth) that enable the electronic device 1200 to communicate with one or more other computing devices. Such communication can occur via Input / Output (I / O) interface(s) 1250. Still yet, the electronic device 1200 can communicate with one or more networks, such as one or more local area networks (LANs), wide area networks (WANs), and / or the Internet, through network adapter 1260. As depicted, network adapter 1260 communicates with the other components of the electronic device 1200 via bus 1230. It should be appreciated that although not shown, other hardware and / or software components could be used in conjunction with the electronic device 1200. These include, but are not limited to, microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data archival storage systems, etc.

[0180] Through the above description of the embodiments, those skilled in the art can easily understand that the example embodiments described herein can be implemented by software, or by software in combination with necessary hardware. Therefore, the technical solutions according to the embodiments of the present disclosure can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, a USB flash disk, a mobile hard disk, etc.) or a network, and includes a number of instructions to enable a computing device (which can be a personal computer, a server, a terminal device, or a network device, etc.) to perform the methods according to the embodiments of the present disclosure.

[0181] In particular, according to the embodiments of the present disclosure, the processes described above with reference to the flowcharts can be implemented as a computer program product or a computer program, which includes computer instructions stored in a computer readable storage medium. The processor of the computer device reads the computer instructions from the computer readable storage medium, and the processor executes the computer instructions, so that the computer device executes the methods in the above embodiments.

[0182] In the example embodiments of the present disclosure, a computer readable storage medium is also provided, which can be a readable signal medium or a readable storage medium. A program product capable of implementing the above methods of the present disclosure is stored thereon. In some possible embodiments, various aspects of the present disclosure can also be implemented in the form of a program product, which includes program codes for causing the terminal device to execute the steps according to various example embodiments of the present disclosure described in the above "example method" section of the specification when the program product runs on the terminal device.

[0183] More specific examples of the computer-readable storage medium in the present disclosure can include but are not limited to an electrical connection having one or more wires, a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0184] In the present disclosure, a computer-readable storage medium can include a data signal carrying the readable program code in a baseband or propagated as a carrier wave in a propagation medium. Such a propagated signal can take a wide variety of forms, including but not limited to electro-magnetic, optical, or any suitable combination thereof. A computer-readable medium can also be any medium that can be read by a computer or device or combination of devices. The computer-readable medium can be any medium that can be read by a computer or device or combination of devices.

[0185] Optionally, program code embodied on a computer-readable storage medium can be transmitted by any appropriate medium, including but not limited to wireless, wired, optical fiber cable, RF, etc., or any suitable combination of the foregoing.

[0186] In an implementation, the program code for carrying out operations of the present disclosure can be written in any combination of one or more programming languages, including an object oriented programming language such as Java, C++, etc., or conventional procedural programming languages, such as the "C" programming language, or similar programming languages. The program code can execute entirely on the user's computing device, partly on the user's device, as a stand-alone software package, partly on the user's computing device and partly on a remote computing device or entirely on the remote computing device or server. In the latter scenario, the remote computing device can be connected to the user's computing device through any type of network, including a local area network (LAN) or a wide area network (WAN), or the connection can be made to an external computing device, such as through the Internet using an Internet Service Provider.

[0187] It should be noted that, although several modules or units for a device to perform actions are mentioned in the foregoing detailed description, such a division into modules or units is not mandatory. In fact, according to an embodiment of the present disclosure, features and functions of two or more modules or units described above can be embodied in one module or unit. Conversely, a feature or function of one module or unit described above can be further divided into multiple modules or units.

[0188] Moreover, although individual steps of the methods in the present disclosure are described in a particular order in the drawings, this is not required or implied, nor is it necessary to perform all of the steps shown to achieve the desired result. Additionally or alternatively, certain steps can be omitted, combined into a single step, performed in a different order, broken down into multiple steps, and / or the like.

[0189] Those skilled in the art will readily understand that the example embodiments described herein can be implemented by software and / or by hardware coupled with software, as described above. Thus, the technical solutions according to the embodiments of the present disclosure can be embodied in the form of a software product. The software product can be stored in a non-volatile storage medium (which can be a CD-ROM, a USB flash disk, a mobile hard disk, or the like) or on a network, and includes a number of instructions to enable a computing device (which can be a personal computer, a server, a mobile terminal, or a network device, etc.) to perform the methods according to the embodiments of the present disclosure.

[0190] Other embodiments of the present disclosure will be apparent to those skilled in the art from consideration of the specification and practice of the application disclosed herein. The present disclosure is intended to cover any variations, uses, or adaptations of the present disclosure following the general principles thereof and including such departures from the present disclosure that come within known use or custom in the art to which the present disclosure pertains. The specification and examples are to be regarded as illustrative only, and the true scope and spirit of the present disclosure are indicated by the appended claims.

Claims

1. A method for estimating road surface friction coefficient based on smart tire, characterized in that, The method comprises: Taking one rotation of the intelligent tire as a period, sampling according to a preset angle as a sampling interval to obtain acceleration signals of the intelligent tire in multiple periods; the acceleration signals comprise longitudinal acceleration signals, lateral acceleration signals and vertical acceleration signals; Performing spatial short-time Fourier transform on the acceleration signals to obtain a transformation result of the acceleration signals in three directions in an angular space dimension; the spatial short-time Fourier transform is a short-time Fourier transform in the angular space dimension based on input in the angular space dimension; the transformation result represents signal intensity at different angular space positions and angular space frequencies; the angular space frequency represents the number of spatial change periods contained in a unit angle range; Inputting the transformation result into a convolutional neural network to obtain a road surface type and a corresponding confidence; Determining a first friction coefficient of the road surface according to the road surface type, the corresponding confidence and a road surface type friction coefficient relationship table.

2. The method of claim 1, wherein, The method further comprises: Determining a second friction coefficient by performing adaptive Kalman filtering on the first friction coefficient and a first confidence corresponding to the first friction coefficient.

3. The method of claim 2, wherein, The adaptive Kalman filtering comprises a prediction update process and a measurement update process; The determination of the second friction coefficient by performing adaptive Kalman filtering on the first friction coefficient and the first confidence corresponding to the first friction coefficient comprises: In the prediction update process, determining a friction coefficient prediction value of a current time and an error covariance prediction value of the current time; In the measurement update process, determining a Kalman gain according to the error covariance prediction value of the current time and the first confidence; Determining the second friction coefficient of the current time according to the first friction coefficient, the Kalman gain and the friction coefficient prediction value of the current time.

4. The method of claim 3, wherein, The determination of the Kalman gain according to the error covariance prediction value of the current time and the first confidence in the measurement update process comprises: Determining a measurement covariance according to the first confidence; the larger the first confidence is, the smaller the measurement covariance is; Determining the Kalman gain according to the measurement covariance and the error covariance prediction value of the current time.

5. The method of claim 3, wherein, The determination of the error covariance prediction value comprises: Obtaining an error covariance update value in a measurement update process of the last adaptive Kalman filtering; the error covariance update value is determined according to the Kalman gain of the last time and an error covariance prediction value in a prediction update process of the last adaptive Kalman filtering; Determining the error covariance prediction value of the current time according to the error covariance update value.

6. The method of claim 3, wherein, The method further comprises: In the process of determining the second friction coefficient by performing adaptive Kalman filtering on the first friction coefficient and the first confidence corresponding to the first friction coefficient, determining whether there is an effective first friction coefficient in a current period; If there is, performing the measurement update process; If there is not, skipping the measurement update process, taking the friction coefficient prediction value of the current time as the second friction coefficient of the current time, and taking the error covariance prediction value of the current time as an error covariance update value of the current time.

7. A road surface friction coefficient estimation device based on intelligent tires, characterized in that, ​ A sampling unit is configured to sample the acceleration signals of the intelligent tire in a plurality of cycles, each cycle corresponding to one rotation of the intelligent tire, and each sampling interval corresponding to a preset angle. The acceleration signals include longitudinal acceleration signals, lateral acceleration signals, and vertical acceleration signals. A Fourier transform unit is configured to perform spatial short-time Fourier transform on the acceleration signals to obtain transform results of the acceleration signals in three directions in an angular space dimension. An estimation unit is configured to input the transform results into a convolutional neural network to obtain a road surface type and a corresponding confidence level. A determination unit is configured to determine a first friction coefficient of the road surface based on the road surface type, the corresponding confidence level, and a road surface type-friction coefficient relationship table.

8. An electronic device, comprising: The method comprises: a processor; and a memory for storing executable instructions of the processor; wherein the processor is configured to execute the executable instructions to perform the method of any one of claims 1-6.

9. A computer readable storage medium having stored thereon a computer program, characterized in that, The computer program is executed by the processor to implement the method of any one of claims 1-6.

10. A computer program product, comprising: The computer program or instructions are executed by the processor to implement the method of any one of claims 1-6.

Images