Tire burst detection method, control device, vehicle, storage medium and program product

Abstract

The present application discloses a tire blowout detection method, a control device, a vehicle, a storage medium, and a program product. Applied to a vehicle, the method includes: obtaining status data of the vehicle; processing the status data based on a preset detection neural network model to output a tire blowout detection result. The preset detection neural network model can accurately identify changes in the status data, so the detection neural network model can be used to process the status data. When the change in the status data conforms to the change before and after the tire blowout, the detection neural network model will immediately identify the situation and output the tire blowout detection result as a tire blowout, otherwise the tire blowout detection result is output as no tire blowout. In this way, the present application uses the detection neural network model and the status data acquired in real time to perform tire blowout detection. The detection speed of the detection neural network model is fast and the detection accuracy is high, so that the present application can achieve fast and high-precision tire blowout identification, thereby being able to detect tire blowouts in a timely manner.

Description

Technical Field

[0001] The present application relates to the field of vehicle technology, and more specifically, to a tire blowout detection method, a control device, a vehicle, a non-volatile computer-readable storage medium, and a computer program product. Background Art

[0002] During the operation of a vehicle, a tire may burst. When a tire bursts, the vehicle can easily deviate from the intended route, causing unstable control and increasing the chance of a traffic accident. Therefore, how to detect a tire burst in a timely manner has become an urgent problem to be solved. Summary of the invention

[0003] Embodiments of the present application provide a tire blowout detection method, a control device, a vehicle, a non-volatile computer-readable storage medium, and a computer program product.

[0004] The tire blowout detection method of the embodiment of the present application is applied to a vehicle, and the method includes: acquiring status data of the vehicle; processing the status data based on a preset detection neural network model to output a tire blowout detection result.

[0005] The control device of the embodiment of the present application includes a processor, a memory and a computer program, wherein the computer program is stored in the memory and executed by the processor, and the computer program includes instructions for executing a tire blowout detection method. The tire blowout detection method is applied to a vehicle, and the method includes: obtaining status data of the vehicle; processing the status data based on a preset detection neural network model to output a tire blowout detection result.

[0006] The vehicle of the embodiment of the present application includes a control device, the control device includes a processor, a memory and a computer program, wherein the computer program is stored in the memory and executed by the processor, and the computer program includes instructions for executing a tire blowout detection method. The tire blowout detection method is applied to a vehicle, and the method includes: obtaining status data of the vehicle; processing the status data based on a preset detection neural network model to output a tire blowout detection result.

[0007] The non-volatile computer-readable storage medium of the embodiment of the present application includes a computer program, and when the computer program is executed by a processor, the processor executes a tire blowout detection method. The tire blowout detection method is applied to a vehicle, and the method includes: obtaining status data of the vehicle; processing the status data based on a preset detection neural network model to output a tire blowout detection result.

[0008] The computer program product of the embodiment of the present application includes a computer program, wherein the computer program includes instructions for executing a tire blowout detection method. The tire blowout detection method is applied to a vehicle, and the method includes: obtaining status data of the vehicle; processing the status data based on a preset detection neural network model to output a tire blowout detection result.

[0009] The tire blowout detection method, control device, vehicle, non-volatile computer-readable storage medium, and computer program product of the embodiments of the present application can obtain the status data of the vehicle in real time, and the status data before and after the tire blowout will change. The preset detection neural network model can accurately identify the change of the status data, so the detection neural network model can be used to process the status data. When the change of the status data conforms to the change before and after the tire blowout, the detection neural network model will immediately identify the situation and output the tire blowout detection result as a tire blowout, otherwise the tire blowout detection result is output as no tire blowout. In this way, the present application uses the detection neural network model and the status data obtained in real time to perform tire blowout detection. The detection speed of the detection neural network model is fast and the detection accuracy is high, so that the present application can achieve fast and high-precision tire blowout identification, so that the tire blowout situation can be detected in time.

[0010] Additional aspects and advantages of the embodiments of the present application will be given in part in the description below, and in part will become apparent from the description below, or will be learned through the practice of the embodiments of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] The above and / or additional aspects and advantages of the present application will become apparent and easily understood from the description of the embodiments in conjunction with the following drawings, in which:

[0012] Figure 1 is a flow chart of a tire blowout detection method according to certain embodiments of the present application;

[0013] Figure 2 is a flow chart of a tire blowout detection method according to certain embodiments of the present application;

[0014] Figure 3 is a flow chart of a tire blowout detection method according to certain embodiments of the present application;

[0015] Figure 4 is a schematic diagram of a tire blowout detection method according to some embodiments of the present application;

[0016] Figure 5 is a flow chart of a tire blowout detection method according to certain embodiments of the present application;

[0017] Figure 6 is a flow chart of a tire blowout detection method according to certain embodiments of the present application;

[0018] Figure 7 is a flow chart of a tire blowout detection method according to certain embodiments of the present application;

[0019] Figure 8 is a flow chart of a tire blowout detection method according to certain embodiments of the present application;

[0020] Fig. 9 is a flow chart of a tire blowout detection method according to certain embodiments of the present application;

[0021] Fig.10 is a flow chart of a tire blowout detection method according to certain embodiments of the present application;

[0022] Fig.11 is a flow chart of a tire blowout detection method according to certain embodiments of the present application;

[0023] Fig.12 is a flow chart of a tire blowout detection method according to certain embodiments of the present application;

[0024] Fig.13 is a schematic diagram of a module of a control device in certain embodiments of the present application;

[0025] Fig.14 is a schematic diagram of the structure of a vehicle according to some embodiments of the present application;

[0026] Fig.15 is a schematic diagram of the connection state of a non-volatile computer-readable storage medium and a processor in certain embodiments of the present application;

[0027] Fig.16 It is a module diagram of a computer program product of certain embodiments of the present application. DETAILED DESCRIPTION

[0028] The embodiments of the present application are described in detail below, and examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals represent the same or similar elements or elements having the same or similar functions from beginning to end. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the embodiments of the present application, and cannot be understood as limiting the embodiments of the present application.

[0029] The embodiment of the present application provides a tire blowout detection method, which will be described in detail below:

[0030] See also Figure 1 The embodiment of the present application provides a tire blowout detection method, which is applied to a vehicle. The tire blowout detection method includes:

[0031] Step 01: Get the vehicle status data;

[0032] Specifically, various status data of the vehicle before and after a tire blowout, for example, the tire pressure of the vehicle will drop sharply after a tire blowout. Therefore, tire blowout detection of the wheel can be performed based on the status data related to the vehicle, and the status data includes data that can reflect the current state of the vehicle to reflect the current state of the wheel (i.e., whether there is a tire blowout). The status data can be acquired through corresponding sensors. In some embodiments, the status of the wheel, the inertial sensor, and the motor connected to each wheel can be used to reflect the current state of the vehicle, so the status data of the vehicle includes at least one of the status data of the wheel of the vehicle, the status data of the inertial sensor of the vehicle, and the status data of the motor connected to the wheel. Among them, each wheel of the vehicle is connected to a motor, and the number of motors can be one or more, for example, the number of motors can be 1, 2, 3 or 4, and the motors connected to different wheels can be the same or different.

[0033] For example, the status data of the wheel includes the wheel speed signal and / or tire pressure signal of the wheel. The status data of the inertial sensor includes the acceleration signal and / or angular velocity signal of the inertial sensor in three directions (usually the X-axis, Y-axis and Z-axis, representing the lateral, longitudinal and vertical directions of the vehicle, respectively). The status data of the motor includes the speed signal and / or torque signal.

[0034] Among them, after a tire blowout, the lateral acceleration of the wheel will increase significantly, and the yaw rate of the vehicle will increase significantly. At the same time, in order to maintain the stability of the vehicle, the driver may take braking measures, which will cause changes in the longitudinal acceleration. In addition, after a tire blowout, the suspension system of the vehicle will be subjected to a greater impact due to the loss of tire pressure. This may lead to a significant change in vertical acceleration, especially when the vehicle passes through uneven roads. After a tire blowout, the pitch rate and roll rate of the vehicle may also change, but the specific degree of change depends on the location of the tire blowout and the characteristics of the vehicle's suspension system. As the tire loses air pressure, the rolling resistance increases, and the speed of the motor corresponding to the wheel with a tire blowout will decrease rapidly. After a tire blowout, the torque of the vehicle will change. Taking the right rear wheel blowout as an example, the resistance of the wheel will increase sharply at the moment of the tire blowout. This change will cause the vehicle to be subjected to a clockwise lateral torque, causing the vehicle to deviate to the right. Each wheel is connected to a corresponding motor, so the data of the motor can also reflect the status of the connected wheel.

[0035] Therefore, during the driving process of the vehicle, the corresponding status data of the vehicle and the related status data can be obtained in real time through the corresponding sensors, so as to judge whether each wheel has a tire blowout in real time according to the status data corresponding to the vehicle. In some embodiments, the step length of the acquired status data can be obtained by actual vehicle test calibration, and will not be changed during the detection process. For example, if the step length is 4s, then each time the status data is processed, the entire segment of the status data at that moment and the previous 4s continuous status data will be processed.

[0036] Step 02: Process the status data based on the preset detection neural network model to output the tire blowout detection result.

[0037] Specifically, a detection neural network model that can detect tire blowouts based on state data can be pre-trained. The detection neural network model can extract the features of each type of data in the state data, and accurately identify the changes of each type of data based on the features. When the changes in the state data are consistent with the actual changes before and after the tire blowout, the detection neural network model will immediately identify the situation, and the detection neural network model can determine that the wheel corresponding to the state data has a tire blowout. When the changes in the state data do not conform to the actual changes before and after the tire blowout, the detection neural network model can determine that the wheel has not had a tire blowout at this time.

[0038] The detection neural network model can process the state data, extract the corresponding features from the state data, and determine whether the wheel has a flat tire based on the features, thereby outputting a flat tire detection result. In one embodiment, the detection neural network model can directly obtain the result of whether the vehicle has a flat tire after processing the features. In another embodiment, the detection neural network model will obtain the probability of a flat tire and the probability of no flat tire after processing the features. When the probability of a flat tire is greater than the preset flat tire probability threshold, the flat tire detection result is a flat tire, otherwise it is no flat tire. When the flat tire detection result is a flat tire, an alarm can be triggered to promptly inform the driver or initiate emergency measures. Among them, the preset flat tire probability threshold is the minimum value of the flat tire probability predicted by the detection neural network model in the event of a flat tire.

[0039] The prior art usually adopts the method of monitoring tire pressure, which has a slow response speed, so it is difficult to achieve rapid identification after a vehicle tire blowout. However, the present application has previously trained a detection neural network model that can detect tire blowouts based on state data, and in actual use, the state data obtained in real time is input into the detection neural network model. The neural network model processes data quickly, and the trained detection neural network model has a high accuracy in detecting tire blowouts. At the same time, the state data of the present application may not only include tire pressure, but the sampling frequency of other state data may be higher than the sampling frequency of tire pressure, for example, the sampling frequency of rotation speed or wheel speed is higher than the sampling frequency of tire pressure, so that compared with the method of only using tire pressure for detection, the present application can also achieve a faster detection speed when detecting whether there is a tire blowout. Therefore, compared with the prior art that uses a single signal for tire blowout detection, the present application has a faster recognition speed and accuracy, so that the present application can identify the tire blowout more promptly. Therefore, the present application is also applicable to tire blowout scenarios caused by abnormalities such as non-tire aging but tire encountering sharp objects, that is, the present application can quickly identify various sudden tire blowout situations.

[0040] The tire blowout detection method of the embodiment of the present application can obtain the status data of the vehicle in real time, and the status data before and after the tire blowout will change. The preset detection neural network model can accurately identify the change of the status data, so the detection neural network model can be used to process the status data. When the change of the status data conforms to the change before and after the tire blowout, the detection neural network model will immediately identify the situation and output the tire blowout detection result as a tire blowout, otherwise the tire blowout detection result is output as no tire blowout. In this way, the present application uses the detection neural network model and the status data obtained in real time to perform tire blowout detection. The detection speed of the detection neural network model is fast and the detection accuracy is high, so that the present application can achieve fast and high-precision tire blowout identification, so as to detect the tire blowout in time.

[0041] See also Figure 2 In some embodiments, step 02: processing multimodal data based on a preset detection neural network model to output a tire blowout detection result includes:

[0042] Step 021: extracting features from the state data to obtain state features, where the state features are used to at least characterize changes in various state data;

[0043] Step 022: Process the state features to output the tire blowout detection result.

[0044] Specifically, features can be direct measurements in the original data or the result of some transformation or processing. Feature extraction is a key step in neural networks, and its goal is to identify the most representative information from the original data, which can reveal the inherent laws and patterns of the data. Neural models that can extract features include convolutional neural networks, recurrent neural networks, or long short-term memory networks. Of course, some algorithms can also be used for feature extraction, such as the KLT (Kanade-Lucas-Tomasi) algorithm. Features can at least be used to characterize the changes in the corresponding data. For example, the features corresponding to the wheel speed signal at a certain moment can be used to characterize the wheel speed at the current moment, and the features corresponding to the wheel speed signals at multiple consecutive moments can be used to characterize the changes in the wheel speed within the consecutive moments.

[0045] Therefore, a suitable neural network model or other algorithm that can extract features can be used to extract features from the state data to extract representative key features from the state data, so as to facilitate the subsequent judgment of whether the vehicle has a flat tire based on the key features. For example, if the state data is speed data, the key feature can be the average speed, speed change rate or speed fluctuation range within a certain period of time. At the same time, representative key features are extracted from the state data, so as to facilitate the subsequent judgment of whether the vehicle has a flat tire based on the key features.

[0046] Next, the state features may be processed to utilize the state features to discover changes in the state data, and based on the changes and actual changes in the state data before and after the tire blowout, it may be determined whether the vehicle has a tire blowout.

[0047] In this way, after obtaining the status data, the detection neural network model can use the corresponding feature extraction method to perform feature extraction to obtain the status characteristics that can characterize the corresponding changes, and then accurately determine the changes based on the status characteristics to obtain the vehicle's status information, thereby facilitating more accurate tire blowout detection.

[0048] See also Figure 3 and Figure 4 In some embodiments, the detection neural network model includes a convolutional neural network model, the convolutional neural network includes an input layer, multiple convolutional layers, multiple pooling layers and an output layer, the convolutional layers and the pooling layers are arranged alternately, and the output layer corresponds to the pooling layer. In the case where the multimodal data includes state data, step 021: extracting features from the multimodal data to obtain multimodal features, including:

[0049] Step 0211: Input the state data into the convolutional neural network to obtain state features.

[0050] Specifically, convolutional neural networks can be used to extract features from state data. Figure 4 As shown in FIG. 1 , the convolutional neural network includes an input layer, multiple convolutional layers, multiple pooling layers, and an output layer. Multiple convolutional layers and multiple pooling layers are arranged alternately, and the output layer corresponds to the pooling layer, that is, the layer before the output layer is the pooling layer. At this time, the convolutional neural network is a one-dimensional convolutional neural network.

[0051] The input layer is used to receive data. The convolution layer convolves the input data with a set of learnable convolution kernels through convolution operations to extract the features of the input data. The function of the convolution layer is to extract local features of the input data through convolution operations, thereby realizing feature extraction and dimensionality reduction of the input data. The parameter sharing and sparse connection characteristics of the convolution layer enable it to effectively reduce the number of parameters of the neural network and improve the training efficiency and generalization ability of the model. The pooling layer is a neural network layer used to reduce the dimension of the feature map. It performs pooling operations on the local area of ​​the input data and statistically summarizes the feature values ​​in the local area to obtain a smaller feature map. The function of the pooling layer is to reduce the dimension of the feature map through pooling operations, thereby reducing the computational complexity and memory consumption of the model. Common operations of the pooling layer include maximum pooling and average pooling, which can effectively improve the robustness and generalization ability of the model and reduce the risk of overfitting. The output layer is used to output the extracted features. Among them, the convolution layer uses a linear rectifier function as the activation function, and the window size of the convolution layer is preset and trained multiple times to select a suitable value. The result after convolution enters the pooling layer. The pooling layer uses maximum pooling for operation, traverses its input with a fixed stride, and takes the maximum value in the window as the pooling output result. The window size of the pooling layer is preset and trained multiple times to select a suitable value.

[0052] The convolutional layers and pooling layers are arranged alternately. The convolution result of the convolutional layer enters the pooling layer of the next layer, and the output result of the pooling layer enters the convolutional layer of the next layer. By repeating the operation for multiple rounds, high-level features of the image can be extracted layer by layer.

[0053] The convolution layer is responsible for extracting local features, while the pooling layer reduces the dimension and filters these features, so that the subsequent layers can process more abstract and advanced features. At the same time, the pooling layer reduces the size of the feature map through downsampling operations, thereby reducing the amount of calculation of the subsequent convolutional layers.

[0054] Therefore, after obtaining the status data, the status data can be input into a convolutional neural network to continuously perform feature extraction and dimensionality reduction on the status data using multiple convolutional layers and pooling layers, thereby accurately extracting status features in the status data that can reflect the current vehicle operation information.

[0055] See also Figure 4 In some embodiments, the tire blowout detection method includes obtaining the sampling frequency of each type of collected data in the status data, and taking the highest sampling frequency as the reference frequency; resampling the target sampling data based on the reference frequency, the target sampling data, and the sampling frequency of the target sampling data so that the data amount of all the sampling data is the same, and the target sampling data is the data in the sampling data whose sampling frequency is less than the reference frequency; and updating the status data based on the resampled target sampling data.

[0056] Specifically, the sampling frequencies of various collected data in the state data may be different, so that the data volumes of various collected data may be different, and the different data volumes may affect subsequent processing.

[0057] Therefore, the sampling frequency of each type of collected data in the state data can be determined first, and the highest sampling frequency can be used as the reference frequency. Then, the data with a sampling frequency less than the reference frequency in the sampled data can be identified as the target sampled data, and the target sampled data can be resampled based on the reference frequency and the sampling frequency of the target sampled data to make the data volume of all the sampled data the same.

[0058] For example, the sampling frequency of the wheel speed signal is 10Hz, and the wheel speed signal is sampled 10 times in 1 second; the sampling frequency of the tire pressure signal is 5Hz, and the wheel speed signal is sampled 5 times in 1 second. It can be determined that the reference frequency is 10Hz, and the tire pressure signal is the target sampling data. Obviously, the data volume of the tire pressure signal is half that of the wheel speed signal, so the tire pressure signal is resampled to make the data volume of the tire pressure signal the same as that of the wheel speed signal. The data volume of the tire pressure signal is half that of the wheel speed signal, that is, the tire pressure signal is doubled, so a new tire pressure signal can be added after each data in the original data of the tire pressure signal, and the newly added tire pressure signal can be determined based on the original data of the tire pressure signal. For example, the tire pressure signals are 2.48bar, 2.46bar, 2.47bar, 2.49bar and 2.5bar. After the tire pressure signal is expanded, the tire pressure signal can be 2.48bar, 2.48bar (newly added), 2.46bar, 2.46bar (newly added), 2.47bar, 2.47bar (newly added), 2.49bar, 2.49bar (newly added), 2.5bar and 2.5bar (newly added). Among them, the newly added data must be determined based on the original data before the time corresponding to the newly added data. In this way, after the tire pressure signal is expanded, the data volume of the tire pressure signal is the same as the data volume of the wheel speed signal, and the newly added data are all determined based on the original data of the tire pressure signal, so the data accuracy of the expanded tire pressure signal can be ensured.

[0059] Then the state data can be updated according to the resampled target sampling data, that is, the corresponding original data in the state data is replaced with the resampled data, and the data in the state data whose sampling frequency is equal to the reference frequency does not need to be updated, so that the data amount of each sampling data in the state data is the same.

[0060] In this way, the amount of data for each sampled data in the state data can be made the same, so that when the convolutional neural network processes the state data, there is no need to consider the conversion and interpolation problems between different sampling frequencies, which is conducive to simplifying the data processing process and reducing the complexity and time cost of data processing. At the same time, the consistency of data volume helps to ensure that the input data is consistent in time and space, which helps the neural network better capture the patterns and features in the data and improve the accuracy and reliability of the model.

[0061] See also Figure 4 and Figure 5 In some embodiments, step 022: processing state features to output tire blowout detection results includes:

[0062] Step 0221: Determine the feature vector corresponding to each moment according to the state feature corresponding to each moment;

[0063] Step 0222: determining a feature matrix according to the feature vector corresponding to the current moment and the feature vector corresponding to each moment in a preset time period before the current moment;

[0064] Step 0223: inputting the feature matrix into the feature processing neural network model of the detection neural network model to obtain the probability of tire blowout and the probability of no tire blowout;

[0065] Step 0224: Output the tire blowout detection result based on the tire blowout probability and the tire non-blowout probability.

[0066] Specifically, the state features obtained by the above method are in the form of column vectors, and in order to reduce the amount of calculation, even if other feature extraction methods are used, the final state features are generally in the form of column vectors. Therefore, at this time, the feature vector corresponding to each moment can be directly determined according to the state features corresponding to each moment.

[0067] Then, the feature matrix is ​​determined according to the feature vector corresponding to the current moment and the feature vector corresponding to each moment in the preset time period before the current moment, that is, the feature vectors in the current moment and the period before the current moment are concatenated to obtain the feature matrix. In some embodiments, the width of the feature matrix, that is, the length of the preset time period, is obtained by calibration of actual vehicle tests.

[0068] Then, the feature matrix is ​​input into the feature processing neural network model of the detection neural network model, and the feature processing neural network model is used to determine the state change of the vehicle according to the feature matrix, so as to determine the probability of a tire blowout and the probability of no tire blowout according to the state change. That is, the feature processing neural network model can determine the state change of the vehicle during this period of time according to the state characteristics of each moment in the feature matrix, so as to determine whether the vehicle has a tire blowout, so as to output the probability of a tire blowout and the probability of no tire blowout. When the probability of a tire blowout is greater than the preset probability threshold of a tire blowout, it can be determined that the vehicle has a tire blowout, and the tire blowout detection result is a tire blowout, otherwise the tire blowout detection result is no tire blowout.

[0069] In one embodiment, the feature processing neural network model includes a bidirectional long short-term memory neural network. The bidirectional long short-term memory neural network (BiLSTM) is an improved recurrent neural network (RNN) specifically designed to process sequence data. BiLSTM captures the bidirectional dependency in the sequence by combining the outputs of two forward and backward long short-term memory networks (LSTM). At this time, a bidirectional long short-term memory neural network consisting of an input gate, a forget gate, and an output gate can be constructed, and the aforementioned feature matrix can be input into the network to predict the tire blowout. The forget gate uses the tanh activation function, that is, the output gate uses the sigmoid activation function, and the output is the probability of being classified as a tire blowout and a tire not blown out.

[0070] In this way, the feature processing neural network model can combine the state features at the current moment and within a preset time period to determine the state change of the vehicle during this period, thereby accurately judging whether the vehicle has a tire blowout based on the state change.

[0071] See also Figure 4 and Figure 6 In some embodiments, the tire blowout detection method further comprises:

[0072] Step 03: Obtain image data collected by the vehicle;

[0073] Step 04: Process the image data and status data based on the preset detection neural network model to output the tire blowout detection result.

[0074] Specifically, when performing tire blowout detection, tire blowout detection can also be performed in combination with image data and state data collected from the vehicle. In some embodiments, after a tire blowout, the driver's ability to control the wheel with a blowout is reduced, the vehicle is likely to deviate from the established driving route, and the motion trajectory of the wheel may deviate. Therefore, the collected image data includes motion image data of the vehicle, and the motion image data of the vehicle can be collected based on the vehicle's front-view camera or other cameras that can capture road conditions. The motion trajectory of the wheel can be determined based on the image data, and when the motion trajectory is abnormally offset, it can be considered that the wheel may have a tire blowout. In other embodiments, the image data collected by the image includes wheel image data, and holes may appear on the wheel after a tire blowout. At this time, it can be determined whether the vehicle has a tire blowout based on the wheel image data. Therefore, two adjacent frames of image data can be used to reflect changes in the state of the vehicle, so that the image data can also be used for tire blowout detection.

[0075] The detection neural network model can be trained to be a neural network model that can detect tire blowouts based on state data and image data. That is, the detection neural network model can extract the features of each data in the state data and image data, and accurately identify the changes of each data based on the features. When the changes in the state data and image data are consistent with the actual changes before and after the tire blowout, the detection neural network model will immediately identify the situation, and the detection neural network model can determine that the wheel corresponding to the state data and image data has a tire blowout. When the changes in the state data and image data do not conform to the actual changes before and after the tire blowout, the detection neural network model can determine that the wheel has not had a tire blowout at this time.

[0076] Therefore, during the operation of the vehicle, image data and status data can be collected by corresponding sensors, and then the image data and status data can be input into the detection neural network model together, so that the detection neural network model can extract corresponding features from the status data and image data, and determine whether the wheel has a flat tire based on the features, thereby outputting the flat tire detection result.

[0077] In this way, the present application introduces a signal with a fast response and a high sampling frequency and integrates multimodal information (status data and image data), that is, combines information from multiple aspects and angles to determine whether a tire blowout has occurred, thereby achieving more comprehensive detection, thereby facilitating the improvement of the speed and accuracy of tire blowout detection.

[0078] See also Figure 4 and Figure 7 In some embodiments, step 04: processing the image data and the state data based on a preset detection neural network model to output a tire blowout detection result includes:

[0079] Step 041: extracting features from the state data to obtain state features, where the state features are used to at least characterize changes in various state data;

[0080] Step 042: extracting features from the image data to obtain image features, where the image features are used to at least characterize changes in the vehicle's motion trajectory;

[0081] Step 043: Process the state features and image features to output the tire blowout detection result.

[0082] Specifically, a suitable neural model or other algorithm that can extract features, such as the KLT (Kanade-Lucas-Tomasi) algorithm, can be used to extract features from state data and image data, wherein the state features are used to at least characterize the changes in various state data, and the image features are used to at least characterize the changes in the vehicle's motion trajectory, so that the vehicle is subsequently judged based on the key features to determine whether it has a tire blowout. For example, the motion displacement of the same pixel point between two frames is determined based on the image data. When the motion displacement of the pixel point is abnormally offset, it can be considered that the vehicle has a tire blowout. Among them, the method for extracting state features can refer to the feature extraction method in the above-mentioned embodiment, which will not be repeated here.

[0083] Next, the state features and image features can be processed to use the state features and image features to discover changes in the state data and image data, and determine whether the wheel has a tire blowout based on these two changes and the changes in the state data and image data before and after the tire blowout.

[0084] The image data may be preprocessed before input, such as performing image enhancement or denoising on the image data, and then feature extraction may be performed on the preprocessed image data to improve the feature extraction effect of the image data.

[0085] In this way, after acquiring the image data and the status data, the corresponding feature extraction methods can be used to perform feature extraction respectively to obtain image features and status features that can characterize the corresponding changes, and then the changes can be accurately determined based on the image features and the status features at the same time to obtain the vehicle status information from multiple angles, thereby facilitating the use of the multi-angle information to achieve more accurate tire blowout detection.

[0086] See also Figure 4 and Figure 8 In some embodiments, step 042: extracting features from the image data to obtain image features, includes:

[0087] Step 0421: Pixel matching is performed on the image data of the current frame and the image data of the previous frame to generate image features.

[0088] Specifically, in two adjacent frames of images, the two pixels with the smallest error value and the closest distance are likely to correspond to the same point on the object, so the positions of these two pixels can be used to reflect the motion information of the object, for example, the motion information includes motion displacement, and the motion displacement can be used to determine the motion direction and motion distance. Therefore, the motion information of each pixel between the current moment and the previous moment can be determined by matching the pixel points of the image data of the current frame (i.e., the image data collected at the current moment) and the image data of the previous frame (i.e., the image data collected at the previous moment), wherein these motion information are used to characterize the motion displacement of each pixel. It can be understood that the motion displacement of each pixel can be used to characterize the motion displacement of the vehicle, i.e., the motion trajectory of the vehicle. Therefore, according to the motion information of each pixel, an image feature that at least characterizes the change of the vehicle's motion trajectory can be generated.

[0089] In this way, when acquiring image features, the pixel matching method can be used to obtain image features that can characterize changes in the vehicle's motion trajectory, thereby ensuring that the subsequent detection neural network model can accurately perform tire blowout detection based on the image features.

[0090] See also Figure 4 and Fig. 9 In some embodiments, step 0421: performing pixel matching on the image data of the current frame and the image data of the previous frame to generate image features includes:

[0091] Step 04211: performing pixel matching on the image data of the current frame and the image data of the previous frame to generate a motion information matrix corresponding to the image data of the current frame. The characteristics of the motion information matrix include motion information corresponding to a plurality of pixel points, and the motion information is used to characterize the motion displacement of each pixel point.

[0092] Step 04212: Generate image features based on the motion information matrix.

[0093] Specifically, the motion information corresponding to each pixel in the image data of the current frame can be obtained by pixel matching, wherein the motion information is used to characterize the motion displacement of each pixel. Then, a motion information matrix is ​​generated based on the motion information corresponding to all pixels. It can be understood that the characteristics of the motion information matrix include the motion information corresponding to all pixels of the image data of the current frame, so that the motion information matrix can more comprehensively reflect the motion displacement of the vehicle between the current moment and the previous moment. Then, image features characterizing the changes in the vehicle's motion trajectory can be generated based on the motion information matrix.

[0094] In one embodiment, the KLT algorithm can be used to determine the motion information of each pixel. The basic principle of the KLT algorithm is to estimate the motion of feature points by tracking their position changes between consecutive frames. Its core idea is to use the pixel grayscale information in the local window to match the feature points, thereby achieving tracking of the feature points. The specific process is as follows:

[0095] First, the brightness of each pixel in the image data of the current frame and the image data of the previous frame, i.e., the grayscale value, needs to be calculated. Then, based on the preset error function and the brightness of each pixel, the image data of the current frame and the image data of the previous frame are matched pixel by pixel to obtain the position of the same pixel on different image data, thereby obtaining the motion information of each pixel. For example, the window second-order moment error function is defined for each pixel: , of which 2 w +1 is the window width, express t In the moment image ) position. Then, based on the pixel points of the previous frame image data, find the pixel points near the corresponding pixel points of the current frame image that can keep the value of the error function within a certain range and minimize it, that is, optimize the function , it can be understood that the pixel point that keeps the value of the error function within a certain range and minimizes it is the pixel point corresponding to the pixel point of the previous frame image in the current frame image, and the obtained That is, the position change of the pixel points between the two frames of images is recorded and used as motion information. After pixel matching is performed on each pixel point, the motion information matrix can be determined according to the motion information of each pixel point. The features in the motion information matrix can be used to reflect the motion information and position of each pixel point, thereby completing feature extraction.

[0096] In one embodiment, the features in the motion information matrix can be used to reflect the motion information and position of each pixel, so the image features can be directly determined based on the motion information matrix.

[0097] In another embodiment, due to the large number of pixels, there are many features in the motion information matrix. If the subsequent neural network directly processes the motion information matrix, it will cause the neural network to have a large amount of calculation. Therefore, at this time, the motion information matrix can also be subjected to data dimensionality reduction to obtain key features. Then, image features are generated based on each key feature and its corresponding position information. Among them, the key feature is the feature that can best represent the main structure and information of the original data. Data dimensionality reduction is to map high-dimensional data to a low-dimensional space through some kind of mapping or conversion to reduce the dimension of the data. Common dimensionality reduction methods include principal component analysis and linear discriminant analysis. The principal component analysis method finds the main direction of change of the data and projects the data to these directions, thereby reducing the dimension of the data. The linear discriminant analysis method finds a projection direction so that samples of the same type are as close as possible in the projected space, and samples of different types are as far away as possible.

[0098] Taking the principal component analysis method as an example of data dimensionality reduction, the principal component analysis method can determine the contribution rate of each feature based on the eigenvalue of each feature in the motion information matrix; the features with a contribution rate higher than the preset contribution rate threshold in the motion information matrix are determined as key features. Among them, the contribution rate refers to the explanatory power of a feature to explain the variability of the data. Features with high contribution rates can capture the main fluctuations or changes in the data. Therefore, the higher the contribution rate, the stronger its explanatory power to explain the data, and the more it can represent the image data.

[0099] In the principal component analysis method, the original data (i.e., the features in the motion information matrix) is standardized based on the eigenvalue of each feature in the motion information matrix to ensure that each feature has zero mean and unit variance. Next, the covariance matrix of the standardized data is calculated, and the eigenvalue decomposition of the covariance matrix is ​​performed to obtain the eigenvalues ​​and corresponding eigenvectors. Then, the eigenvalues ​​are sorted from large to small, and the proportion of each eigenvalue to the sum of all eigenvalues ​​is calculated. This proportion is the contribution rate of the feature corresponding to each eigenvalue.

[0100] The preset contribution rate threshold is the minimum contribution rate when the feature can represent the image data to a large extent. After obtaining the contribution rate corresponding to each feature, the feature with a contribution rate higher than the preset contribution rate threshold can be determined as the key feature (that is, the principal component), that is, the feature with a high contribution rate is retained as the key feature, thereby completing data dimensionality reduction.

[0101] Next, image features can be generated based on each key feature and its corresponding position information. For example, key point position vectors and key motion information vectors can be generated based on each key feature and its corresponding position information to obtain image features that can characterize image data. Among them, the key point position vector and the key motion information vector are both in the form of column vectors. The two vectors can be separate column vectors or spliced ​​into a whole column vector. The features in the key motion information vector can be The form can also be based on The form of the determined movement direction is not limited here.

[0102] In this way, when extracting image features, the image data of the current frame and the previous frame can be matched pixel by pixel to obtain the motion information of each pixel. It can be understood that the motion information of each pixel can be used to reflect the motion information of the feature point on the object corresponding to the pixel. Then, a motion information matrix can be generated based on the motion information of each pixel, and the motion information matrix can be reduced in data dimension, only the features with a higher contribution rate are retained, and image features are generated. At this time, the amount of image feature data is small, and the image feature can accurately reflect the motion information of the feature points on the object, thereby reducing the subsequent calculation amount and improving the subsequent calculation accuracy.

[0103] See also Figure 4 and Fig.10 In some embodiments, step 043: processing state features and image features to output tire blowout detection results includes:

[0104] Step 0431: concatenate the state features and image features corresponding to each moment to obtain the feature vector corresponding to each moment;

[0105] Step 0432: determining a feature matrix according to the feature vector corresponding to the current moment and the feature vector corresponding to each moment in a preset time period before the current moment;

[0106] Step 0433: inputting the feature matrix into the feature processing neural network model of the detection neural network model to obtain the probability of tire blowout and the probability of no tire blowout;

[0107] Step 0434: Output the tire blowout detection result based on the tire blowout probability and the tire non-blowout probability.

[0108] Specifically, the image features and state features of the above method are both in the form of column vectors, and in order to reduce the amount of calculation, even if other feature extraction methods are used, the final image features and state features will generally be in the form of column vectors. At this time, the state features and image features corresponding to each moment can be spliced ​​using the concat method or other methods to obtain the feature vector corresponding to each moment. Then, the feature matrix is ​​determined based on the feature vector corresponding to the current moment and the feature vector corresponding to each moment in the preset time period before the current moment, that is, the feature vectors in the current moment and the period before the current moment are spliced ​​to obtain the feature matrix. In some embodiments, the width of the feature matrix, that is, the length of the preset time period, is calibrated by actual vehicle tests.

[0109] Then, the feature matrix is ​​input into the feature processing neural network model of the detection neural network model, and the feature processing neural network model is used to determine the state change of the vehicle according to the feature matrix, so as to determine the probability of a tire blowout and the probability of no tire blowout according to the state change. That is, the feature processing neural network model can determine the state change of the vehicle during this period of time according to the image features and state features at each moment in the feature matrix, so as to determine whether the vehicle has a tire blowout, so as to output the probability of a tire blowout and the probability of no tire blowout. When the probability of a tire blowout is greater than the preset probability threshold of a tire blowout, it can be determined that the vehicle has a tire blowout, and the tire blowout detection result is a tire blowout, otherwise the tire blowout detection result is no tire blowout.

[0110] In one embodiment, the feature processing neural network model includes a bidirectional long short-term memory neural network. The bidirectional long short-term memory neural network (BiLSTM) is an improved recurrent neural network (RNN) specifically designed to process sequence data. BiLSTM captures the bidirectional dependency in the sequence by combining the outputs of the forward and backward LSTM networks. At this time, a bidirectional long short-term memory neural network consisting of an input gate, a forget gate, and an output gate can be constructed, and the aforementioned feature matrix can be input into the network to predict the tire blowout. The forget gate uses the tanh activation function, that is, the output gate uses the sigmoid activation function, and the output is the probability of being classified as a tire blowout and a tire not blown out.

[0111] In this way, the feature processing neural network model can combine the image features and state features at the current moment and within a preset time period to determine the state change of the vehicle during this period, thereby accurately judging whether the vehicle has a tire blowout based on the state change.

[0112] See also Figure 4 and Fig.11 In some embodiments, the tire blowout detection method includes:

[0113] Step 05: Obtain historical status data related to the vehicle;

[0114] Step 06: Input the historical status data into the training model to obtain the training operation results;

[0115] Step 07: Adjust the parameters of the training model based on the calculation results, and re-enter the step of inputting the historical state data into the training model according to the training model after the parameter adjustment to obtain the training calculation results, until the training model converges to obtain the detection neural network model.

[0116] Specifically, the detection neural network model can be obtained through training. First, the historical state data can be obtained, and in particular, the data before and after the tire blowout needs to be collected. Then the historical state data is input into a preset training model. Among them, the structure and training process of the training model are consistent with the steps in the above-mentioned tire blowout detection method. After the training model processes the historical state data, a training operation result will be obtained, and the training operation result can be whether there is a tire blowout, or the probability of a tire blowout and no tire blowout.

[0117] In the historical status data, the tire blowout situation (i.e., whether there is a tire blowout) of each data is known. Therefore, the parameters of the training model can be adjusted according to the actual tire blowout situation of each data and the calculation results. For example, the loss value is calculated according to the actual tire blowout situation and the calculation results, and then the parameters of the training model are adjusted according to the loss value. Then, the training is performed again according to the adjusted training model, that is, the historical status data is input into the adjusted training model to obtain the training calculation results. The training is repeated in this way until the training model converges, and the converged training model is the detection neural network model. Among them, the convergence standard of the training model can be that the loss value is less than the preset loss value threshold, or the number of training times reaches the preset number of training times.

[0118] After a sufficient training process, the various structures and parameters in the detection neural network model can be fixed, such as the structure and parameters of the aforementioned one-dimensional convolutional neural network and the parameters of the bidirectional long short-term memory neural network, and the back-propagation training link in the network can be cancelled. Then the input layer of the detection neural network model is connected to the vehicle motor speed, motor torque, wheel speed, tire pressure, and inertial sensor signal. Then, the detection neural network model can be officially put into use and continuously output tire blowout prediction results while the vehicle is driving.

[0119] In this way, it can be ensured that the detection neural network model obtained after training can obtain the changes in the state data before and after the tire blowout, so that the detection neural network model can quickly and accurately perform tire blowout detection based on the state data obtained in real time.

[0120] In some embodiments, the detection neural network model needs to be combined with image data and status data for tire blowout detection. Therefore, when training the training model, historical image data and historical status data can be obtained at the same time, and the training model can be trained based on the historical image data and historical status data. The training process refers to the above training process to obtain the detection neural network model. Then the input layer of the detection neural network model is connected to the real-time image taken by the front-view camera of the real vehicle and the vehicle motor speed, motor torque, wheel speed, tire pressure, and inertial sensor signal. Then, the detection neural network model can be officially put into use and continuously output the tire blowout prediction results during the driving process of the vehicle.

[0121] In this way, the present application learns, trains and classifies the feature vectors extracted from the data collected during the real-time operation of the whole vehicle, and trains the training model by integrating multimodal information, so that the training model can learn the changes of various data before and after the tire blowout from multiple angles and aspects, so that the detection neural network model obtained after the training model converges can obtain the real changes of the image data and status data before and after the tire blowout, thereby facilitating the detection neural network model to quickly and accurately detect the tire blowout based on the image data and status data obtained in real time.

[0122] See also Figure 4 and Fig.12 In some embodiments, the tire blowout detection method further comprises:

[0123] Step 08: Obtain the audio data collected by the vehicle;

[0124] Step 09: Process the audio data and status data based on the preset detection neural network model to output the tire blowout detection result.

[0125] Specifically, a tire blowout may produce a sharp sound, so the vehicle can also identify whether there is a tire blowout through audio data.

[0126] The detection neural network model can be trained to be a neural network model that can detect tire blowouts based on state data and audio data, or based on state data, image data, and audio data. That is, the detection neural network model can extract the features of each of the state data and audio data, or can extract the features of each of the state data, image data, and audio data. Then the detection neural network model accurately identifies the changes in each data based on the features. When the changes in each data are consistent with the actual changes before and after the tire blowout, the detection neural network model will immediately identify the situation, and the detection neural network model can determine that a tire blowout has occurred. When the changes in each data do not conform to the actual changes before and after the tire blowout, the detection neural network model can determine that the vehicle has not had a tire blowout at this time.

[0127] Therefore, during the operation of the vehicle, audio data can be collected through corresponding sensors, and then the audio data and status data (or status data, image data and audio data) are input into the detection neural network model together, so that the detection neural network model can extract corresponding features from each input data, and determine whether the vehicle has a tire blowout based on the features, thereby outputting the tire blowout detection result.

[0128] In this way, when performing tire blowout detection, the present application integrates multimodal information (audio data and status data or status data, image data and audio data), that is, combines information from multiple aspects and angles to determine whether a tire blowout has occurred, thereby achieving more comprehensive detection, thereby facilitating improving the accuracy of tire blowout detection.

[0129] See also Fig.13 The control device 100 of the embodiment of the present application includes a processor 20, a memory 30 and a computer program. The computer program is stored in the memory 30 and executed by the processor 20. The computer program includes instructions for executing the tire blowout detection method of any of the above embodiments.

[0130] See also Fig.14 The vehicle 200 of the embodiment of the present application includes the control device 100 of any one of the above-mentioned embodiments.

[0131] See also Fig.15 The embodiment of the present application further provides a computer-readable storage medium 300 on which a computer program 310 is stored. When the computer program 310 is executed by a processor 320, the steps of the tire blowout detection method of any of the above embodiments are implemented. For the sake of brevity, they are not repeated here.

[0132] See also Fig.16 The embodiment of the present application further provides a computer program product 400, including a computer program 410, wherein the computer program 410 includes instructions for executing the tire blowout detection method of any of the above embodiments.

[0133] In the description of this specification, the descriptions with reference to the terms "certain embodiments", "in an example", "exemplarily", etc., mean that the specific features, structures, materials or characteristics described in conjunction with the embodiments or examples are included in at least one embodiment or example of the present application. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine the different embodiments or examples described in this specification and the features of the different embodiments or examples, unless they are contradictory.

[0134] Any process or method description in a flowchart or otherwise described herein may be understood to represent a module, fragment or portion of code that includes one or more executable instructions for implementing the steps of a specific logical function or process, and the scope of the preferred embodiments of the present application includes alternative implementations in which functions may not be performed in the order shown or discussed, including performing functions in a substantially simultaneous manner or in the reverse order depending on the functions involved, which should be understood by technicians in the technical field to which the embodiments of the present application belong.

[0135] Although the embodiments of the present application have been shown and described above, it can be understood that the above embodiments are exemplary and cannot be understood as limitations on the present application. Ordinary technicians in this field can change, modify, replace and modify the above embodiments within the scope of the present application.

Claims

1. A tire blowout detection method, characterized in that: Applied to a vehicle, the method comprises: Acquiring status data of the vehicle; Processing the state data based on a preset detection neural network model to output a tire blowout detection result; The method further comprises: Acquire image data collected by the vehicle, wherein the image data includes motion image data collected by the vehicle; Processing the image data and the state data based on the preset detection neural network model to output a tire blowout detection result; The status data includes at least one of status data of wheels of the vehicle, status data of inertial sensors of the vehicle, and status data of motors connected to the wheels; The processing of the image data and the state data based on a preset detection neural network model to output a tire blowout detection result includes: Extracting features from the state data to obtain state features, where the state features are used to at least characterize changes in various state data; Extracting features from the image data to obtain image features, where the image features are used to at least characterize changes in the motion trajectory of the vehicle; The state feature and the image feature are processed to output the tire blowout detection result.

2. The tire blowout detection method according to claim 1, characterized in that: The wheel status data includes wheel speed signals and / or tire pressure signals; the inertial sensor status data includes acceleration signals and / or angular velocity signals in three directions of the inertial sensor; the motor status data includes speed signals and / or torque signals.

3. The tire blowout detection method according to claim 1, characterized in that: The detection neural network model includes a convolutional neural network model, the convolutional neural network includes an input layer, multiple convolutional layers, multiple pooling layers and an output layer, the convolutional layers and the pooling layers are arranged alternately, and the output layer corresponds to the pooling layer, and the feature extraction of the state data to obtain the state feature includes: The state data is input into the convolutional neural network model to obtain the state features.

4. The tire blowout detection method according to claim 1, characterized in that: The extracting features from the image data to obtain image features includes: The image data of the current frame is matched with the image data of the previous frame by pixels to generate image features.

5. The tire blowout detection method according to claim 4, characterized in that: The pixel matching of the image data of the current frame and the image data of the previous frame to generate image features includes: Perform pixel matching on the image data of the current frame and the image data of the previous frame to generate a motion information matrix corresponding to the image data of the current frame, wherein the feature of the motion information matrix includes motion information corresponding to a plurality of pixel points, and the motion information is used to characterize the motion displacement of each of the pixel points; Image features are generated according to the motion information matrix.

6. The tire blowout detection method according to claim 5, characterized in that: The pixel matching of the image data of the current frame and the image data of the previous frame to generate a motion information matrix corresponding to the image data of the current frame includes: Based on a preset error function and the brightness of each pixel, pixel matching is performed on the image data of the current frame and the image data of the previous frame to obtain motion information of each pixel; A motion information matrix is ​​determined according to the motion information of each pixel point.

7. The tire blowout detection method according to claim 5, characterized in that: The step of generating image features according to the motion information matrix includes: Performing data dimension reduction on the motion information matrix to obtain key features; Image features are generated according to each of the key features and their corresponding position information.

8. The tire blowout detection method according to claim 7, characterized in that: The performing data dimension reduction on the motion information matrix to obtain key features includes: Determine the contribution rate of each feature based on the eigenvalue of each feature in the motion information matrix; Determine a feature in the motion information matrix whose contribution rate is higher than a preset contribution rate threshold as a key feature.

9. The tire blowout detection method according to claim 5, characterized in that: The processing of the state feature and the image feature to output the tire blowout detection result includes: Concatenate the state features and image features corresponding to each moment to determine the feature vector corresponding to each moment; Determine a feature matrix according to the feature vector corresponding to the current moment and the feature vector corresponding to each moment in a preset time period before the current moment; Inputting the feature matrix into the feature processing neural network model of the detection neural network model to obtain the probability of tire blowout and the probability of no tire blowout; The tire blowout detection result is output based on the tire blowout probability and the tire non-blowout probability.

10. The tire blowout detection method according to claim 1, characterized in that: Also includes: Obtain historical status data related to the vehicle; Inputting the historical state data into a training model to obtain a training operation result; The parameters of the training model are adjusted based on the calculation results, and the step of inputting the historical state data into the training model is re-entered according to the training model after the parameter adjustment to obtain the training calculation results, until the training model converges to obtain the detection neural network model.

11. The tire blowout detection method according to claim 1, characterized in that: The method further comprises: Acquiring audio data collected by the vehicle; The audio data and the status data are processed based on a preset detection neural network model to output a tire blowout detection result.

12. A control device, characterized in that: include: Processor, memory; and A computer program, wherein the computer program is stored in the memory and executed by the processor, and the computer program includes instructions for executing the tire burst detection method according to any one of claims 1 to 11.

13. A vehicle, characterized in that: include: The control device as claimed in claim 12.

14. A non-volatile computer-readable storage medium containing a computer program, characterized in that: When the computer program is executed by a processor, the processor executes the tire burst detection method according to any one of claims 1 to 11.

15. A computer program product, characterized in that The invention comprises a computer program, wherein the computer program comprises instructions for executing the tire burst detection method according to any one of claims 1 to 11.

Images