Vehicle control state recognition method, equipment, device, medium and product

Abstract

The invention provides a vehicle control state recognition method, equipment and device, a medium and a product, and relates to the technical field of data processing.The method comprises the steps that to-be-recognized sensor time sequence data of a vehicle are obtained, and the to-be-recognized sensor time sequence data comprise one-dimensional time sequence data in all dimensions; determining a time sequence period of the time sequence data of the to-be-identified sensor based on the one-dimensional time sequence data on each dimension, and performing sliding window operation on the time sequence data of the to-be-identified sensor based on the time sequence period of the time sequence data of the to-be-identified sensor to obtain multiple pieces of time window data; and the multiple pieces of time window data are sequentially input into the vehicle control state classification model, and a vehicle control state recognition result output by the vehicle control state classification model is obtained. According to the invention, the precision and reliability of vehicle control state identification are effectively improved.

Description

Technical Field

[0001] The present invention relates to the field of data processing technology, and in particular to a vehicle control state recognition method, equipment, device, medium and product. Background Art

[0002] In the field of vehicle dynamic control and intelligent driving technology, accurate identification of the vehicle's handling state is crucial to enhancing driving safety, optimizing the driving experience, and realizing autonomous driving functions. Traditional vehicle handling state identification methods are mostly based on empirical rules and basic signal processing technology. These methods mainly focus on building experience-driven models and algorithms and using vehicle sensor data for analysis. However, these traditional methods have obvious limitations when dealing with complex and changing road conditions and driving environments.

[0003] Since these methods rely on fixed rules, they often lack the ability to flexibly adapt to real-time data, which may lead to a high rate of misjudgment in practical applications. Furthermore, when faced with high-dimensional data, these methods are usually difficult to effectively extract the temporal features and nonlinear relationships in the data, which to some extent restricts the accuracy and robustness of vehicle control state recognition. Summary of the invention

[0004] The present invention provides a vehicle control state recognition method, equipment, device, medium and product, which are used to solve the defect that the vehicle control state recognition method in the prior art has obvious limitations when dealing with complex and changeable road conditions and driving environments.

[0005] The present invention provides a method for identifying a vehicle control state, the method comprising: Acquire time series data of a sensor to be identified of the vehicle, wherein the time series data of the sensor to be identified includes one-dimensional time series data in each dimension; Determine the time series period of the to-be-identified sensor time series data based on the one-dimensional time series data on each dimension, and perform a sliding window operation on the to-be-identified sensor time series data based on the time series period of the to-be-identified sensor time series data to obtain a plurality of time window data; Inputting the plurality of time window data into a vehicle handling state classification model in sequence to obtain a vehicle handling state recognition result output by the vehicle handling state classification model; The vehicle handling state classification model is obtained in the following manner: Acquire sensor time series sample data of the vehicle, wherein the sensor time series sample data includes one-dimensional time series sample data in each dimension; Determine the time series period of the sensor time series sample data based on the one-dimensional time series sample data on each dimension, and perform a sliding window operation on the sensor time series sample data based on the time series period of the sensor time series sample data to obtain a plurality of time window sample data; Inputting the plurality of time window sample data into the pre-training model in sequence, and determining the total loss of the pre-training model according to the output of the pre-training model, wherein the total loss includes the cross entropy loss between the vehicle control prediction state label corresponding to the time window sample data and the vehicle control real state label and the autocorrelation loss between the vehicle control prediction state labels corresponding to the plurality of time window sample data; The parameters of the pre-trained model are updated according to the total loss until the pre-trained model converges to obtain a trained vehicle control state classification model.

[0006] According to a vehicle control state recognition method provided by the present invention, the determining the time series period of the sensor time series data to be recognized based on the one-dimensional time series data on each dimension includes: For the one-dimensional time series data on each dimension, determining the time series period of the one-dimensional time series data on each dimension; Determine that the maximum time series period among the time series periods of the one-dimensional time series data in all dimensions is the time series period of the sensor time series data to be identified.

[0007] According to a vehicle control state recognition method provided by the present invention, for the one-dimensional time series data on each dimension, the time series period of the one-dimensional time series data is obtained by: Determine the mean and standard deviation of the one-dimensional time series data; Determine a peak threshold and a trough threshold of the one-dimensional time series data based on the mean and standard deviation of the one-dimensional time series data; Based on the peak threshold and the trough threshold of the one-dimensional time series data, traverse the one-dimensional time series data to determine the candidate peaks and the candidate troughs in the one-dimensional time series data; Based on the candidate peaks and candidate troughs in the one-dimensional time series data, combined with a preset period restriction condition, determine an index range of a valid period of the one-dimensional time series data; Determine a period feature point within the index range of the valid period, wherein the period feature point includes a valid trough and a valid peak within the upper and lower limit index ranges corresponding to the valid trough; The time series period of the one-dimensional time series data is determined based on the time intervals between the effective troughs in all adjacent periodic feature points in the one-dimensional time series data.

[0008] According to a vehicle control state recognition method provided by the present invention, the pre-training model includes an attention mechanism layer, a long short-term memory network layer, a convolution-pooling layer and a fully connected layer, and the method includes: For each time window sample data, the current time window sample data is downsampled and normalized and then input into the attention mechanism layer and the convolution-pooling layer respectively, wherein in the attention mechanism layer, different attention weights are assigned to data in different dimensions in the time window sample data; Input the output result of the attention mechanism layer corresponding to the current time window sample data and the output result of the long short-term memory network layer corresponding to the previous time window sample data into the long short-term memory network layer; The output results of the convolution-pooling layer and the output results of the long short-term memory network layer corresponding to the current time window sample data are feature merged and input into the fully connected layer for classification calculation to obtain the vehicle control prediction state label corresponding to the current time window sample data.

[0009] According to a vehicle handling state recognition method provided by the present invention, the cross entropy loss of the pre-trained model is obtained in the following manner: Determine the vehicle control real state label corresponding to each of the time window sample data; Based on the vehicle control prediction state label and the vehicle control actual state label corresponding to each of the time window sample data, the cross entropy loss of the pre-trained model is obtained.

[0010] According to a vehicle control state recognition method provided by the present invention, the autocorrelation loss of the pre-trained model is obtained in the following manner: The vehicle control prediction state label corresponding to each of the time window sample data is applied to the autocorrelation function to obtain the autocorrelation loss of the pre-trained model.

[0011] The present invention also provides a vehicle control state recognition device, the device comprising: A first vehicle control state recognition module is used to obtain time series data of a sensor to be recognized of the vehicle, wherein the time series data of the sensor to be recognized includes one-dimensional time series data in each dimension; a second vehicle control state recognition module, configured to determine a time series period of the to-be-recognized sensor time series data based on the one-dimensional time series data on each dimension, and perform a sliding window operation on the to-be-recognized sensor time series data based on the time series period of the to-be-recognized sensor time series data to obtain a plurality of time window data; a third vehicle handling state recognition module, configured to sequentially input the plurality of time window data into a vehicle handling state classification model to obtain a vehicle handling state recognition result output by the vehicle handling state classification model; The vehicle handling state classification model is obtained in the following manner: Acquire sensor time series sample data of the vehicle, wherein the sensor time series sample data includes one-dimensional time series sample data in each dimension; Determine the time series period of the sensor time series sample data based on the one-dimensional time series sample data on each dimension, and perform a sliding window operation on the sensor time series sample data based on the time series period of the sensor time series sample data to obtain a plurality of time window sample data; Inputting the plurality of time window sample data into the pre-training model in sequence, and determining the total loss of the pre-training model according to the output of the pre-training model, wherein the total loss includes the cross entropy loss between the vehicle control prediction state label corresponding to the time window sample data and the vehicle control real state label and the autocorrelation loss between the vehicle control prediction state labels corresponding to the plurality of time window sample data; The parameters of the pre-trained model are updated according to the total loss until the pre-trained model converges to obtain a trained vehicle control state classification model.

[0012] The present invention also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, any of the above-mentioned vehicle control state identification methods is implemented.

[0013] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the vehicle control state identification method as described in any one of the above is implemented.

[0014] The present invention also provides a computer program product, comprising a computer program, wherein when the computer program is executed by a processor, the vehicle control state identification method as described above is implemented.

[0015] The vehicle control state identification method, equipment, device, medium and product provided by the present invention process sensor time series data by introducing time series period analysis and sliding window operation, and use the total loss including cross entropy loss and autocorrelation loss to train the vehicle control state classification model, which significantly improves the recognition accuracy and robustness of the vehicle control state classification model for the vehicle control state, enables it to more flexibly adapt to real-time data changes, capture time series characteristics and nonlinear relationships, and effectively improve the accuracy and reliability of vehicle control state identification. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0018] Figure 1 It is a flow chart of the vehicle control state identification method provided by the present invention; Figure 2 It is a schematic diagram of timing cycle inference provided by the present invention; Figure 3 It is a structural schematic diagram of a vehicle control state identification device provided by the present invention; Figure 4 It is a structural schematic diagram of the electronic device provided by the present invention. DETAILED DESCRIPTION

[0019] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the drawings of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0020] Figure 1 is a flow chart of a vehicle control state identification method provided by an embodiment of the present invention, such as Figure 1 As shown, the method includes: Step 110, obtaining time series data of a sensor to be identified of the vehicle, wherein the time series data of the sensor to be identified includes one-dimensional time series data in each dimension; First, the time series data measured by various sensors installed on the vehicle (such as accelerometers, gyroscopes, etc.) are collected as the sensor time series data to be identified.

[0021] Specifically, the sensor time series data to be identified contains data (such as acceleration data) of the vehicle in different dimensions (such as the x-axis, y-axis, and z-axis). These data usually exist in the form of a time series, that is, each time point has a corresponding measurement value. Therefore, the sensor time series data to be identified is divided into one-dimensional time series data in each dimension.

[0022] For example, a three-axis accelerometer installed on a vehicle can record the acceleration data of the vehicle on the x-axis, y-axis, and z-axis in real time. These acceleration data change over time, forming one-dimensional time series data on the x-axis, y-axis, and z-axis.

[0023] Step 120, determining the time series period of the to-be-identified sensor time series data based on the one-dimensional time series data on each dimension, and performing a sliding window operation on the to-be-identified sensor time series data based on the time series period of the to-be-identified sensor time series data to obtain a plurality of time window data; After obtaining the one-dimensional time series data on each dimension, period recognition is performed on the one-dimensional time series data on each dimension, and then the period recognition results are used to determine the time period of the sensor time series data to be identified. For example, the maximum time period among the time periods of the one-dimensional time series data on all dimensions is determined as the time period of the sensor time series data to be identified, or the average time period of the time periods of the one-dimensional time series data on all dimensions is determined as the time period of the sensor time series data to be identified.

[0024] Next, the sliding window size of the sliding window operation is set according to the length of the timing cycle. Generally, the sliding window size should be able to cover at least one complete timing cycle to ensure that the data in the sliding window can fully reflect the periodic characteristics. Furthermore, the step length is also set according to the length of the timing cycle. The step length setting is usually related to the sliding window size. For example, the time step of the sliding window is set to a timing cycle, and 1 / 6 of the time step is set as the step length.

[0025] Here, the sensor time series data to be identified is divided into multiple continuous time window data through a sliding window operation. Each sliding window contains a fixed number of data points, and there is a certain overlap between the sliding windows to ensure the continuity of the data.

[0026] Step 130, inputting the plurality of time window data into a vehicle handling state classification model in sequence, and obtaining a vehicle handling state recognition result output by the vehicle handling state classification model; It should be noted here that since the sensor time series data to be identified is multi-dimensional data, that is, the time window data under each sliding window is multi-dimensional data, in this embodiment, the time window data under each sliding window is defined as a tensor data with a shape of (N, T, F) and input into the vehicle control state classification model.

[0027] Among them, N is the number of sampling points in the time window data, T is the data length of the time window data, and F refers to the number of features of the time window data, which is related to the dimension of the time window data. For example, if the time window data contains the acceleration data of the vehicle on the x-axis, y-axis, and z-axis, then its feature number is 3.

[0028] The time window data under each sliding window is input into the vehicle handling state classification model. The vehicle handling state classification recognition model will determine the vehicle handling state recognition result based on the time window data under all sliding windows.

[0029] Specifically, in this embodiment, the vehicle handling state classification model is obtained by: Acquire sensor time series sample data of the vehicle, wherein the sensor time series sample data includes one-dimensional time series sample data in each dimension; Determine the time series period of the sensor time series sample data based on the one-dimensional time series sample data on each dimension, and perform a sliding window operation on the sensor time series sample data based on the time series period of the sensor time series sample data to obtain a plurality of time window sample data; Inputting the plurality of time window sample data into the pre-training model in sequence, and determining the total loss of the pre-training model according to the output of the pre-training model, wherein the total loss includes the cross entropy loss between the vehicle control prediction state label corresponding to the time window sample data and the vehicle control real state label and the autocorrelation loss between the vehicle control prediction state labels corresponding to the plurality of time window sample data; The parameters of the pre-trained model are updated according to the total loss until the pre-trained model converges to obtain a trained vehicle control state classification model.

[0030] Here, first collect the time series data measured by various sensors (such as accelerometers, gyroscopes, etc.) installed on the vehicle under known vehicle control states as sensor time series sample data. For example, obtain the acceleration time series data measured by the vehicle's accelerometer when the vehicle passes a curve at a speed of 100 km / h, the acceleration time series data measured by the vehicle's accelerometer when the vehicle passes a curve at a speed of 80 km / h, the acceleration time series data measured by the vehicle's accelerometer when the vehicle turns at a steering angle of 100°, and the acceleration time series data measured by the vehicle's accelerometer when the vehicle changes lanes at a speed of 130 km / h, and use the corresponding labels to identify the acceleration time series data measured by the vehicle's accelerometer under each vehicle control state, and obtain the acceleration time series sample data under each known vehicle control state.

[0031] Then, for each sensor time series sample data under a known vehicle control state, the sensor time series sample data is segmented into a plurality of time window sample data. Here, the sensor time series sample data is segmented in the same manner as the above sensor time series data to be identified.

[0032] After dividing the sensor time series sample data into multiple time window sample data, the time window data under each sliding window is input into a pre-trained model. This pre-trained model can be composed of basic neural networks such as convolutional neural network (CNN), recurrent neural network (RNN) or long short-term memory network (LSTM).

[0033] It should be noted that since these time window sample data are continuous, they contain the sensor data of the vehicle at different time points. Therefore, by inputting multiple time window sample data under each known vehicle control state into the pre-trained model in sequence, the model can learn and understand the data change rules under different vehicle control states.

[0034] Specifically, the pre-trained model outputs the vehicle handling prediction state label for each time window sample data. In order to evaluate the accuracy of these prediction results, the cross entropy loss between the vehicle handling prediction state label and the vehicle handling true state label is calculated. Cross entropy loss is a commonly used loss function used to measure the difference between two probability distributions.

[0035] In addition to the cross entropy loss, the present embodiment also calculates the autocorrelation loss between the vehicle control prediction state labels corresponding to the sample data of multiple time windows. The autocorrelation loss is used to measure the consistency and stability of the pre-trained model when predicting continuous time window data.

[0036] Based on this, the total loss ,in, is the cross entropy loss, is the autocorrelation loss, is a weight coefficient used to adjust the impact of autocorrelation loss. Through this total loss, when optimizing the model, not only the accuracy on the given task is taken into account, but also the autocorrelation characteristics of the output.

[0037] Here, you can use optimization techniques such as back propagation algorithm and gradient descent to update the parameters of the pre-trained model. By adjusting parameters such as model weights and biases, the total loss value is minimized. When the total loss value of the model gradually decreases and tends to be stable, the model is considered to have converged, and a trained vehicle control state classification model can be obtained.

[0038] The vehicle control state recognition method proposed in this embodiment introduces time series period analysis and sliding window operation to process sensor time series data, and uses the total loss including cross entropy loss and autocorrelation loss to train the vehicle control state classification model, which significantly improves the recognition accuracy and robustness of the vehicle control state classification model for the vehicle control state, enables it to more flexibly adapt to real-time data changes, capture time series characteristics and nonlinear relationships, and effectively improve the accuracy and reliability of vehicle control state recognition.

[0039] It should be noted that each implementation method of the present application can be freely combined, the order can be changed, or it can be executed separately, and does not need to rely on or depend on a fixed execution order.

[0040] In some embodiments, determining the time series period of the to-be-identified sensor time series data based on the one-dimensional time series data on each dimension includes: For the one-dimensional time series data on each dimension, determining the time series period of the one-dimensional time series data on each dimension; Determine that the maximum time series period among the time series periods of the one-dimensional time series data in all dimensions is the time series period of the sensor time series data to be identified.

[0041] It should be noted that the one-dimensional time series data in each dimension may show different periodic characteristics. In this embodiment, for the one-dimensional time series data in each dimension, you can first find the peaks and troughs, which represent the extreme states of the vehicle in this dimension. By analyzing the peaks and troughs of the one-dimensional time series data in each dimension, the time series period of the data in this dimension can be determined.

[0042] After determining the timing period in each dimension respectively, compare the timing periods in all dimensions to find the maximum timing period. Since the maximum timing period can ensure that the periodic changes of data in any dimension can be completely contained in one or more time windows, in this embodiment, the maximum timing period will be used as the timing period of the sensor timing data to be identified.

[0043] In some embodiments, for the one-dimensional time series data on each dimension, the time series period of the one-dimensional time series data is obtained by: Determine the mean and standard deviation of the one-dimensional time series data; Determine a peak threshold and a trough threshold of the one-dimensional time series data based on the mean and standard deviation of the one-dimensional time series data; Based on the peak threshold and the trough threshold of the one-dimensional time series data, traverse the one-dimensional time series data to determine the candidate peaks and the candidate troughs in the one-dimensional time series data; Based on the candidate peaks and candidate troughs in the one-dimensional time series data, combined with a preset period restriction condition, determine an index range of a valid period of the one-dimensional time series data; Determine a period feature point within the index range of the valid period, wherein the period feature point includes a valid trough and a valid peak within the upper and lower limit index ranges corresponding to the valid trough; The time series period of the one-dimensional time series data is determined based on the time intervals between the effective troughs in all adjacent periodic feature points in the one-dimensional time series data.

[0044] Here, in order to reduce the impact of noise data, before determining the mean and standard deviation, the one-dimensional time series data can be preprocessed by denoising and normalization, and then the mean and standard deviation of all values ​​after the preprocessing operation are calculated.

[0045] Next, after determining the mean and standard deviation, set the peak threshold and trough threshold based on the two means and standard deviations. The peak threshold is usually set as the mean plus one or more standard deviations, while the trough threshold is set as the mean minus one or more standard deviations, for example, refer to the following formula: ; ; in, represents the peak threshold, Represents the trough threshold.

[0046] Then, the one-dimensional time series data is traversed according to the previously set peak threshold and trough threshold to detect candidate peaks and candidate troughs. In one example, the one-dimensional time series data can be regarded as a discrete time signal, a signal graph is drawn, and the peak threshold and trough threshold are identified in the signal graph to detect candidate peaks and candidate troughs.

[0047] refer to Figure 2 As shown in the figure, the one-dimensional time series data is regarded as a discrete time signal, and a signal graph is drawn. When drawing a signal graph, the horizontal axis represents the signal index, which indicates the order of each data point in the signal, and the vertical axis represents the amplitude of the signal, which reflects the strength or size of each data point in the signal. After the peak threshold and the trough threshold are identified in the signal graph, based on the signal significance condition in the current signal graph, the candidate troughs (the blue points in the figure) are first identified based on the trough threshold, and then the candidate peaks on the left side of each candidate trough (the red points in the figure) and the candidate peaks on the right side (the green points in the figure) are identified.

[0048] Furthermore, after the candidate peaks and candidate troughs are determined, the effective period index range is further determined in combination with the preset period restriction condition. Here, the preset period restriction condition includes but is not limited to: if any candidate peak is located on the left side of the first candidate trough, it is marked, such as marked as feature point A, and its index is recorded, or if any candidate peak is located on the right side of the first candidate trough, it is marked, such as marked as feature point E, and its index is recorded.

[0049] Based on this, the index range of the effective period is determined as follows: ; in, is a list of indices containing all valid trough points; express The number of valid valley points in .

[0050] It should be understood here that, in this embodiment, the index of the first valid period is numbered starting from the starting index 0. Therefore, the set {0} is used to represent the starting index of the first valid period.

[0051] Set {i | i = 1, 2, ..., - 1} contains the numbers from 1 to - 1. The set { - 1} indicates the end index of the last valid period.

[0052] After determining the index range of the valid period, further determine the period characteristic points of each valid period. Specifically, the period characteristic points include valid troughs and valid peaks within the upper and lower limit index ranges corresponding to the valid troughs. For each valid trough, find a valid peak within the upper and lower limit index ranges of the valid trough, so that a series of period characteristic points A (the last valid peak on the left side of the current valid trough), C (the current valid trough), and E (the first valid peak on the right side of the current valid trough) can be determined.

[0053] Specifically, for each valid trough, the upper and lower limits are as follows: ; ; When i = 0, start = 0, indicating that the starting index of the first valid period is 0. For other valid periods, the starting index of the current valid period is the valid trough index of the previous valid period. and the effective trough index of the current effective period The average value of .

[0054] Among them, when i= - 1, end=n, indicating that the end index of the last valid cycle is n, where n refers to the total length of the one-dimensional time series data. In other words, the end index of the last valid cycle is the end data index of the entire one-dimensional time series data. For other valid cycles, the end index of the current valid cycle is the valid trough index of the current valid cycle. and the effective trough index of the next effective period The average value of .

[0055] After extracting multiple periodic feature points, the time intervals between adjacent effective troughs are calculated, and the average value of these time intervals is taken as the timing period.

[0056] The vehicle control state recognition method proposed in this embodiment first determines the time series period of the one-dimensional time series data in each dimension, and then selects the maximum time series period therein as the time series period of the entire sensor time series data to be identified. It can provide a reasonable time frame for subsequent sliding window operations, thereby helping subsequent models to more accurately identify the vehicle control state.

[0057] In some embodiments, the pre-trained model includes an attention mechanism layer, a long short-term memory network layer, a convolution-pooling layer, and a fully connected layer, and the method includes: For each time window sample data, the current time window sample data is downsampled and normalized and then input into the attention mechanism layer and the convolution-pooling layer respectively, wherein in the attention mechanism layer, different attention weights are assigned to data in different dimensions in the time window sample data; Input the output result of the attention mechanism layer corresponding to the current time window sample data and the output result of the long short-term memory network layer corresponding to the previous time window sample data into the long short-term memory network layer; The output results of the convolution-pooling layer and the output results of the long short-term memory network layer corresponding to the current time window sample data are feature merged and input into the fully connected layer for classification calculation to obtain the vehicle control prediction state label corresponding to the current time window sample data.

[0058] Here, the input layer of the pre-trained model is connected to the input of the attention mechanism layer and the input of the convolution-pooling layer, the output of the attention mechanism layer is connected to the input of the long short-term memory network layer, the output of the convolution-pooling layer and the output of the long short-term memory network layer are connected to the input of the fully connected layer, and the output of the fully connected layer is connected to the output layer of the pre-trained model.

[0059] In this embodiment, when using multiple time window sample data to train the pre-trained model, for each time window sample data, downsampling is first performed to reduce the number of data points and reduce computational complexity while retaining key information. Then, standardization is performed to ensure that all data are in the same scale range.

[0060] The processed time window sample data is input into the attention mechanism layer and the convolution-pooling layer respectively. Among them, the attention mechanism layer can assign different weights to the data in different dimensions in the time window sample data, that is, the subsequent model can pay more attention to the features that are more important for the recognition of the vehicle control state. Here, the convolution-pooling layer refers to the combination of the convolution layer and the global average pooling layer. The convolution layer extracts local features in the data by applying convolution filters, while the global average pooling layer compresses the feature map output by the convolution layer.

[0061] After the sample data of the current time window is processed and output by the attention mechanism layer, it is input into the long short-term memory network layer together with the output of the sample data of the previous time window processed by the long short-term memory network layer for processing. In this way, the ability of the long short-term memory network layer is utilized to capture the long-term dependencies in time series data. At the same time, the attention-weighted features of the sample data of the current time window and the output of the sample data of the previous time window processed by the long short-term memory network layer are combined to provide richer contextual information.

[0062] Finally, the output of the long short-term memory network layer and the output of the convolution-pooling layer corresponding to the sample data of the current time window are merged, and the merged features are input into the fully connected layer for classification calculation, and finally the vehicle control prediction state label corresponding to the sample data of the current time window is obtained.

[0063] The vehicle control state recognition method proposed in this embodiment improves the accuracy and robustness of vehicle control prediction by combining the attention mechanism layer, the long short-term memory network layer, the convolution-pooling layer, etc. to build a multi-level pre-training model. By giving different attention weights to data of different dimensions through the attention mechanism layer, the model can capture key features more accurately; the long short-term memory network layer helps to capture long-term dependencies in time series data; the convolution-pooling layer can extract local features; finally, the fully connected layer is responsible for integrating these features and performing classification predictions.

[0064] In some embodiments, the cross entropy loss of the pre-trained model is obtained by: Determine the vehicle control real state label corresponding to each of the time window sample data; Based on the vehicle control prediction state label and the vehicle control actual state label corresponding to each of the time window sample data, the cross entropy loss of the pre-trained model is obtained.

[0065] It should be understood that each time window sample data has a corresponding vehicle control real state label. These labels are usually determined in advance through experiments and represent the actual control state of the vehicle in the time window.

[0066] After the pre-trained model processes the sample data of each time window, it will output a vehicle control prediction state label, which represents the predicted probability of the vehicle being in various control states. In this embodiment, for each time window sample data, the predicted probability distribution label is compared with the vehicle control real state label. In vehicle control state identification, the vehicle control real state label is usually a one-hot encoding vector (one-hot vector), in which only the element corresponding to the real control state is 1, and the rest are 0. Then, based on the vehicle control prediction state label and the vehicle control real state label corresponding to each time window sample data, the cross entropy loss of the pre-trained model is obtained: ; in, is the number of sample data in the time window, Refers to l The vehicle control prediction state label corresponding to the sample data of the time window, Refers to l The actual vehicle control state label corresponding to the sample data in each time window.

[0067] The vehicle handling state recognition method proposed in this embodiment can gradually learn how to more accurately predict the vehicle handling state by minimizing the cross entropy loss during the training process.

[0068] In some embodiments, the autocorrelation loss of the pre-trained model is obtained by: The vehicle control prediction state label corresponding to each of the time window sample data is applied to the autocorrelation function to obtain the autocorrelation loss of the pre-trained model.

[0069] Here, the autocorrelation function is used to calculate the correlation of time series data at different time lags. In this embodiment, the vehicle control prediction state label corresponding to each time window sample data output by the pre-training model is regarded as a time series, and its autocorrelation value is calculated.

[0070] Specifically, for each time window sample data, the corresponding vehicle handling prediction state label can be extracted. One or more time lags (e.g., lags of 1, 2, ..., N time steps) are selected. For each lag, the autocorrelation coefficient between the vehicle handling prediction state label of the current time window sample data and the vehicle handling prediction state label lagged by a certain time step is calculated. Repeat the above steps to obtain the autocorrelation value of each time window sample data at different lags.

[0071] ; in, is the amount of hysteresis, For the j The target autocorrelation value at the lag is j The difference between the average values ​​of the autocorrelation values ​​of the sample data in all time windows at the lag amount.

[0072] The vehicle control state recognition method proposed in this embodiment can better capture the long-term dependencies in time series data by minimizing the autocorrelation loss during the training process, thereby improving the continuity and accuracy of the prediction.

[0073] Based on any of the above embodiments, the present invention also provides a model training device, Figure 3 It is a structural diagram of the model training provided by the present invention, such as Figure 3 As shown, the device comprises: A first vehicle control state recognition module 310 is used to obtain time series data of a sensor to be recognized of the vehicle, wherein the time series data of the sensor to be recognized includes one-dimensional time series data in each dimension; A second vehicle control state recognition module 320 is used to determine the time series period of the sensor time series data to be identified based on the one-dimensional time series data on each dimension, and perform a sliding window operation on the sensor time series data to be identified based on the time series period of the sensor time series data to be identified to obtain multiple time window data; A third vehicle handling state recognition module 330, configured to sequentially input the plurality of time window data into a vehicle handling state classification model to obtain a vehicle handling state recognition result output by the vehicle handling state classification model; The vehicle handling state classification model is obtained in the following manner: Acquire sensor time series sample data of the vehicle, wherein the sensor time series sample data includes one-dimensional time series sample data in each dimension; Determine the time series period of the sensor time series sample data based on the one-dimensional time series sample data on each dimension, and perform a sliding window operation on the sensor time series sample data based on the time series period of the sensor time series sample data to obtain a plurality of time window sample data; Inputting the plurality of time window sample data into the pre-training model in sequence, and determining the total loss of the pre-training model according to the output of the pre-training model, wherein the total loss includes the cross entropy loss between the vehicle control prediction state label corresponding to the time window sample data and the vehicle control real state label and the autocorrelation loss between the vehicle control prediction state labels corresponding to the plurality of time window sample data; The parameters of the pre-trained model are updated according to the total loss until the pre-trained model converges to obtain a trained vehicle control state classification model.

[0074] The device provided by the embodiment of the present invention processes sensor time series data by introducing time series period analysis and sliding window operation, and uses the total loss including cross entropy loss and autocorrelation loss to train the vehicle control state classification model, thereby significantly improving the recognition accuracy and robustness of the vehicle control state classification model for the vehicle control state, enabling the model to more flexibly adapt to real-time data changes, capture time series characteristics and nonlinear relationships, and effectively improve the accuracy and reliability of vehicle control state recognition.

[0075] The vehicle control state identification device described in this embodiment and the vehicle control state identification method provided by the present invention described above can refer to each other, and will not be described in detail here.

[0076] Figure 4 An example of a physical structure diagram of an electronic device is shown in FIG. Figure 4 As shown, the electronic device may include: a processor 410, a communications interface 420, a memory 430 and a communication bus 440, wherein the processor 410, the communications interface 420 and the memory 430 communicate with each other via the communication bus 440. The processor 410 may call the logic instructions in the memory 430 to execute the vehicle control state identification method, which includes: Acquire time series data of a sensor to be identified of the vehicle, wherein the time series data of the sensor to be identified includes one-dimensional time series data in each dimension; Determine the time series period of the to-be-identified sensor time series data based on the one-dimensional time series data on each dimension, and perform a sliding window operation on the to-be-identified sensor time series data based on the time series period of the to-be-identified sensor time series data to obtain a plurality of time window data; Inputting the plurality of time window data into a vehicle handling state classification model in sequence to obtain a vehicle handling state recognition result output by the vehicle handling state classification model; The vehicle handling state classification model is obtained in the following manner: Acquire sensor time series sample data of the vehicle, wherein the sensor time series sample data includes one-dimensional time series sample data in each dimension; Determine the time series period of the sensor time series sample data based on the one-dimensional time series sample data on each dimension, and perform a sliding window operation on the sensor time series sample data based on the time series period of the sensor time series sample data to obtain a plurality of time window sample data; Inputting the plurality of time window sample data into the pre-training model in sequence, and determining the total loss of the pre-training model according to the output of the pre-training model, wherein the total loss includes the cross entropy loss between the vehicle control prediction state label corresponding to the time window sample data and the vehicle control real state label and the autocorrelation loss between the vehicle control prediction state labels corresponding to the plurality of time window sample data; The parameters of the pre-trained model are updated according to the total loss until the pre-trained model converges to obtain a trained vehicle control state classification model.

[0077] In addition, the logic instructions in the above-mentioned memory 430 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when it is sold or used as an independent product. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc. Various media that can store program codes.

[0078] On the other hand, the present invention further provides a computer program product, the computer program product includes a computer program, the computer program can be stored in a non-transitory computer-readable storage medium, when the computer program is executed by a processor, the computer can execute the vehicle control state identification method provided by the above methods, the method includes: Acquire time series data of a sensor to be identified of the vehicle, wherein the time series data of the sensor to be identified includes one-dimensional time series data in each dimension; Determine the time series period of the to-be-identified sensor time series data based on the one-dimensional time series data on each dimension, and perform a sliding window operation on the to-be-identified sensor time series data based on the time series period of the to-be-identified sensor time series data to obtain a plurality of time window data; Inputting the plurality of time window data into a vehicle handling state classification model in sequence to obtain a vehicle handling state recognition result output by the vehicle handling state classification model; The vehicle handling state classification model is obtained in the following manner: Acquire sensor time series sample data of the vehicle, wherein the sensor time series sample data includes one-dimensional time series sample data in each dimension; Determine the time series period of the sensor time series sample data based on the one-dimensional time series sample data on each dimension, and perform a sliding window operation on the sensor time series sample data based on the time series period of the sensor time series sample data to obtain a plurality of time window sample data; Inputting the plurality of time window sample data into the pre-training model in sequence, and determining the total loss of the pre-training model according to the output of the pre-training model, wherein the total loss includes the cross entropy loss between the vehicle control prediction state label corresponding to the time window sample data and the vehicle control real state label and the autocorrelation loss between the vehicle control prediction state labels corresponding to the plurality of time window sample data; The parameters of the pre-trained model are updated according to the total loss until the pre-trained model converges to obtain a trained vehicle control state classification model.

[0079] In another aspect, the present invention further provides a non-transitory computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the vehicle control state identification method provided by the above methods is implemented, and the method includes: Acquire time series data of a sensor to be identified of the vehicle, wherein the time series data of the sensor to be identified includes one-dimensional time series data in each dimension; Determine the time series period of the to-be-identified sensor time series data based on the one-dimensional time series data on each dimension, and perform a sliding window operation on the to-be-identified sensor time series data based on the time series period of the to-be-identified sensor time series data to obtain a plurality of time window data; Inputting the plurality of time window data into a vehicle handling state classification model in sequence to obtain a vehicle handling state recognition result output by the vehicle handling state classification model; The vehicle handling state classification model is obtained in the following manner: Acquire sensor time series sample data of the vehicle, wherein the sensor time series sample data includes one-dimensional time series sample data in each dimension; Determine the time series period of the sensor time series sample data based on the one-dimensional time series sample data on each dimension, and perform a sliding window operation on the sensor time series sample data based on the time series period of the sensor time series sample data to obtain a plurality of time window sample data; Inputting the plurality of time window sample data into the pre-training model in sequence, and determining the total loss of the pre-training model according to the output of the pre-training model, wherein the total loss includes the cross entropy loss between the vehicle control prediction state label corresponding to the time window sample data and the vehicle control real state label and the autocorrelation loss between the vehicle control prediction state labels corresponding to the plurality of time window sample data; The parameters of the pre-trained model are updated according to the total loss until the pre-trained model converges to obtain a trained vehicle control state classification model.

[0080] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative labor.

[0081] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0082] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical coding feature diagrams therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for identifying vehicle control status, characterized in that: include: Acquire time series data of a sensor to be identified of the vehicle, wherein the time series data of the sensor to be identified includes one-dimensional time series data in each dimension; Determine the time series period of the to-be-identified sensor time series data based on the one-dimensional time series data on each dimension, and perform a sliding window operation on the to-be-identified sensor time series data based on the time series period of the to-be-identified sensor time series data to obtain a plurality of time window data; Inputting the plurality of time window data into a vehicle handling state classification model in sequence to obtain a vehicle handling state recognition result output by the vehicle handling state classification model; The vehicle handling state classification model is obtained in the following manner: Acquire sensor time series sample data of the vehicle, wherein the sensor time series sample data includes one-dimensional time series sample data in each dimension; Determine the time series period of the sensor time series sample data based on the one-dimensional time series sample data on each dimension, and perform a sliding window operation on the sensor time series sample data based on the time series period of the sensor time series sample data to obtain a plurality of time window sample data; Inputting the plurality of time window sample data into the pre-training model in sequence, and determining the total loss of the pre-training model according to the output of the pre-training model, wherein the total loss includes the cross entropy loss between the vehicle control prediction state label corresponding to the time window sample data and the vehicle control real state label and the autocorrelation loss between the vehicle control prediction state labels corresponding to the plurality of time window sample data; The parameters of the pre-trained model are updated according to the total loss until the pre-trained model converges to obtain a trained vehicle control state classification model.

2. The vehicle control state recognition method according to claim 1, characterized in that: The determining the time series period of the to-be-identified sensor time series data based on the one-dimensional time series data on each dimension includes: For the one-dimensional time series data on each dimension, determining the time series period of the one-dimensional time series data on each dimension; Determine that the maximum time series period among the time series periods of the one-dimensional time series data in all dimensions is the time series period of the sensor time series data to be identified.

3. The vehicle control state recognition method according to claim 2, characterized in that: For the one-dimensional time series data on each dimension, the time series period of the one-dimensional time series data is obtained in the following manner: Determine the mean and standard deviation of the one-dimensional time series data; Determine a peak threshold and a trough threshold of the one-dimensional time series data based on the mean and standard deviation of the one-dimensional time series data; Based on the peak threshold and the trough threshold of the one-dimensional time series data, traverse the one-dimensional time series data to determine the candidate peaks and the candidate troughs in the one-dimensional time series data; Based on the candidate peaks and candidate troughs in the one-dimensional time series data, and in combination with a preset period restriction condition, determine an index range of a valid period of the one-dimensional time series data; Determine a period feature point within the index range of the valid period, wherein the period feature point includes a valid trough and a valid peak within the upper and lower limit index ranges corresponding to the valid trough; The time series period of the one-dimensional time series data is determined based on the time intervals between the effective troughs in all adjacent periodic feature points in the one-dimensional time series data.

4. The vehicle control state recognition method according to claim 1, characterized in that: The pre-training model includes an attention mechanism layer, a long short-term memory network layer, a convolution-pooling layer and a fully connected layer, and the method includes: For each time window sample data, the current time window sample data is downsampled and normalized and then input into the attention mechanism layer and the convolution-pooling layer respectively, wherein in the attention mechanism layer, different attention weights are assigned to data in different dimensions in the time window sample data; Input the output result of the attention mechanism layer corresponding to the current time window sample data and the output result of the long short-term memory network layer corresponding to the previous time window sample data into the long short-term memory network layer; The output results of the convolution-pooling layer and the output results of the long short-term memory network layer corresponding to the current time window sample data are feature merged and input into the fully connected layer for classification calculation to obtain the vehicle control prediction state label corresponding to the current time window sample data.

5. The vehicle control state recognition method according to claim 4, characterized in that: The cross entropy loss of the pre-trained model is obtained in the following way: Determine the vehicle control real state label corresponding to each of the time window sample data; Based on the vehicle control prediction state label and the vehicle control actual state label corresponding to each of the time window sample data, the cross entropy loss of the pre-trained model is obtained.

6. The vehicle control state recognition method according to claim 4, characterized in that: The autocorrelation loss of the pre-trained model is obtained in the following way: The vehicle control prediction state label corresponding to each of the time window sample data is applied to the autocorrelation function to obtain the autocorrelation loss of the pre-trained model.

7. A vehicle control state recognition device, characterized in that: include: A first vehicle control state recognition module is used to obtain time series data of a sensor to be recognized of the vehicle, wherein the time series data of the sensor to be recognized includes one-dimensional time series data in each dimension; a second vehicle control state recognition module, configured to determine a time series period of the to-be-recognized sensor time series data based on the one-dimensional time series data on each dimension, and perform a sliding window operation on the to-be-recognized sensor time series data based on the time series period of the to-be-recognized sensor time series data to obtain a plurality of time window data; a third vehicle handling state recognition module, configured to sequentially input the plurality of time window data into a vehicle handling state classification model to obtain a vehicle handling state recognition result output by the vehicle handling state classification model; The vehicle handling state classification model is obtained in the following manner: Acquire sensor time series sample data of the vehicle, wherein the sensor time series sample data includes one-dimensional time series sample data in each dimension; Determine the time series period of the sensor time series sample data based on the one-dimensional time series sample data on each dimension, and perform a sliding window operation on the sensor time series sample data based on the time series period of the sensor time series sample data to obtain a plurality of time window sample data; Inputting the plurality of time window sample data into the pre-training model in sequence, and determining the total loss of the pre-training model according to the output of the pre-training model, wherein the total loss includes the cross entropy loss between the vehicle control prediction state label corresponding to the time window sample data and the vehicle control real state label and the autocorrelation loss between the vehicle control prediction state labels corresponding to the plurality of time window sample data; The parameters of the pre-trained model are updated according to the total loss until the pre-trained model converges to obtain a trained vehicle control state classification model.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the program, the vehicle control state identification method as described in any one of claims 1 to 6 is implemented.

9. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the vehicle control state identification method as claimed in any one of claims 1 to 6 is implemented.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the vehicle control state identification method as claimed in any one of claims 1 to 6 is implemented.

Images