Vehicle handling state recognition method, device, apparatus, medium and product

Abstract

The application provides a vehicle operation state recognition method, device, apparatus, medium and product, and relates to the technical field of data processing. The method comprises the following steps: acquiring time sequence data of a sensor to be recognized of a vehicle, wherein the time sequence data of the sensor to be recognized comprises one-dimensional time sequence data in each dimension; determining a time sequence period of the time sequence data of the sensor to be recognized based on the one-dimensional time sequence data in each dimension, and performing a sliding window operation on the time sequence data of the sensor to be recognized based on the time sequence period of the time sequence data of the sensor to be recognized, to obtain a plurality of time window data; and inputting the plurality of time window data into a vehicle operation state classification model in sequence, to obtain a vehicle operation state recognition result output by the vehicle operation state classification model. The application effectively improves the accuracy and reliability of vehicle operation state recognition.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of data processing, and in particular to a vehicle operation state recognition method, device, apparatus, medium and product. BACKGROUND

[0002] In the field of vehicle dynamic control and intelligent driving technology, accurately recognizing the operation state of a vehicle is crucial for enhancing driving safety, optimizing driving experience, and realizing automatic driving functions. Traditional vehicle operation state recognition methods are mostly based on empirical rules and basic signal processing techniques. These methods mainly focus on constructing experience-driven models and algorithms, and use vehicle sensor data for analysis. However, these traditional methods have obvious limitations when dealing with complex and variable road conditions and driving environments.

[0003] Since these methods rely on fixed rules, they often lack flexibility in adapting to real-time data, which can lead to high error rates in actual applications. Furthermore, when faced with high-dimensional data, these methods often struggle to effectively extract temporal features and nonlinear relationships in the data, which to some extent restricts the accuracy and robustness of vehicle operation state recognition. SUMMARY

[0004] The present application provides a vehicle operation state recognition method, device, apparatus, medium and product to solve the problem that the existing vehicle operation state recognition method has obvious limitations when dealing with complex and variable road conditions and driving environments.

[0005] The present application provides a vehicle operation state recognition method, which comprises:

[0006] Obtaining sensor time series data to be recognized of a vehicle, the sensor time series data to be recognized comprising one-dimensional time series data in each dimension;

[0007] Determining a time series period of the sensor time series data to be recognized based on the one-dimensional time series data in each dimension, and performing a sliding window operation on the sensor time series data to be recognized based on the time series period of the sensor time series data to be recognized, to obtain a plurality of time window data;

[0008] Inputting the plurality of time window data into a vehicle operation state classification model in sequence to obtain a vehicle operation state recognition result output by the vehicle operation state classification model;

[0009] The vehicle operation state classification model is obtained by the following method:

[0010] Obtaining sensor time series sample data of a vehicle, the sensor time series sample data comprising one-dimensional time series sample data in each dimension;

[0011] determine a timing cycle of the sensor timing sample data based on the one-dimensional timing sample data on each dimension, and perform a sliding window operation on the sensor timing sample data based on the timing cycle of the sensor timing sample data to obtain a plurality of time window sample data;

[0012] input the plurality of time window sample data into a pre-training model in sequence, and determine a total loss of the pre-training model according to an output of the pre-training model, wherein the total loss includes a cross-entropy loss between a vehicle control predicted state label corresponding to the time window sample data and a vehicle control real state label, and an autocorrelation loss between vehicle control predicted state labels corresponding to the plurality of time window sample data;

[0013] perform parameter updating on the pre-training model according to the total loss until the pre-training model converges, to obtain a trained vehicle control state classification model.

[0014] According to the vehicle control state recognition method provided by the application, the timing cycle of the sensor timing sample data to be recognized is determined based on the one-dimensional timing data on each dimension, which comprises:

[0015] For the one-dimensional timing data on each dimension, the timing cycle of the one-dimensional timing data on each dimension is determined.

[0016] The maximum timing cycle in the timing cycles of the one-dimensional timing data on all dimensions is determined as the timing cycle of the sensor timing sample data to be recognized.

[0017] According to the vehicle control state recognition method provided by the application, for the one-dimensional timing data on each dimension, the timing cycle of the one-dimensional timing data is obtained by the following way:

[0018] The mean and standard deviation of the one-dimensional timing data are determined.

[0019] Based on the mean and standard deviation of the one-dimensional timing data, the peak threshold and the trough threshold of the one-dimensional timing data are determined.

[0020] Based on the peak threshold and the trough threshold of the one-dimensional timing data, the one-dimensional timing data is traversed to determine the candidate peaks and the candidate troughs in the one-dimensional timing data.

[0021] Based on the candidate peaks and the candidate troughs in the one-dimensional timing data, the index range of the effective cycle of the one-dimensional timing data is determined in combination with a preset cycle limitation condition.

[0022] determining a period feature point in the index range of the effective period, wherein the period feature point includes an effective trough and an effective peak in the upper and lower limit index range corresponding to the effective trough;

[0023] determining a timing period of the one-dimensional timing data based on a time interval between effective troughs in all adjacent period feature points in the one-dimensional timing data.

[0024] According to the vehicle operation state recognition method provided by the application, the pre-training model includes an attention mechanism layer, a long short-term memory network layer, a convolution-pooling layer and a full connection layer, and the method comprises:

[0025] For each time window sample data, the current time window sample data is input into the attention mechanism layer and the convolution-pooling layer after being down-sampled and standardized, wherein in the attention mechanism layer, different dimensions of data in the time window sample data are given different attention weights.

[0026] The output result of the attention mechanism layer corresponding to the current time window sample data and the output result of the long short-term memory network layer corresponding to the previous time window sample data are input into the long short-term memory network layer.

[0027] The output result of the convolution-pooling layer corresponding to the current time window sample data and the output result of the long short-term memory network layer are merged and input into the full connection layer for classification calculation to obtain a vehicle operation prediction state label corresponding to the current time window sample data.

[0028] According to the vehicle operation state recognition method provided by the application, the cross-entropy loss of the pre-training model is obtained in the following way:

[0029] determining a vehicle operation real state label corresponding to each time window sample data;

[0030] Based on the vehicle operation prediction state label and the vehicle operation real state label corresponding to each time window sample data, the cross-entropy loss of the pre-training model is obtained.

[0031] According to the vehicle operation state recognition method provided by the application, the autocorrelation loss of the pre-training model is obtained in the following way:

[0032] Based on the vehicle operation prediction state label corresponding to each time window sample data, the autocorrelation loss of the pre-training model is obtained by applying a self-correlation function.

[0033] The application also provides a vehicle operation state recognition device, which comprises:

[0034] The first vehicle operation state identification module is configured to obtain sensor time sequence data of a vehicle to be identified, the sensor time sequence data including one-dimensional time sequence data in each dimension.

[0035] The second vehicle operation state identification module is configured to determine a time sequence period of the sensor time sequence data to be identified based on the one-dimensional time sequence data in each dimension, and perform a sliding window operation on the sensor time sequence data to be identified based on the time sequence period of the sensor time sequence data, to obtain a plurality of time window data.

[0036] The third vehicle operation state identification module is configured to sequentially input the plurality of time window data into a vehicle operation state classification model, to obtain a vehicle operation state identification result output by the vehicle operation state classification model.

[0037] The vehicle operation state classification model is obtained by the following method:

[0038] The sensor time sequence sample data of the vehicle is obtained, the sensor time sequence sample data including one-dimensional time sequence sample data in each dimension.

[0039] The time sequence period of the sensor time sequence sample data is determined based on the one-dimensional time sequence sample data in each dimension, and a sliding window operation is performed on the sensor time sequence sample data based on the time sequence period of the sensor time sequence sample data, to obtain a plurality of time window sample data.

[0040] The plurality of time window sample data is sequentially input into a pre-training model, and a total loss of the pre-training model is determined according to an output of the pre-training model, wherein the total loss includes a cross-entropy loss between vehicle operation prediction state labels corresponding to the time window sample data and vehicle operation real state labels, and a self-correlation loss between vehicle operation prediction state labels corresponding to the plurality of time window sample data.

[0041] The pre-training model is updated according to the total loss until the pre-training model converges, to obtain a trained vehicle operation state classification model.

[0042] The application further provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the vehicle operation state identification method according to any one of the above.

[0043] The application further provides a non-transitory computer readable storage medium, which stores a computer program executable by a processor, and the computer program implements the vehicle operation state identification method according to any one of the above when executed by the processor.

[0044] The application further provides a computer program product comprising a computer program which, when executed by a processor, implements any one of the vehicle operation state recognition methods described above.

[0045] The vehicle operation state recognition method, device, apparatus, medium and product provided by the application improve the recognition accuracy and robustness of the vehicle operation state classification model for vehicle operation states by introducing time series cycle analysis and sliding window operation to process sensor time series data, and training the vehicle operation state classification model using a total loss containing cross-entropy loss and autocorrelation loss, so that the model can more flexibly adapt to real-time data changes, capture time series characteristics and nonlinear relationships, and effectively improve the accuracy and reliability of vehicle operation state recognition. BRIEF DESCRIPTION OF DRAWINGS

[0046] In order to more clearly illustrate the technical solutions in the application or prior art, the following will briefly introduce the drawings needed to be used in the embodiments or prior art description. Obviously, the drawings in the following description are some embodiments of the application, and other drawings can be obtained by those skilled in the art without creative labor.

[0047] Figure 1 is a flowchart of the vehicle operation state recognition method provided by the application;

[0048] Figure 2 is a time series cycle inference diagram provided by the application;

[0049] Figure 3 is a structural diagram of the vehicle operation state recognition apparatus provided by the application;

[0050] Figure 4 is a structural diagram of the electronic device provided by the application. DETAILED DESCRIPTION

[0051] In order to make the objects, technical solutions and advantages of the application clearer, the technical solutions in the application will be described clearly and completely below with reference to the drawings in the application. Obviously, the described embodiments are some embodiments of the application, not all embodiments. Based on the embodiments in the application, all other embodiments obtained by those skilled in the art without creative labor are within the protection scope of the application.

[0052] Figure 1 is a flowchart of the vehicle operation state recognition method provided by the embodiment of the application, as shown in Figure 1 The method comprises:

[0053] In step 110, the sensor time series data to be identified of the vehicle is obtained, which includes one-dimensional time series data in each dimension.

[0054] Firstly, the time series data measured by various sensors (such as accelerometers, gyroscopes, etc.) installed on the vehicle is collected as the sensor time series data to be identified.

[0055] Specifically, the sensor time series data to be identified contains data (such as acceleration data) of the vehicle in different dimensions (such as x-axis, y-axis, z-axis), which usually exists in the form of time series, i.e., there is a corresponding measurement value at each time point. Therefore, the sensor time series data to be identified is divided into one-dimensional time series data in each dimension.

[0056] For example, a three-axis accelerometer installed on the vehicle can record acceleration data of the vehicle in the x-axis, y-axis and z-axis in real time. These acceleration data change over time and form one-dimensional time series data in the x-axis, y-axis and z-axis.

[0057] In step 120, the time series period of the sensor time series data to be identified is determined based on the one-dimensional time series data in each dimension, and a sliding window operation is performed on the sensor time series data to be identified based on the time series period of the sensor time series data to be identified, to obtain a plurality of time window data.

[0058] After obtaining the one-dimensional time series data in each dimension, the one-dimensional time series data in each dimension is subjected to period identification, and then the time series period of the sensor time series data to be identified is determined based on the period identification results of the one-dimensional time series data in each dimension. For example, the maximum time series period in the time series periods of the one-dimensional time series data in all dimensions is determined as the time series period of the sensor time series data to be identified, or the average time series period in the time series periods of the one-dimensional time series data in all dimensions is determined as the time series period of the sensor time series data to be identified.

[0059] Then, the size of the sliding window of the sliding window operation is set according to the length of the time series period. In general, the size of the sliding window should cover at least one complete time series period to ensure that the data in the sliding window can fully reflect the periodic characteristics. Further, the length of the step is also set according to the length of the time series period, and the setting of the step is usually related to the size of the sliding window. For example, the time step of the sliding window is set to one time series period, and the step is set to 1 / 6 of the time step.

[0060] Here, the sensor time series data to be identified is divided into a plurality of continuous time window data through the sliding window operation, each sliding window contains a fixed number of data points, and there is a certain overlap between the sliding windows to ensure the continuity of the data.

[0061] In step 130, the plurality of time window data is sequentially input into the vehicle operation state classification model to obtain a vehicle operation state recognition result output by the vehicle operation state classification model.

[0062] It should be noted that, since the to-be-recognized sensor time series data is multi-dimensional data, i.e., the time window data under each sliding window is multi-dimensional data, in this embodiment, the time window data under each sliding window is defined as a tensor data with a shape of (N, T, F) and is input into the vehicle operation state classification model.

[0063] wherein N is the number of sampling points in the time window data, T is the data length of the time window data, and F represents the number of features of the time window data, which is related to the dimension of the time window data. For example, if the time window data includes acceleration data of a vehicle on the x-axis, y-axis and z-axis, the number of features is 3.

[0064] The time window data under each sliding window is input into the vehicle operation state classification model, and the vehicle operation state classification recognition model determines the vehicle operation state recognition result according to the time window data under all sliding windows.

[0065] Specifically, in this embodiment, the vehicle operation state classification model is obtained by the following method:

[0066] obtaining sensor time series sample data of a vehicle, the sensor time series sample data including one-dimensional time series sample data in each dimension;

[0067] determining a time series period of the sensor time series sample data based on the one-dimensional time series sample data in each dimension, and performing a sliding window operation on the sensor time series sample data based on the time series period of the sensor time series sample data to obtain a plurality of time window sample data;

[0068] sequentially inputting the plurality of time window sample data into a pre-training model, and determining a total loss of the pre-training model according to an output of the pre-training model, wherein the total loss includes a cross-entropy loss between vehicle operation prediction state labels corresponding to the time window sample data and vehicle operation real state labels, and an autocorrelation loss between vehicle operation prediction state labels corresponding to the plurality of time window sample data;

[0069] updating parameters of the pre-training model according to the total loss until the pre-training model converges to obtain a trained vehicle operation state classification model.

[0070] Here, first, the time series data measured by various sensors (such as accelerometers, gyroscopes, etc.) installed on the vehicle under known vehicle control states are collected as sensor time series sample data. For example, the acceleration time series data measured by the accelerometer of the vehicle when the vehicle passes a curve at a speed of 100 km / h, the acceleration time series data measured by the accelerometer of the vehicle when the vehicle passes a curve at a speed of 80 km / h, the acceleration time series data measured by the accelerometer of the vehicle when the vehicle turns at a steering angle of 100°, and the acceleration time series data measured by the accelerometer of the vehicle when the vehicle changes lanes at a speed of 130 km / h are obtained, and the acceleration time series data measured by the accelerometer of the vehicle under each vehicle control state is identified using the corresponding label, obtaining the acceleration time series sample data under each known vehicle control state.

[0071] Then, for the sensor time series sample data under each known vehicle control state, the sensor time series sample data is divided into multiple time window sample data. Here, the division method of the sensor time series sample data is the same as the division method of the above-mentioned to-be-recognized sensor time series data.

[0072] After dividing the sensor time series sample data into multiple time window sample data, the time window data under each sliding window is input into a pre-trained model, which can be composed of a convolutional neural network (CNN), a recurrent neural network (RNN), or a long short-term memory network (LSTM) basic neural network.

[0073] It should be noted that since these time window sample data are continuous, they contain sensor data of the vehicle at different time points. Therefore, by sequentially inputting multiple time window sample data under each known vehicle control state into the pre-trained model, the model can learn and understand the data variation law under different vehicle control states.

[0074] Specifically, the pre-trained model outputs the vehicle control prediction state label for each time window sample data. In order to evaluate the accuracy of these prediction results, the cross-entropy loss between the vehicle control prediction state label and the vehicle control true state label is calculated. Cross-entropy loss is a commonly used loss function for measuring the difference between two probability distributions.

[0075] In addition, in addition to the cross-entropy loss, the autocorrelation loss between the vehicle control prediction state labels corresponding to the multiple time window sample data is also calculated in this embodiment. The autocorrelation loss is used to measure the consistency and stability of the pre-trained model when predicting consecutive time window data.

[0076] Based on this, the total loss where, is the cross-entropy loss, For autocorrelation loss, These are the weighting coefficients used to adjust for the impact of autocorrelation loss. This total loss ensures that, when optimizing the model, not only accuracy on a given task is considered, but also the autocorrelation characteristics of the output.

[0077] Here, optimization techniques such as backpropagation and gradient descent can be used to update the parameters of the pre-trained model. By adjusting parameters such as model weights and biases, the total loss value is minimized. When the total loss value of the model gradually decreases and tends to stabilize, the model is considered to have converged, and the trained vehicle handling state classification model can be obtained.

[0078] The vehicle control state recognition method proposed in this embodiment introduces time-series periodic analysis and sliding window operation to process sensor time-series data, and uses a total loss including cross-entropy loss and autocorrelation loss to train the vehicle control state classification model. This significantly improves the vehicle control state classification model's recognition accuracy and robustness, enabling it to adapt more flexibly to real-time data changes, capture time-series features and nonlinear relationships, and effectively improve the accuracy and reliability of vehicle control state recognition.

[0079] It should be noted that each implementation method of this application can be freely combined, rearranged, or executed individually, and does not need to rely on or depend on a fixed execution order.

[0080] In some embodiments, determining the time period of the sensor time data to be identified based on the one-dimensional time series data in each dimension includes:

[0081] For the one-dimensional time series data in each dimension, determine the time series period of the one-dimensional time series data in each dimension;

[0082] The longest time period among the time periods of the one-dimensional time series data across all dimensions is determined as the time period of the sensor time series data to be identified.

[0083] It should be noted that one-dimensional time-series data in each dimension may exhibit different periodicity characteristics. In this embodiment, for one-dimensional time-series data in each dimension, peaks and troughs can be identified first; these points represent the extreme states of the vehicle in that dimension. By analyzing the peaks and troughs of the one-dimensional time-series data in each dimension, the time-series period of the data in that dimension can be determined.

[0084] After determining the time series period of each dimension respectively, compare the time series periods of all dimensions to find the maximum time series period, since the maximum time series period can ensure that the periodic change of data of any dimension can be completely contained in one or more time windows, therefore, in this embodiment, the maximum time series period will be used as the time series period of the sensor time series data to be identified.

[0085] In some embodiments, for the one-dimensional time series data of each dimension, the time series period of the one-dimensional time series data is obtained by the following way:

[0086] Determine the mean and standard deviation of the one-dimensional time series data;

[0087] Based on the mean and standard deviation of the one-dimensional time series data, determine the peak threshold and trough threshold of the one-dimensional time series data;

[0088] Based on the peak threshold and trough threshold of the one-dimensional time series data, traverse the one-dimensional time series data to determine the candidate peaks and candidate troughs in the one-dimensional time series data;

[0089] Based on the candidate peaks and candidate troughs in the one-dimensional time series data, determine the index range of the effective period of the one-dimensional time series data in combination with a preset period limit condition;

[0090] Determine the period feature points within the index range of the effective period, wherein the period feature points include effective troughs and effective peaks within the upper and lower limit index ranges corresponding to the effective troughs;

[0091] Based on the time interval between the effective troughs in all adjacent period feature points in the one-dimensional time series data, determine the time series period of the one-dimensional time series data.

[0092] Here, in order to reduce the influence of noise data, before determining the mean and standard deviation, the one-dimensional time series data can be preprocessed by denoising and normalization, and then the mean and standard deviation of all values after the preprocessing operation are calculated.

[0093] Then, after determining the mean and standard deviation, the peak threshold and trough threshold are set according to the two means and standard deviations. The peak threshold is usually set to the value of the mean plus one or more standard deviations, and the trough threshold is set to the value of the mean minus one or more standard deviations, for example, refer to the following formula:

[0094] ;

[0095] ;

[0096] Wherein, represents the peak threshold, representative trough threshold.

[0097] After that, the one-dimensional time series data is traversed according to the previously set peak threshold and trough threshold to detect candidate peaks and candidate troughs. In one example, the one-dimensional time series data can be regarded as a discrete time signal, a signal graph is drawn, and the peak threshold and trough threshold are marked in the signal graph to detect candidate peaks and candidate troughs.

[0098] Reference Figure 2 As shown, the one-dimensional time series data is regarded as a discrete time signal, a signal graph is drawn, and when drawing the signal graph, the horizontal coordinate axis represents the signal index, which indicates the sequential arrangement of each data point in the signal, and the vertical coordinate axis represents the amplitude of the signal, which reflects the intensity or size of each data point in the signal. After marking the peak threshold and trough threshold in the signal graph, based on the signal significance condition in the current signal graph, first, based on the trough threshold, candidate troughs are marked (such as blue points in the figure), and then candidate peaks to the left of each candidate trough (such as red points in the figure) and candidate peaks to the right of each candidate trough (such as green points in the figure) are marked.

[0099] Further, after determining the candidate peaks and candidate troughs, further combining a preset period limit condition to determine the index range of the effective period. Here, the preset period limit condition includes but is not limited to: if there is any candidate peak to the left of the first candidate trough, marking it, such as marking it as feature point A, and recording its index, or if there is any candidate peak to the right of the first candidate trough, marking it, such as marking it as feature point E, and recording its index.

[0100] Based on this, the index range of the determined effective period is as follows:

[0101] ;

[0102] wherein, is an index list containing all the effective trough points; represents the number of effective trough points in .

[0103] Here, it should be understood that in the embodiment, the index of the first effective period starts from the starting index 0. Therefore, the set {0} is used to represent the starting index of the first effective period.

[0104] The set {i | i = 1, 2,..., n - 1} contains all integers from 1 to n - 1. The set {i | i = 1, 2,..., n - 1} contains all integers from 1 to n - 1. The set {i | i = 1, 2,..., n - 1} contains all integers from 1 to n - 1. -1} indicates the end index of the last valid period.

[0105] After determining the index range of the effective period, the periodic feature points of each effective period are further determined. Specifically, the periodic feature points include effective troughs and effective peaks within the upper and lower limit index ranges corresponding to the effective troughs. For each effective trough, effective peaks are searched within the upper and lower limit index ranges of that effective trough, thus determining a series of periodic feature points A (the last effective peak to the left of the current effective trough), C (the current effective trough), and E (the first effective peak to the right of the current effective trough).

[0106] Specifically, for each valid trough, its upper and lower limits are as follows:

[0107] ;

[0108] ;

[0109] When i = 0, start = 0, indicating that the starting index of the first valid period is 0. For other valid periods, the starting index of the current valid period is the valid trough index of the previous valid period. And the effective trough index of the current effective period The average value.

[0110] Where, when i= When the index is -1, end=n, indicating that the end index of the last valid period is n. Here, n represents the total length of the one-dimensional time series data. In other words, the end index of the last valid period is the end index of the entire one-dimensional time series data. For other valid periods, the ending index end of the current valid period is the valid trough index of the current valid period. And the effective trough index of the next effective period The average value.

[0111] After extracting multiple periodic feature points, the time interval between adjacent effective valleys is calculated, and the average of these time intervals is taken as the time series period.

[0112] The vehicle control state recognition method proposed in this embodiment first determines the time period of the one-dimensional time series data in each dimension, and then selects the largest time period as the time period of the entire sensor time series data to be identified. This can provide a reasonable time frame for subsequent sliding window operations, thereby helping the subsequent model to more accurately identify the vehicle control state.

[0113] In some embodiments, the pre-trained model includes an attention mechanism layer, a long short-term memory network layer, a convolutional-pooling layer, and a fully connected layer, and the method includes:

[0114] For each time window sample data, the current time window sample data is down-sampled and standardized and then input to the attention mechanism layer and the convolution-pooling layer, respectively. In the attention mechanism layer, different dimensions of data in the time window sample data are given different attention weights.

[0115] The output result of the attention mechanism layer corresponding to the current time window sample data and the output result of the long short-term memory network layer corresponding to the previous time window sample data are input to the long short-term memory network layer.

[0116] The output result of the convolution-pooling layer corresponding to the current time window sample data and the output result of the long short-term memory network layer are merged and then input to the full connection layer for classification calculation to obtain the vehicle operation prediction state label corresponding to the current time window sample data.

[0117] Here, the input layer of the pre-trained model is connected with the input of the attention mechanism layer and the input of the convolution-pooling layer, the output of the attention mechanism layer is connected with the input of the long short-term memory network layer, the output of the convolution-pooling layer and the output of the long short-term memory network layer are connected with the input of the full connection layer, and the output of the full connection layer is connected with the output layer of the pre-trained model.

[0118] In this embodiment, when the pre-trained model is trained using multiple time window sample data, for each time window sample data, first, down-sampling processing is performed to reduce the number of data points, reduce the calculation complexity, and retain key information. Then, standardization processing is performed to ensure that all data are in the same scale range.

[0119] The processed time window sample data is input to the attention mechanism layer and the convolution-pooling layer, respectively. The attention mechanism layer can assign different weights to different dimensions of data in the time window sample data, that is, the subsequent model can pay more attention to the features that are more important for vehicle operation state recognition. Here, the convolution-pooling layer refers to the combination of the convolution layer and the global average pooling layer. The convolution layer extracts local features in the data by applying a convolution filter, and the global average pooling layer compresses the feature map output by the convolution layer.

[0120] The output of the current time window sample data after being processed by the attention mechanism layer is input into the long short-term memory network layer together with the output of the previous time window sample data after being processed by the long short-term memory network layer, so as to utilize the ability of the long short-term memory network layer to capture long-term dependencies in time series data, and combine the attention weighted features of the current time window sample data and the output of the previous time window sample data after being processed by the long short-term memory network layer to provide more rich context information.

[0121] Finally, the output of the long short-term memory network layer corresponding to the current time window sample data and the output of the convolution-pooling layer are merged, and the merged features are input into the fully connected layer for classification calculation, and finally the vehicle control prediction state label corresponding to the current time window sample data is obtained.

[0122] The vehicle control state recognition method proposed in the embodiment improves the accuracy and robustness of vehicle control prediction by combining the attention mechanism layer, the long short-term memory network layer, the convolution-pooling layer and other layers to build a multi-level pre-training model. The model can more accurately capture key features by giving different attention weights to data of different dimensions through the attention mechanism layer. The long short-term memory network layer helps to capture long-term dependencies in time series data. The convolution-pooling layer can extract local features. Finally, the fully connected layer is responsible for integrating and classifying the features.

[0123] In some embodiments, the cross-entropy loss of the pre-training model is obtained in the following way:

[0124] Determine the vehicle control real state label corresponding to each of the time window sample data;

[0125] Based on the vehicle control prediction state label and the vehicle control real state label corresponding to each of the time window sample data, the cross-entropy loss of the pre-training model is obtained.

[0126] It should be understood that each time window sample data has a corresponding vehicle control real state label. These labels are usually determined in advance through experiments, indicating the actual control state of the vehicle in the time window.

[0127] After processing the sample data for each time window, the pre-trained model outputs a vehicle handling prediction state label, which represents the predicted probability of the vehicle being in various handling states. In this embodiment, for each time window sample data, the predicted probability distribution label is compared with the vehicle handling true state label. In vehicle handling state recognition, the vehicle handling true state label is typically a one-hot encoded vector, where only the element corresponding to the true handling state is 1, and the rest are 0. Then, based on the vehicle handling prediction state label and the vehicle handling true state label corresponding to each time window sample data, the cross-entropy loss of the pre-trained model is obtained.

[0128] ;

[0129] in, The number of sample data points within the time window. Refers to the first l Vehicle handling prediction status labels corresponding to sample data within a time window Refers to the first l The vehicle control real-state label corresponding to the sample data of each time window.

[0130] The vehicle control state recognition method proposed in this embodiment can gradually learn how to predict the vehicle control state more accurately by minimizing the cross-entropy loss during the training process.

[0131] In some embodiments, the autocorrelation loss of the pre-trained model is obtained in the following manner:

[0132] Based on the vehicle handling prediction state label corresponding to each time window sample data, the autocorrelation function is applied to obtain the autocorrelation loss of the pre-trained model.

[0133] Here, the autocorrelation function is used to calculate the correlation of time series data at different time lags. In this embodiment, the vehicle handling prediction state label corresponding to each time window sample data output by the pre-trained model is regarded as a time series, and its autocorrelation value is calculated.

[0134] Specifically, for each time window of sample data, the corresponding vehicle handling prediction state label can be extracted. One or more time lags can be selected (e.g., lags of 1, 2, ..., N time steps). For each lag, the autocorrelation coefficient between the vehicle handling prediction state label of the current time window sample data and the vehicle handling prediction state label lags by a certain time step is calculated. The above steps are repeated to obtain the autocorrelation value of each time window sample data under different lags.

[0135] ;

[0136] wherein, is the number of lags, is the target autocorrelation value at the j th lag, and j is the average of the autocorrelation values of all time window sample data at the

[0137] The vehicle control state recognition method provided in the embodiment can better capture the long-term dependence in time series data by minimizing the autocorrelation loss in the training process, thereby improving the continuity and accuracy of the prediction.

[0138] Based on any of the above embodiments, the present application also provides a model training device, Figure 3 is a structural schematic diagram of model training provided by the present application, as Figure 3 shown, the device comprises:

[0139] The first vehicle control state recognition module 310 is configured to obtain sensor time series data of a vehicle to be recognized, wherein the sensor time series data to be recognized comprises one-dimensional time series data in each dimension.

[0140] The second vehicle control state recognition module 320 is configured to determine a time series period of the sensor time series data to be recognized based on the one-dimensional time series data in each dimension, and perform a sliding window operation on the sensor time series data to be recognized based on the time series period of the sensor time series data to be recognized, to obtain a plurality of time window data.

[0141] The third vehicle control state recognition module 330 is configured to sequentially input the plurality of time window data into a vehicle control state classification model, to obtain a vehicle control state recognition result output by the vehicle control state classification model.

[0142] The vehicle control state classification model is obtained in the following manner:

[0143] The sensor time series sample data of the vehicle comprises one-dimensional time series sample data in each dimension.

[0144] The time series period of the sensor time series sample data is determined based on the one-dimensional time series sample data in each dimension, and a sliding window operation is performed on the sensor time series sample data based on the time series period of the sensor time series sample data, to obtain a plurality of time window sample data.

[0145] Multiple time window sample data are sequentially input into the pre-trained model, and the total loss of the pre-trained model is determined based on the output of the pre-trained model. The total loss includes the cross-entropy loss between the vehicle control prediction state label and the vehicle control real state label corresponding to the time window sample data and the autocorrelation loss between the vehicle control prediction state labels corresponding to multiple time window sample data.

[0146] The parameters of the pre-trained model are updated based on the total loss until the pre-trained model converges, resulting in a trained vehicle handling state classification model.

[0147] The apparatus provided in this invention introduces time-series periodic analysis and sliding window operation to process sensor time-series data, and uses a total loss including cross-entropy loss and autocorrelation loss to train a vehicle control state classification model. This significantly improves the vehicle control state classification model's recognition accuracy and robustness, enabling it to adapt more flexibly to real-time data changes, capture time-series features and nonlinear relationships, and effectively improve the accuracy and reliability of vehicle control state recognition.

[0148] The vehicle control state recognition device described in this embodiment can be referred to in correspondence with the vehicle control state recognition method provided by the present invention described above, and will not be described in detail here.

[0149] Figure 4 An example is a schematic diagram of the physical structure of an electronic device, such as... Figure 4 As shown, the electronic device may include: a processor 410, a communications interface 420, a memory 430, and a communication bus 440, wherein the processor 410, the communications interface 420, and the memory 430 communicate with each other via the communication bus 440. The processor 410 can call logical instructions in the memory 430 to execute a vehicle control state recognition method, which includes:

[0150] Acquire the timing data of the vehicle's sensors to be identified, wherein the timing data of the sensors to be identified includes one-dimensional timing data in each dimension;

[0151] The timing period of the sensor time series data to be identified is determined based on the one-dimensional time series data in each dimension, and a sliding window operation is performed on the sensor time series data to be identified based on the timing period of the sensor time series data to be identified to obtain multiple time window data.

[0152] The data from the multiple time windows are sequentially input into the vehicle control state classification model to obtain the vehicle control state recognition result output by the vehicle control state classification model.

[0153] The vehicle operation state classification model is obtained by the following manner:

[0154] Obtain sensor time sequence sample data of the vehicle, the sensor time sequence sample data comprising one-dimensional time sequence sample data in each dimension;

[0155] Determine a time sequence period of the sensor time sequence sample data based on the one-dimensional time sequence sample data in each dimension, and perform a sliding window operation on the sensor time sequence sample data based on the time sequence period of the sensor time sequence sample data to obtain a plurality of time window sample data;

[0156] Input the plurality of time window sample data to a pre-training model in sequence, and determine a total loss of the pre-training model according to an output of the pre-training model, wherein the total loss comprises a cross-entropy loss between a vehicle operation prediction state label and a vehicle operation real state label corresponding to the time window sample data, and an autocorrelation loss between vehicle operation prediction state labels corresponding to the plurality of time window sample data;

[0157] Perform parameter updating on the pre-training model according to the total loss until the pre-training model converges to obtain a trained vehicle operation state classification model.

[0158] In addition, the logic instructions in the memory 430 described above can be implemented in the form of a software function unit and sold or used as an independent product, and can be stored in a computer readable storage medium. Based on such understanding, the technical solutions of the present application essentially or the part that contributes to the prior art or part of the technical solutions can be embodied in the form of a software product. The computer software product is stored in a storage medium, and includes a plurality of instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the method described in the various embodiments of the present application. The aforementioned storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM, Read-Only Memory), a random access memory (RAM, Random Access Memory), a magnetic disk or an optical disk, and various media that can store program codes.

[0159] On the other hand, the present application also provides a computer program product, the computer program product comprising a computer program, the computer program being storable on a non-transitory computer readable storage medium, and the computer program being executable by a processor, so that the computer can execute the vehicle operation state recognition method provided by the above-mentioned method, the method comprising:

[0160] Obtain sensor time sequence sample data of the vehicle, the sensor time sequence sample data comprising one-dimensional time sequence sample data in each dimension;

[0161] determine a time sequence period of the to-be-identified sensor time sequence data based on the one-dimensional time sequence data on each dimension, and perform a sliding window operation on the to-be-identified sensor time sequence data based on the time sequence period of the to-be-identified sensor time sequence data, to obtain a plurality of time window data;

[0162] input the plurality of time window data into the vehicle operation state classification model in sequence, to obtain a vehicle operation state recognition result output by the vehicle operation state classification model;

[0163] The vehicle operation state classification model is obtained by the following manner:

[0164] obtain sensor time sequence sample data of a vehicle, the sensor time sequence sample data including one-dimensional time sequence sample data on each dimension;

[0165] determine a time sequence period of the sensor time sequence sample data based on the one-dimensional time sequence sample data on each dimension, and perform a sliding window operation on the sensor time sequence sample data based on the time sequence period of the sensor time sequence sample data, to obtain a plurality of time window sample data;

[0166] input the plurality of time window sample data into a pre-training model in sequence, and determine a total loss of the pre-training model according to an output of the pre-training model, wherein the total loss includes a cross-entropy loss between vehicle operation prediction state labels corresponding to the time window sample data and vehicle operation real state labels, and an autocorrelation loss between vehicle operation prediction state labels corresponding to the plurality of time window sample data;

[0167] perform parameter updating on the pre-training model according to the total loss until the pre-training model converges, to obtain a trained vehicle operation state classification model.

[0168] In another aspect, the present application also provides a non-transitory computer readable storage medium having a computer program stored thereon, the computer program being executed by a processor to implement a vehicle operation state recognition method provided by each of the above methods, the method comprising:

[0169] obtain to-be-identified sensor time sequence data of a vehicle, the to-be-identified sensor time sequence data including one-dimensional time sequence data on each dimension;

[0170] determine a time sequence period of the to-be-identified sensor time sequence data based on the one-dimensional time sequence data on each dimension, and perform a sliding window operation on the to-be-identified sensor time sequence data based on the time sequence period of the to-be-identified sensor time sequence data, to obtain a plurality of time window data;

[0171] The plurality of time window data are sequentially input to the vehicle operation state classification model to obtain a vehicle operation state recognition result output by the vehicle operation state classification model.

[0172] The vehicle operation state classification model is obtained by the following manner:

[0173] The sensor time sequence sample data of the vehicle are obtained, and the sensor time sequence sample data comprises one-dimensional time sequence sample data in each dimension.

[0174] The time sequence period of the sensor time sequence sample data is determined based on the one-dimensional time sequence sample data in each dimension, and a sliding window operation is performed on the sensor time sequence sample data based on the time sequence period of the sensor time sequence sample data to obtain a plurality of time window sample data.

[0175] The plurality of time window sample data are sequentially input to a pre-training model, and a total loss of the pre-training model is determined according to an output of the pre-training model, wherein the total loss comprises a cross-entropy loss between a vehicle operation prediction state label and a vehicle operation real state label corresponding to the time window sample data and a self-correlation loss between vehicle operation prediction state labels corresponding to the plurality of time window sample data.

[0176] The pre-training model is updated in parameters according to the total loss until the pre-training model converges to obtain a trained vehicle operation state classification model.

[0177] The device embodiments described above are only schematic, wherein the units shown as separate components can or can not be physically separate, and the components shown as units can or can not be physical units, i.e., can be located in one place, or can be distributed on multiple network units. Part or all of the modules can be selected according to actual needs to achieve the purpose of the embodiment scheme. Those skilled in the art can understand and implement without creative labor.

[0178] From the above description of the embodiments, those skilled in the art can clearly understand that the embodiments can be realized by means of software plus necessary general hardware platforms, and of course can also be realized by hardware. Based on such understanding, the above technical solutions essentially or in other words make contributions to the prior art, and can be embodied in the form of a software product, which can be stored in a computer readable storage medium, such as a ROM / RAM, a magnetic disk, an optical disk, etc., and includes a plurality of instructions to make a computer device (which can be a personal computer, a server, or a network device, etc.) execute the methods described in each embodiment or some parts of the embodiments.

[0179] Finally, it should be noted that the above examples are only used to illustrate the technical solutions of the present application, and are not intended to limit the same; although the present application has been described in detail with reference to the foregoing examples, those skilled in the art should understand that the technical solutions recorded in the foregoing examples can still be modified, or some technical features recorded in the foregoing examples can be replaced by equivalent features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A vehicle handling state recognition method characterized by, The method comprises: obtaining sensor time series data to be identified of a vehicle, the sensor time series data to be identified comprising one-dimensional time series data on an x-axis, a y-axis and a z-axis; determining a time series period of the sensor time series data to be identified based on the one-dimensional time series data on each dimension, and performing a sliding window operation on the sensor time series data to be identified based on the time series period of the sensor time series data to be identified to obtain a plurality of time window data; inputting the plurality of time window data into a vehicle operation state classification model in sequence to obtain a vehicle operation state recognition result output by the vehicle operation state classification model; wherein the vehicle operation state classification model is obtained by the following method: obtaining sensor time series sample data of a vehicle, the sensor time series sample data comprising one-dimensional time series sample data on each dimension; determining a time series period of the sensor time series sample data based on the one-dimensional time series sample data on each dimension, and performing a sliding window operation on the sensor time series sample data based on the time series period of the sensor time series sample data to obtain a plurality of time window sample data; inputting the plurality of time window sample data into a pre-training model in sequence, and determining a total loss of the pre-training model according to an output of the pre-training model, wherein the total loss comprises a cross-entropy loss between a vehicle operation prediction state label corresponding to the time window sample data and a vehicle operation real state label, and an autocorrelation loss between vehicle operation prediction state labels corresponding to the plurality of time window sample data; updating parameters of the pre-training model according to the total loss until the pre-training model converges to obtain a trained vehicle operation state classification model; the pre-training model comprises an attention mechanism layer, a long short-term memory network layer, a convolution-pooling layer and a full connection layer, and the method comprises: for each time window sample data, inputting a current time window sample data after down-sampling and standardization processing into the attention mechanism layer and the convolution-pooling layer respectively, wherein in the attention mechanism layer, data on different dimensions in the time window sample data are given different attention weights; inputting an output result of the attention mechanism layer corresponding to the current time window sample data and an output result of the long short-term memory network layer corresponding to a previous time window sample data into the long short-term memory network layer; performing feature merging on the output result of the convolution-pooling layer corresponding to the current time window sample data and the output result of the long short-term memory network layer, and inputting the result into the full connection layer for classification calculation to obtain a vehicle operation prediction state label corresponding to the current time window sample data.

2. The vehicle handling state recognition method according to claim 1, characterized in that The method comprises: for each dimension, determining a time series period of the one-dimensional time series data on each dimension; determining a maximum time series period in the time series periods of the one-dimensional time series data on all dimensions as the time series period of the sensor time series data to be identified.

3. The vehicle handling state recognition method according to claim 2, characterized in that For each dimension of the one-dimensional time series data, the time series period of the one-dimensional time series data is obtained by the following way: Determine the mean and standard deviation of the one-dimensional time series data; Based on the mean and standard deviation of the one-dimensional time series data, determine the peak threshold and trough threshold of the one-dimensional time series data; Based on the peak threshold and trough threshold of the one-dimensional time series data, traverse the one-dimensional time series data to determine the candidate peaks and candidate troughs in the one-dimensional time series data; Based on the candidate peaks and candidate troughs in the one-dimensional time series data, determine the index range of the effective period of the one-dimensional time series data in combination with a preset period limit condition; The preset period limit condition includes marking any target candidate peak located to the left of the first candidate trough and recording the index of the target candidate peak; Or, if there is any target candidate peak located to the right of the first candidate trough, mark the target candidate peak and record the index of the target candidate peak; Determine the period feature points within the index range of the effective period, wherein the period feature points include effective troughs and effective peaks within the upper and lower index range corresponding to the effective troughs; Based on the time interval between the effective troughs in all adjacent period feature points in the one-dimensional time series data, determine the time series period of the one-dimensional time series data.

4. The vehicle handling state recognition method according to claim 1, characterized by, The cross-entropy loss of the pre-trained model is obtained by the following way: Determine the vehicle control real state label corresponding to each time window sample data; Based on the vehicle control prediction state label and the vehicle control real state label corresponding to each time window sample data, obtain the cross-entropy loss of the pre-trained model.

5. The vehicle handling state recognition method according to claim 1, characterized by, The autocorrelation loss of the pre-trained model is obtained by the following way: Based on the vehicle control prediction state label corresponding to each time window sample data, apply it to the autocorrelation function to obtain the autocorrelation loss of the pre-trained model.

6. A vehicle handling state recognition device characterized by comprising: It includes: A first vehicle control state recognition module is configured to obtain vehicle sensor time series data to be recognized, which includes one-dimensional time series data on the x-axis, y-axis and z-axis; A second vehicle control state recognition module is configured to determine the time series period of the sensor time series data to be recognized based on the one-dimensional time series data in each dimension, and perform a sliding window operation on the sensor time series data to be recognized based on the time series period of the sensor time series data to be recognized, to obtain a plurality of time window data; A third vehicle control state recognition module is configured to input the plurality of time window data into a vehicle control state classification model in sequence to obtain a vehicle control state recognition result output by the vehicle control state classification model; The vehicle control state classification model is obtained by the following way: Obtain sensor time series sample data of a vehicle, which includes one-dimensional time series sample data in each dimension; determine a time sequence cycle of the sensor time sequence sample data based on the one-dimensional time sequence sample data in each dimension, and perform a sliding window operation on the sensor time sequence sample data based on the time sequence cycle of the sensor time sequence sample data to obtain a plurality of time window sample data; input the plurality of time window sample data into a pre-training model in sequence, and determine a total loss of the pre-training model according to an output of the pre-training model, wherein the total loss includes a cross-entropy loss between a vehicle operation prediction state label corresponding to the time window sample data and a vehicle operation real state label, and a self-correlation loss between vehicle operation prediction state labels corresponding to the plurality of time window sample data; perform parameter updating on the pre-training model according to the total loss until the pre-training model converges, to obtain a trained vehicle operation state classification model; the pre-training model includes an attention mechanism layer, a long short-term memory network layer, a convolution-pooling layer, and a full connection layer, and includes: for each time window sample data, input the current time window sample data after down-sampling and standardization processing into the attention mechanism layer and the convolution-pooling layer respectively, wherein in the attention mechanism layer, data in different dimensions in the time window sample data are given different attention weights; input an output result of the attention mechanism layer corresponding to the current time window sample data and an output result of the long short-term memory network layer corresponding to a previous time window sample data into the long short-term memory network layer; perform feature merging on the output result of the convolution-pooling layer corresponding to the current time window sample data and the output result of the long short-term memory network layer, and input the feature merging result into the full connection layer for classification calculation to obtain a vehicle operation prediction state label corresponding to the current time window sample data.

7. An electronic device comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that, The processor executes the program to implement the vehicle operation state recognition method of any one of claims 1-5.

8. A non-transitory computer-readable storage medium having stored thereon a computer program, characterized in that, The computer program is executed by the processor to implement the vehicle operation state recognition method of any one of claims 1-5.

9. A computer program product comprising a computer program, characterized in that, The computer program is executed by the processor to implement the vehicle operation state recognition method of any one of claims 1-5.

Images