Automobile state data anomaly detection method and system based on log serialization
By converting vehicle status data into time-series logs and applying a time-series neural network model for anomaly detection, the challenge of processing high-dimensional vehicle status data is solved, and the accuracy of anomaly identification is improved.
Patent Information
- Application Number
- CN202311120176.6
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2023-08-31
- Publication Date
- 2025-11-18
- Estimated Expiration
- 2043-08-31
AI Technical Summary
Existing technologies struggle to effectively process high-dimensional vehicle state data, especially when identifying abnormal states, where accuracy is insufficient. Furthermore, traditional methods perform poorly when dealing with complex and heterogeneous data.
By converting vehicle status data into time-series logs, segmenting them using a sliding window, and applying a time-series neural network model for anomaly detection, the system uses a sliding window to segment the serialized time-series logs, resulting in multiple time-series logs with preset step sizes. Anomaly detection is then performed by combining a log key anomaly detection module and a parameter vector anomaly detection module.
Transforming complex, high-dimensional vehicle status data into an easily processed log format simplifies the data processing workflow and improves the accuracy and efficiency of anomaly detection.
Smart Images
Figure CN117011962B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of data processing, and in particular to a method and system for detecting anomalies in vehicle status data based on log serialization. Background Technology
[0002] In the modern automotive industry, the embedding of numerous sensors enables vehicles to generate real-time operational status data. This data encompasses multiple dimensions, including engine temperature, oil pressure, and braking systems, forming extremely rich high-dimensional data. However, processing this massive amount of data and effectively detecting abnormal changes within it is a highly challenging task. Traditional anomaly detection methods, such as statistical and machine learning methods including linear regression, decision trees, and support vector machines, perform well when processing low- or medium-dimensional data, but their effectiveness is unsatisfactory when dealing with high-dimensional data such as the current state of a vehicle.
[0003] Currently, existing technologies employ deep learning for anomaly detection. This involves constructing deep neural network models to learn the normal distribution of data and then using the model to determine the normality of new data points. While deep learning models have achieved significant results in some fields, they still face challenges due to the complexity and heterogeneity of high-dimensional vehicle state data when applied to such data.
[0004] Existing technologies face two main challenges: difficulty in processing high-dimensional state data and the challenge of accurately identifying abnormal states. Due to the high dimensionality of vehicle state data, traditional machine learning methods often fall short, while deep learning methods, although possessing some processing capabilities, still suffer from performance deficiencies when dealing with complex and heterogeneous data. Furthermore, the small proportion of abnormal state data in the total dataset further increases the difficulty for models in capturing abnormal state features. Summary of the Invention
[0005] The purpose of this invention is to provide a method and system for detecting anomalies in vehicle status data based on log serialization, so as to solve the problem of large storage space occupied by redundant log content in the prior art.
[0006] In a first aspect, the present invention provides a method for detecting anomalies in vehicle status data based on log serialization, the method comprising:
[0007] Obtain the vehicle status dataset;
[0008] Convert the vehicle status dataset into a total time-series log;
[0009] The total time series log is segmented by a sliding window to obtain multiple first time series logs with a preset step size; each first time series log includes a log key sequence and a parameter vector sequence.
[0010] Input the current log key sequence into the log key anomaly detection module to predict the next log key and obtain the predicted log key; based on the predicted log key and the actual log key, determine whether the log key is normal or abnormal.
[0011] When the log key is normal, the current parameter vector sequence is input into the parameter vector anomaly detection module to detect whether the parameter vector sequence is normal or abnormal;
[0012] When the parameter vector sequence is abnormal, output the abnormal parameter vector.
[0013] In one possible implementation, the method further includes:
[0014] When a log key is abnormal, output the abnormal log key and continue to check the log key sequence after it; or, when the parameter vector sequence is normal, continue to check the log key sequence after it.
[0015] In one possible implementation, converting the vehicle state dataset into a total time-series action log specifically includes:
[0016] Based on the timestamp of the collected vehicle status data, the vehicle status data in the collected vehicle status dataset is sorted.
[0017] The vehicle status data is filtered for status changes, and vehicle status data whose status changes at consecutive time points are retained.
[0018] The retained vehicle state data is converted into motion data;
[0019] For each action data point, convert the action data into a time-series log.
[0020] The time-series logs are parsed to obtain log keys and parameter vectors.
[0021] In one possible implementation, the step of filtering the vehicle state data for state changes and retaining vehicle state data whose state changes at consecutive time points specifically includes:
[0022] The vehicle status dataset is cleaned;
[0023] For the cleaned vehicle status dataset, based on the time nodes, it is determined whether the vehicle status data of two adjacent time nodes has changed. If it has changed, the vehicle status data of the two adjacent time nodes are retained.
[0024] When no changes have occurred, the fields of the vehicle status data for two adjacent time points are set to null values.
[0025] In one possible implementation, determining whether a log key is normal or abnormal based on the predicted log key and the actual log key specifically includes:
[0026] Input the log key sequence of the most recent h steps before time t into the log key anomaly detection module;
[0027] Obtain the conditional probability that the log key is k at time t under condition w, as output by the log key anomaly detection module;
[0028] If the conditional probability is not in the first C, then the log key at time t is determined to be abnormal; where C is an empirical value.
[0029] In one possible implementation, the method further includes the following preceding steps:
[0030] The log keys from the vehicle's historical time-series logs are input into the log key anomaly detection module to be trained, resulting in the trained log key anomaly detection module.
[0031] In one possible implementation, the step of inputting the log keys from the vehicle's historical time-series logs into the log key anomaly detection module to be trained for training, and obtaining the trained log key anomaly detection module specifically includes:
[0032] Remove the original fully connected layer, activation layer, Dropout layer, and the last LSTM layer from the original training model;
[0033] Freeze the other layers of the original trained model;
[0034] Add new LSTM layers, Dropout layers, fully connected layers, and activation layers to the original training model;
[0035] The log key anomaly detection module is obtained by updating the weights of the newly added LSTM layer, Dropout layer, fully connected layer, and activation layer.
[0036] Secondly, the present invention provides an anomaly detection system for vehicle status data based on log serialization, the system comprising:
[0037] A vehicle status acquisition unit, which is used to acquire a vehicle status dataset;
[0038] A conversion unit, wherein the conversion unit is used to convert the vehicle status dataset into a total time series log;
[0039] A cutting unit is used to cut the total time series log through a sliding window to obtain multiple first time series logs with a preset step size; each first time series log includes a log key sequence and a parameter vector sequence.
[0040] The log key anomaly detection unit is used to input the current log key sequence into the log key anomaly detection module, predict the next log key, and obtain the predicted log key; and determine whether the log key is normal or abnormal based on the predicted log key and the actual log key.
[0041] A parameter vector anomaly detection unit is used to input the current parameter vector sequence into the parameter vector anomaly detection module when the log key is normal, and to detect whether the parameter vector sequence is normal or abnormal.
[0042] A parameter vector anomaly output unit is used to output an abnormal parameter vector when the parameter vector sequence is abnormal.
[0043] Thirdly, the present invention provides a chip system including a processor coupled to a memory, the memory storing program instructions, wherein when the program instructions stored in the memory are executed by the processor, the method for detecting abnormal vehicle status data based on log serialization as described in any of the first aspects is implemented.
[0044] Fourthly, the present invention provides a computer-readable storage medium storing a computer program, wherein the computer program is executed by a processor using the vehicle status data anomaly detection method based on log serialization as described in any one of the first aspects.
[0045] By applying the vehicle status data anomaly detection method based on log serialization provided in this invention, vehicle status data is converted into time-series logs, and then anomaly detection is performed using a time-series neural network model. The goal of this invention is to solve the challenges of vehicle status data processing and improve the accuracy of anomaly identification. The method of this invention can transform complex, high-dimensional vehicle status data into an easily processed log format, thereby greatly simplifying the data processing flow and improving the accuracy and efficiency of anomaly detection. Attached Figure Description
[0046] Figure 1 This is a diagram of an architecture for detecting anomalies in vehicle status data based on log serialization.
[0047] Figure 2 This is one of the flowcharts of the vehicle status data anomaly detection method based on log serialization provided in Embodiment 1 of the present invention;
[0048] Figure 3 for Figure 2 Flowchart for step 120;
[0049] Figure 4 A schematic diagram illustrating the training of the log key anomaly detection model;
[0050] Figure 5 This is the second flowchart of the vehicle status data anomaly detection method based on log serialization provided in Embodiment 1 of the present invention;
[0051] Figure 6 This is one of the schematic diagrams of the vehicle status data anomaly detection system based on log serialization provided in Embodiment 2 of the present invention;
[0052] Figure 7 This is the second schematic diagram of the structure of the vehicle status data anomaly detection system based on log serialization provided in Embodiment 2 of the present invention;
[0053] Figure 8 This is a schematic diagram of the computer server structure provided in Embodiment 3 of the present invention;
[0054] Figure 9 This is a schematic diagram of the chip system structure provided in Embodiment 4 of the present invention;
[0055] Figure 10 This is a schematic diagram of a computer-readable storage medium structure provided in Embodiment 5 of the present invention. Detailed Implementation
[0056] The present application will now be described in further detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and not intended to limit it. Furthermore, it should be noted that, for ease of description, only the parts relevant to the invention are shown in the accompanying drawings.
[0057] It should be noted that, unless otherwise specified, the embodiments and features described in this application can be combined with each other. This application will now be described in detail with reference to the accompanying drawings and embodiments.
[0058] Example 1
[0059] Figure 1 The diagram below shows the architecture of this application. This application includes a log serialization processing module, a log key anomaly detection module, and a parameter vector anomaly detection module. The log serialization processing module is used to process vehicle state training data. Through model training, the log key anomaly detection module and the parameter vector anomaly detection module are used for anomaly detection.
[0060] Embodiment 1 of this invention provides a method for detecting anomalies in vehicle status data based on log serialization. The execution entity of this application is a server or the vehicle's own processor, such as the processor of an autonomous vehicle, or other devices with computing capabilities. This processing device is equipped with... Figure 1 The three modules, such as Figure 2 As shown, the method includes the following steps:
[0061] Step 110: Obtain the vehicle status dataset;
[0062] Specifically, vehicle status data refers to the status of components on a vehicle. This data can be collected by gathering information from various sensors on the vehicle, such as speed sensors and temperature sensors, or from the vehicle controller, body controller, and steering system. For example, data such as turning on the air conditioner, pressing the brake pedal, and opening the driver's side door can be collected.
[0063] Step 120: Convert the vehicle status dataset into a total time series log;
[0064] Specifically, such as Figure 3 As shown, step 120 includes:
[0065] Step 1201: Sort the vehicle status data in the collected vehicle status dataset according to the timestamp of the collected vehicle status data.
[0066] Specifically, the vehicle status data is sorted according to timestamps to ensure consistency of the data over time. The collected vehicle status dataset is shown in Table 1:
[0067]
[0068]
[0069] Table 1
[0070] Step 1202: Filter the vehicle status data for status changes and retain vehicle status data whose status changes at consecutive time points.
[0071] Step 1202 includes:
[0072] Clean the vehicle status dataset;
[0073] For the cleaned vehicle status dataset, based on the time nodes, it is determined whether the vehicle status data of two adjacent time nodes has changed. If it has changed, the vehicle status data of the two adjacent time nodes are retained.
[0074] When no changes have occurred, the fields of the vehicle status data for two adjacent time points are set to null values.
[0075] Specifically, the vehicle status data includes multiple fields, each with a value. The values of each field are normalized and cleaned, removing useless status records, such as null values. Then, status change filtering is performed. This involves iterating through the sorted vehicle status dataset and checking the vehicle status data at each time point. If a field in the vehicle status data changes at consecutive time points, it is retained; otherwise, it is set to null. This process preserves the data in the vehicle status dataset that shows changes in status.
[0076] For example, referring to Table 1, regarding battery temperature, the battery temperature detected at 08:00:00 is 40℃, the battery temperature at 08:00:05 is 50℃, the battery temperatures at 08:00:10, 08:00:15, and 08:00:20 are all 50℃, and the battery temperature at 08:00:25 is 40℃. This indicates that the battery temperature changed between 08:00:00 and 08:00:05 and between 08:00:20 and 08:00:25. Therefore, the battery temperatures from 08:00:00 to 08:00:05 and from 08:00:20 to 08:00:25 are retained, while the battery temperatures at 08:00:10 and 08:00:15 are deleted.
[0077] Step 1203: Convert the retained vehicle state data into motion data;
[0078] Specifically, dense vehicle state data is transformed into sparse motion data. This involves abstracting changes in vehicle state data into specific actions. These motion data include a time period and a specific state change action. Motion data over a time period can be represented by a motion curve that changes over time. For example, continuing to refer to Table 1, the transformed motion data are "Battery temperature rises from 08:00:00 to 08:00:05" and "Battery temperature decreases from 08:00:20 to 08:00:25". For tire pressure, there are motion data "Tire pressure decreases from 08:00:05 to 08:00:10" and "Tire pressure rises from 08:00:20 to 08:00:25". For braking force, there are motion data "Brake force increases from 08:00:05 to 08:00:20" and "Brake force increases from 08:00:20 to 08:00:25".
[0079] Step 1204: For each action data point, convert the action data into a time-series log.
[0080] Specifically, for cases where each time point has more than one action data point, such as those shown in Table 1 (at 08:00:05, the action data includes battery temperature and tire pressure), each action data point can be reassembled into a one-dimensional time-series log. The generated one-dimensional time-series log is shown below:
[0081] 08:00:05 The battery temperature increased to 50℃;
[0082] 08:00:10 Tire pressure dropped to 198 kPa;
[0083] 08:00:15 Depress the brake pedal by 20 degrees;
[0084] 08:00:20 Depress the brake pedal by 30 degrees;
[0085] For multiple action data points at the same time point, such as battery temperature, tire pressure, and brake pedal at 08:00:25, the time series logs can be sorted according to a preset field priority. For example, if the preset field priority is that battery temperature, tire pressure, and brake pedal have decreasing priorities, the time series logs for 08:00:25 would look like this:
[0086] 08:00:25 The battery temperature dropped to 40℃;
[0087] 08:00:25 Tire pressure increased to 200 kPa;
[0088] 08:00:25 Depress the brake pedal at a 35-degree angle;
[0089] After traversing all action data, a time-series log is generated. This time-series log fully preserves abnormal changes. For example, the time-series log could show the brake pedal being pressed at an angle of 20 degrees in the first second, 30 degrees in the second second, and 35 degrees in the third second, etc.
[0090] This application can obtain a total time series log of a preset duration based on the time series log of each action.
[0091] Specifically, each action data corresponds to a time series log. Within a preset time period, since there is at least one action, the time series logs within that time period can be summarized to obtain a total time series log.
[0092] Specifically, each action corresponds to a time series log. Within a preset duration, since there is at least one action, the time series logs within that time period can be aggregated to obtain a time series log for the preset duration.
[0093] Step 1205: Parse the time series logs to obtain log keys and parameter vectors;
[0094] The aforementioned time-series logs can be stored in the database as data tables. For ease of storage and processing, the time-series logs can be parsed to obtain log keys and parameter vectors. The log keys represent the specific actions performed, and the parameter vectors represent the execution time and intensity of the actions. These two parts of the time-series logs facilitate subsequent anomaly detection.
[0095] Specifically, as shown in Table 2, the parameter vector consists of a timestamp and parameter values, and the parameter values can be empty.
[0096]
[0097] Table 2
[0098] Step 130: The total time series log is segmented by a sliding window to obtain multiple first time series logs with a preset step size; each first time series log includes a log key sequence and a parameter vector sequence.
[0099] Specifically, the cutting can be performed according to a preset step size, which can be set based on practical experience.
[0100] Step 140: Input the current log key sequence into the log key anomaly detection module to predict the next log key and obtain the predicted log key; Based on the predicted log key and the actual log key, determine whether the log key is normal or abnormal.
[0101] Specifically, the log key set is a finite set, representing the collection of all log key types. The log key at the current position in the log key sequence strongly depends on the most recently encountered log key. The log key anomaly detection module is a neural network multi-classifier based on Long Short-Term Memory (LSTM). It takes a log key sequence of a preset duration as input and outputs a predicted log key. If the actual log key is within the allowed range of the predicted log key, it is considered normal; if the actual log key is outside the allowed range of the predicted log key, it is considered abnormal.
[0102] In one example, the log key sequence h steps before time t is input to the log key anomaly detection module. The module outputs the conditional probability of the log key being k at time t under condition w. If the conditional probability is not within the top C, the log key at time t is determined to be an anomaly, where C is a manually set candidate parameter for normal log keys. If the actual log key is not within the candidate range of the top C probabilities, it is considered an anomaly. The set of log keys is finite; let S be the set of all log key types. Assume k is the key in the log key sequence at time t, which has a strong dependency on the most recent key. Therefore, an LSTM-based neural network multi-classifier is used for log key anomaly detection. This model receives the log key history sequence of the most recent sliding window size as input and outputs the probability distribution of the next log key.
[0103] The model outputs a probability distribution, representing the probability that each possible log key will be the next log key. Based on this, a parameter `n` is set, representing the number of log keys considered normal. Specifically, the top `n` log keys with the highest probabilities from the model's output probability distribution are selected as normal log keys. If a log key that actually appears is not among these `n` log keys, it is considered abnormal.
[0104] This method is based on historical data. The top n log keys with the highest predicted probability are most likely to be normal, while other log keys are more likely to be abnormal. This parameter n can be adjusted according to actual needs.
[0105] When the log key is normal, proceed to step 150. When the log key is abnormal, output "Log key abnormal" and slide one step to continue detecting the log key sequence of the next step.
[0106] Step 150: When the log key is normal, input the current parameter vector sequence into the parameter vector anomaly detection module to detect whether the parameter vector sequence is normal or abnormal;
[0107] Specifically, if the log key is valid, the parameter vector corresponding to that log key is checked. If the log key is invalid, the invalid log key is output.
[0108] Each log key corresponds to a parameter vector anomaly detection module. When a log key is detected as normal, the parameter vector anomaly detection module corresponding to that log key continues to perform parameter vector anomaly detection.
[0109] Specifically, certain anomalies can lead to abnormal parameter vectors. For example, remotely controlling the air conditioning to turn on heating mode in a high-temperature cabin environment. The "turn on air conditioning" key is the log key and is obviously normal, but the corresponding parameter vector "heating" is abnormal. These anomalies originate from the parameter vector sequences of different log keys, rather than the log keys themselves. This is crucial for performance monitoring and anomaly detection.
[0110] This application treats the parameter vector corresponding to each log key as a multidimensional time series, thus simplifying the problem to multidimensional time series anomaly detection. For example, for a log key such as temperature regulation, the feature vector could be the temperature data at each time step. Therefore, an independent neural network model is constructed for the parameter vector sequence of each log key. In this neural network model, the input is the sequence of parameter value vectors corresponding to a certain log key, and the values in the vectors are normalized using the mean and standard deviation of all values of the same parameter vector in the training data. Normalization is performed on each parameter vector in the parameter vector sequence. Specifically, each parameter vector is subtracted from the mean of the parameter vectors at the corresponding position in the training data, and then divided by its standard deviation, thereby achieving normalization.
[0111] The output of the neural network model for the parameter vector sequence is a real-valued vector. It predicts the next parameter vector based on the historical parameter vector sequence of the most recent sliding window size. The training objective function is to minimize the error between the predicted and actual parameter vectors. For multidimensional time series data, a squared loss function is used to evaluate the neural network model. The model is considered complete when the error between the actual and predicted parameter vectors is within a preset range, and this completed model is used as the network model for the parameter vector sequence. In the anomaly detection phase after training, the error is measured by calculating the difference between the predicted and actual parameter vectors, rather than by setting a threshold. The error between the predicted and actual parameter vectors follows a Gaussian distribution. Therefore, if the error between the predicted and observed value vectors is within the high confidence interval of the Gaussian distribution, the parameter vector of the input log key is considered normal; otherwise, it is considered abnormal.
[0112] In this context, the actual parameter vector typically refers to the parameter vector actually recorded by the vehicle. The predicted parameter vector is the output predicted by the neural network model based on the input parameter vector sequence. Therefore, if the predicted parameter vector and the actual parameter vector differ significantly, it indicates that the actual parameter vector is abnormal. This is because, after extensive training, the neural network model has developed a high level of predictive accuracy. For example, if the input parameter vector sequence is the vehicle interior temperature {22 degrees, 21 degrees, 23 degrees, 20 degrees, 22 degrees}, the neural network model predicts the next time point to be 22 degrees, but the actual temperature recorded by the vehicle's sensors at the next time point is 30 degrees. Clearly, this large discrepancy is an anomaly, potentially caused by a sensor malfunction. In such cases, the anomaly can be determined by the difference between the predicted and actual parameter vectors.
[0113] Step 160: When the parameter vector sequence is abnormal, output the abnormal parameter vector.
[0114] Specifically, if the parameter vector is abnormal, information indicating the anomaly can be output, such as: the log key is normal, but the parameter vector is abnormal. This can also include: the location of the anomaly, the abnormal data, and the predicted data. Subsequently, the detection at the next time point is performed. The log key anomaly detection module and the parameter vector anomaly detection module are pre-trained models; see [link to training process] for details. Figure 4 .
[0115] Specifically, certain anomalies can lead to abnormal parameter vectors. For example, remotely controlling the air conditioning to turn on heating mode in a high-temperature cabin environment. The "turn on air conditioning" key is the log key and is obviously normal, but the corresponding parameter vector "heating" is abnormal. These anomalies originate from the parameter vector sequences of different log keys, rather than the log keys themselves. This is crucial for performance monitoring and anomaly detection.
[0116] This application treats the parameter vector corresponding to each log key as a multidimensional time series, thus simplifying the problem to multidimensional time series anomaly detection. For example, for a log key such as temperature regulation, the feature vector could be the temperature data at each time step. Therefore, an independent neural network model is constructed for the parameter vector sequence of each log key. In this neural network model, the input is the sequence of parameter value vectors corresponding to a certain log key, and the values in the vectors are normalized using the mean and standard deviation of all values of the same parameter vector in the training data. Normalization is performed on each parameter vector in the parameter vector sequence. Specifically, each parameter vector is subtracted from the mean of the parameter vectors at the corresponding position in the training data, and then divided by its standard deviation, thereby achieving normalization.
[0117] The output of the neural network model for the parameter vector sequence is a real-valued vector. It predicts the next parameter vector based on the historical parameter vector sequence of the most recent sliding window size. The training objective function is to minimize the error between the predicted and actual parameter vectors. For multidimensional time series data, a squared loss function is used to evaluate the neural network model. The model is considered complete when the error between the actual and predicted parameter vectors is within a preset range, and this completed model is used as the network model for the parameter vector sequence. In the anomaly detection phase after training, the error is measured by calculating the difference between the predicted and actual parameter vectors, rather than by setting a threshold. The error between the predicted and actual parameter vectors follows a Gaussian distribution. Therefore, if the error between the predicted and observed value vectors is within the high confidence interval of the Gaussian distribution, the parameter vector of the input log key is considered normal; otherwise, it is considered abnormal.
[0118] In this context, the actual parameter vector typically refers to the parameter vector actually recorded by the vehicle. The predicted parameter vector is the output predicted by the neural network model based on the input parameter vector sequence. Therefore, if the predicted parameter vector and the actual parameter vector differ significantly, it indicates that the actual parameter vector is abnormal. This is because, after extensive training, the neural network model has developed a high level of predictive accuracy. For example, if the input parameter vector sequence is the vehicle interior temperature {22 degrees, 21 degrees, 23 degrees, 20 degrees, 22 degrees}, the neural network model predicts the next time point to be 22 degrees, but the actual temperature recorded by the vehicle's sensors at the next time point is 30 degrees. Clearly, this large discrepancy is an anomaly, potentially caused by a sensor malfunction. In such cases, the anomaly can be determined by the difference between the predicted and actual parameter vectors.
[0119] Furthermore, the log key anomaly detection module of this application also retrains the model based on the vehicle model during model training, thereby improving the accuracy of the model.
[0120] Specifically, while different types of vehicles share similarities in driving modes, significant differences exist in specific driving details. For example, sedans and trucks may have significant differences in their driving mode command logs, such as steering and braking. Suppose a pre-trained model has been obtained by training on 100 commercially available vehicle models during the pre-training phase. However, when a completely new vehicle model is released, directly applying the pre-trained model to it will result in poor performance because the sensor data fields recorded by the new model are different, and the new model may have new driving modes. In this case, training a model specifically for the new vehicle model will face challenges such as insufficient data volume and inadequate data coverage of all abnormal scenarios. Therefore, the pre-trained model can be fine-tuned to accurately adapt to specific driving modes and environments.
[0121] The specific fine-tuning includes the following steps:
[0122] First, remove the layer.
[0123] Specifically, the original fully connected layer, activation layer, Dropout layer, and last LSTM layer are removed from the original trained model.
[0124] Secondly, freeze the model layer.
[0125] Specifically, in order to preserve the knowledge learned in the pre-training phase, the other layers of the original training model are frozen, meaning that the weights of the frozen layers will not be updated during subsequent training.
[0126] Next, add a new layer.
[0127] Specifically, new LSTM layers, Dropout layers, fully connected layers, and activation layers are added to the pre-trained model. The initial values of the weights of these newly added layers are randomized and will be updated during the subsequent fine-tuning process.
[0128] Finally, fine-tuning was performed using the log key sequence specific to the vehicle model.
[0129] Specifically, the newly added network layers are fine-tuned using log key sequences for specific vehicle models. This process is similar to the training process of pre-trained models. Since most of the layers in the pre-trained model have been frozen, only the weights of the newly added layers are updated, thus ensuring that the trained target model is more adapted to the driving mode and environment of the specific vehicle model.
[0130] Figure 5 Here is a flowchart illustrating a specific implementation of this application, such as... Figure 5 As shown, it includes the following steps:
[0131] Step 510: Obtain vehicle status data;
[0132] Step 520, Status data log serialization processing;
[0133] Step 530: Parse into an instruction log sequence, which includes the day key and parameter vector;
[0134] Step 540: Segment the window by its size;
[0135] Step 550, Log key anomaly detection module;
[0136] If the log key is abnormal, proceed to step 560; if the log key is normal, proceed to step 570.
[0137] Step 560: Log key error reported;
[0138] Step 570, Parameter Vector Anomaly Detection Module;
[0139] When the parameter vector is abnormal, step 580 is executed; when the parameter vector is normal, step 590 is executed.
[0140] Step 580, the output parameter vector is abnormal;
[0141] Step 590, output the normal log result.
[0142] When the output log shows a normal result, slide one step forward and execute step 550.
[0143] By applying the vehicle status data anomaly detection method based on log serialization provided in this invention, vehicle status data is converted into time-series logs, and then anomaly detection is performed using a time-series neural network model. The goal of this invention is to solve the challenges of vehicle status data processing and improve the accuracy of anomaly identification. The method of this invention can transform complex, high-dimensional vehicle status data into an easily processed log format, thereby greatly simplifying the data processing flow and improving the accuracy and efficiency of anomaly detection.
[0144] Example 2
[0145] Embodiment 2 of the present invention provides a vehicle status data anomaly detection system based on log serialization, such as... Figure 6 As shown, the system includes: a vehicle status acquisition unit 610, a conversion unit 620, a cutting unit 630, a log key anomaly judgment unit 640, a parameter vector anomaly judgment unit 650, and a parameter vector anomaly output unit 660.
[0146] The vehicle status acquisition unit 610 is used to acquire a vehicle status dataset;
[0147] Transformation unit 620 is used to convert the vehicle status dataset into a total time series log;
[0148] The cutting unit 630 is used to cut the total time series log through a sliding window to obtain multiple first time series logs with a preset step size; each first time series log includes a log key sequence and a parameter vector sequence;
[0149] The log key anomaly judgment unit 640 is used to input the current log key sequence into the log key anomaly detection module, predict the next log key, and obtain the predicted log key; based on the predicted log key and the actual log key, it is determined whether the log key is normal or abnormal.
[0150] The parameter vector anomaly detection unit 650 is used to input the current parameter vector sequence into the parameter vector anomaly detection module when the log key is normal, and to detect whether the parameter vector sequence is normal or abnormal.
[0151] The parameter vector anomaly output unit 660 is used to output an abnormal parameter vector when the parameter vector sequence is abnormal.
[0152] Furthermore, such as Figure 7 As shown, the system also includes: a log key exception output unit 710.
[0153] The log key exception output unit 710 is used to output the exception log key when the log key is exception, and continue to detect the log key sequence after the current one; or, when the parameter vector sequence is normal, continue to detect the log key sequence after the current one.
[0154] Furthermore, the conversion unit 620 converts the vehicle status dataset into a total time-series action log, specifically including: sorting the vehicle status data in the collected vehicle status dataset according to the timestamp of the collected vehicle status data; filtering the vehicle status data for status changes and retaining vehicle status data whose status changes at consecutive time points; converting the retained vehicle status data into an action data table; converting the action data table into a one-dimensional time-series log for each action; and obtaining a total time-series log of a preset duration based on the time-series log of each action.
[0155] Furthermore, the conversion unit 620 performs state change filtering on the vehicle state data, retaining vehicle state data whose state changes at consecutive time points. Specifically, this includes: cleaning the vehicle state dataset; for the cleaned vehicle state dataset, determining whether the vehicle state data of two adjacent time points has changed based on the time points; if it has changed, retaining the vehicle state data of the two adjacent time points; if it has not changed, setting the fields of the vehicle state data of the two adjacent time points to null values.
[0156] Furthermore, the log key anomaly judgment unit 640 determines whether the log key is normal or abnormal based on the predicted log key and the actual log key. Specifically, this includes: inputting the log key sequence of the most recent h steps before time t into the log key anomaly detection module; obtaining the conditional probability of the log key being k under condition w at time t, output by the log key anomaly detection module; and determining that the log key at time t is abnormal if the conditional probability is not in the first C steps, where C is an empirical value.
[0157] Furthermore, the system also includes a model training unit 720.
[0158] The model training unit 720 is used to input the log keys in the vehicle's historical time series logs into the log key anomaly detection module to be trained, and obtain the trained log key anomaly detection module.
[0159] Furthermore, the model training unit 720 is specifically used for:
[0160] Remove the original fully connected layers, activation layers, Dropout layers, and the last LSTM layer from the original trained model;
[0161] Freeze the other layers of the original trained model;
[0162] Add new LSTM layers, Dropout layers, fully connected layers, and activation layers to the original trained model;
[0163] The log key anomaly detection module is obtained by updating the weights of the newly added LSTM layer, Dropout layer, fully connected layer, and activation layer.
[0164] Example 3
[0165] Embodiment 3 of the present invention provides a computer server, such as Figure 8 As shown, it includes: memory, processor, and transceiver;
[0166] The processor is used to couple with the memory, read and execute instructions in the memory to implement any of the vehicle status data anomaly detection methods based on log serialization provided in Embodiment 1 above;
[0167] The transceiver is coupled to the processor, and the processor controls the transceiver to send and receive messages.
[0168] Example 4
[0169] Embodiment 4 of the present invention provides a chip system, such as Figure 9 As shown, it includes a processor and a coupled memory. The memory stores program instructions. When the program instructions stored in the memory are executed by the processor, any of the vehicle status data anomaly detection methods based on log serialization provided in Embodiment 1 are implemented.
[0170] Example 5
[0171] Embodiment 5 of the present invention provides a computer-readable storage medium, such as... Figure 10 As shown, it includes a program or instructions that, when run on a computer, implement any of the vehicle status data anomaly detection methods based on log serialization provided in Embodiment 1.
[0172] Those skilled in the art will further recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this invention.
[0173] The steps of the methods or algorithms described in conjunction with the embodiments disclosed herein can be implemented in hardware, a software module executed by a processor, or a combination of both. The software module can be located in random access memory (RAM), main memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disk, removable disk, CD-ROM, or any other form of storage medium known in the art.
[0174] The above specific embodiments further illustrate the purpose, technical solution, and beneficial effects of the present invention. It should be understood that the above are merely specific embodiments of the present invention and are not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.
Claims
1. A method for detecting anomalies in vehicle status data based on log serialization, characterized in that, The method includes: Obtain the vehicle status dataset; Convert the vehicle status dataset into a total time-series log; The total time series log is segmented by a sliding window to obtain multiple first time series logs with a preset step size; each first time series log includes a log key sequence and a parameter vector sequence. Input the current log key sequence into the log key anomaly detection module to predict the next log key and obtain the predicted log key; based on the predicted log key and the actual log key, determine whether the log key is normal or abnormal. When the log key is normal, the current parameter vector sequence is input into the parameter vector anomaly detection module to detect whether the parameter vector sequence is normal or abnormal; When the parameter vector sequence is abnormal, output the abnormal parameter vector; Specifically, converting the vehicle state dataset into a total time-series action log includes: Based on the timestamp of the collected vehicle status data, the vehicle status data in the collected vehicle status dataset is sorted. The vehicle status data is filtered for status changes, and vehicle status data whose status changes at consecutive time points are retained. The retained vehicle state data is converted into motion data; For each action data point, convert the action data into a time-series log. Parse the time-series logs to obtain log keys and parameter vectors; The step of filtering the vehicle status data for status changes and retaining vehicle status data whose status changes at consecutive time points specifically includes: The vehicle status dataset is cleaned; For the cleaned vehicle status dataset, based on the time nodes, it is determined whether the vehicle status data of two adjacent time nodes has changed. If it has changed, the vehicle status data of the two adjacent time nodes are retained. When no changes have occurred, the fields of the vehicle status data for two adjacent time points are set to null values.
2. The method according to claim 1, characterized in that, The method further includes: When a log key is abnormal, output the abnormal log key and continue to check the log key sequence after it; or, when the parameter vector sequence is normal, continue to check the log key sequence after it.
3. The method according to claim 1, characterized in that, The step of determining whether a log key is normal or abnormal based on the predicted log key and the actual log key specifically includes: Input the log key sequence of the most recent h steps before time t into the log key anomaly detection module; Obtain the conditional probability that the log key is k at time t under condition w, as output by the log key anomaly detection module; If the conditional probability is not in the first C, then the log key at time t is determined to be abnormal; where C is an empirical value.
4. The method according to claim 1, characterized in that, The method is preceded by: The log keys from the vehicle's historical time-series logs are input into the log key anomaly detection module to be trained, resulting in the trained log key anomaly detection module.
5. The method according to claim 4, characterized in that, The method further includes: Remove the original fully connected layers, activation layers, Dropout layers, and the last LSTM layer from the original trained model; Freeze the other layers of the original trained model; Add new LSTM layers, Dropout layers, fully connected layers, and activation layers to the original training model; The log key anomaly detection module is obtained by updating the weights of the newly added LSTM layer, Dropout layer, fully connected layer, and activation layer.
6. A vehicle status data anomaly detection system based on log serialization, characterized in that, The system includes: A vehicle status acquisition unit, which is used to acquire a vehicle status dataset; A conversion unit, wherein the conversion unit is used to convert the vehicle status dataset into a total time series log; A cutting unit is used to cut the total time series log through a sliding window to obtain multiple first time series logs with a preset step size; each first time series log includes a log key sequence and a parameter vector sequence. The log key anomaly detection unit is used to input the current log key sequence into the log key anomaly detection module, predict the next log key, and obtain the predicted log key; and determine whether the log key is normal or abnormal based on the predicted log key and the actual log key. A parameter vector anomaly detection unit is used to input the current parameter vector sequence into the parameter vector anomaly detection module when the log key is normal, and to detect whether the parameter vector sequence is normal or abnormal. A parameter vector anomaly output unit is used to output an abnormal parameter vector when the parameter vector sequence is abnormal. Specifically, the conversion unit transforms the vehicle state dataset into a total time-series action log, including: Based on the timestamp of the collected vehicle status data, the vehicle status data in the collected vehicle status dataset is sorted. The vehicle status data is filtered for status changes, and vehicle status data whose status changes at consecutive time points are retained. The retained vehicle state data is converted into motion data; For each action data point, convert the action data into a time-series log. Parse the time-series logs to obtain log keys and parameter vectors; The conversion unit filters the vehicle status data for status changes, retaining vehicle status data whose status changes at consecutive time points, specifically including: The vehicle status dataset is cleaned; For the cleaned vehicle status dataset, based on the time nodes, it is determined whether the vehicle status data of two adjacent time nodes has changed. If it has changed, the vehicle status data of the two adjacent time nodes are retained. When no changes have occurred, the fields of the vehicle status data for two adjacent time points are set to null values.
7. A chip system, characterized in that, The system includes a processor coupled to a memory, the memory storing program instructions, which, when executed by the processor, implement the vehicle status data anomaly detection method based on log serialization as described in any one of claims 1-5.
8. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program, which is executed by a processor according to any one of claims 1-5, the method for detecting abnormal vehicle status data based on log serialization.
Citation Information
Patent Citations
Remote log diagnosis method and device, electronic equipment and storage medium
CN116030548A