Cloud edge end time sequence anomaly detection system and method based on pre-training language model

Abstract

The application discloses a cloud edge end time sequence anomaly detection system and method based on a pre-training language model, and an anomaly detection field in a cloud edge end architecture, wherein the system contains an edge device module and a cloud server module, and the two modules work cooperatively through a network, and jointly complete data acquisition, real-time detection and model optimization, and a detection method is provided accordingly.Based on the pre-training language model and the cloud edge end cooperative architecture, an efficient data completion, anomaly detection and model optimization mechanism is established between the edge device and the cloud server.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of anomaly detection in cloud-edge-end architecture, and particularly relates to a cloud-edge-end time series anomaly detection system and method based on a pre-trained language model, which performs real-time analysis on time series data collected by vehicle sensors, solves the problem of data missing, and improves the accuracy and response speed of anomaly detection. BACKGROUND

[0002] The statements in this section merely provide background information related to the present disclosure and can not constitute the prior art.

[0003] Time series anomaly detection has become an important research topic in many key fields such as traffic monitoring and industrial Internet of Things. Existing technologies mainly include statistical methods, classical machine learning methods and deep learning methods. Statistical methods such as moving average, exponential smoothing and autoregressive integrated moving average (ARIMA) model are usually used to capture the trend and period of time series, but they have limitations in dealing with complex, nonlinear and high-dimensional data.

[0004] Traditional machine learning methods include clustering algorithms (such as k-means and density-based methods) and classification algorithms (such as decision trees and support vector machines). Although these methods have certain anomaly detection capabilities, they often perform poorly when faced with complex high-dimensional time series data, and have weak processing capabilities for noise and missing data.

[0005] In recent years, deep learning methods have shown strong advantages in time series anomaly detection. Common deep learning techniques include autoencoders, variational autoencoders (VAE), recurrent neural networks (RNN) and long short-term memory networks (LSTM), etc. These methods can effectively learn the long-term dependencies in time series data, and are particularly suitable for anomaly detection of nonlinear and high-dimensional data. In addition, generative adversarial networks (GAN) and deep reinforcement learning (DRL) methods have also been applied in anomaly detection. However, these methods have high demand for computing resources, and there are real-time challenges in applying them to edge devices.

[0006] Currently, the data collected by vehicle sensors is easily affected by environmental conditions and often accompanied by data missing and noise, which can significantly reduce the accuracy of anomaly detection. Traditional time series analysis methods have obvious shortcomings in dealing with noise and data missing, and the combination of cloud-edge-end architecture provides new possibilities for data completion and noise reduction. How to efficiently complete the missing data and eliminate the noise at the edge, while taking advantage of the powerful computing power of the cloud to optimize the data and correct the model, is an important technical challenge to ensure the robustness and detection effect of the model.

[0007] Meanwhile in driver behavior detection and car crash prediction applications, the model needs to be constantly updated to cope with environmental changes and differences in data distribution. Under the cloud-edge architecture, the model is usually trained and updated in the cloud to maintain high accuracy, but it also needs to be quickly deployed to edge devices to meet real-time requirements. However, the data characteristics of different driving environments differ, which requires the model to have sufficient robustness to cope with diverse data scenarios. Therefore, how to achieve efficient collaborative updating of the model between the cloud and edge devices, both to ensure the real-time response of the model and to balance the computational resource constraints of edge devices, is a technical difficulty that needs in-depth research.

[0008] To cope with real-time reaction application scenarios such as car crash detection, combining cloud-edge architecture has become an effective solution. Existing cloud-edge architecture generally works collaboratively by edge devices and cloud servers, with edge devices responsible for data collection and preliminary processing, and cloud servers for more complex analysis and model training. SUMMARY

[0009] The present application provides a cloud-edge time series anomaly detection system and method based on a pre-trained language model, aiming to achieve dynamic optimization and efficient detection through domain slicing and sequence completion; the edge device is pre-installed with a basic model and domain slice when it leaves the factory, performs local completion, noise processing and personalized local optimization, and generates error feedback after each drive; the cloud aggregates data from multiple edge devices, optimizes the domain slice through incremental training, and then distributes the updated slice to the edge device, ensuring global optimization and personalized ability of the model. This solution takes into account real-time, adaptability and computational cost, providing solid technical support for efficient anomaly detection in complex driving scenarios.

[0010] The technical solutions of the present application are as follows:

[0011] The cloud-edge time series anomaly detection system based on a pre-trained language model comprises:

[0012] A data acquisition module, a sequence completion module, an anomaly detection module and an error collection module are provided at the edge device end, and a weighted aggregation module, an incremental training module and a domain slice update module are provided at the cloud server end;

[0013] The data acquisition module is responsible for collecting data from various sensors during driving;

[0014] The sequence completion module completes missing data based on a fine-tuned pre-trained language model;

[0015] The anomaly detection module uses a fine-tuned pre-trained language model to detect anomalies in the completed data;

[0016] The error collection module records error information between the abnormality detection result and the actual situation after the driving task is completed.

[0017] The weighted aggregation module receives error information from multiple edge devices, and performs weighted aggregation on the error information to make full use of feedback information from different scenes.

[0018] The incremental training module optimizes the domain slice of the fine-tuned pre-trained language model based on the aggregated data using an incremental training method.

[0019] The domain slice update module distributes the updated domain slice to the edge device.

[0020] The application also provides a cloud-edge-end time series anomaly detection method based on a pre-trained language model, and the cloud-edge-end time series anomaly detection system, which comprises:

[0021] Step S1: data acquisition; the edge device acquires vehicle data in real time through a sensor;

[0022] Step S2: selecting a domain slice; the pre-trained language model uses prior knowledge to select a domain slice;

[0023] Step S3: data completion; if there is missing data in the sensor data, the sequence completion module completes the data under the prior condition of the domain slice, to ensure the integrity of the data;

[0024] Step S4: real-time anomaly detection; after the data is completed, the data is input into the anomaly detection module, and the driving behavior is detected for abnormality;

[0025] Step S5: error feedback and local update; after each driving task is completed, the abnormality detection result is compared with the actual situation, error information is generated, and the error information is uploaded to the cloud server together with road data when the network allows;

[0026] Step S6: cloud incremental training; the cloud server performs weighted aggregation on the received error information, and optimizes the pre-trained language model through incremental training to update the domain slice;

[0027] Step S7: domain slice distribution; the updated domain slice is distributed to the edge device, to realize continuous optimization.

[0028] Further, the step S1 comprises:

[0029] Step S11: sensor data acquisition; the edge device acquires multiple time series data in real time through a vehicle-mounted sensor;

[0030] Step S12: sequence instance normalization and slicing; the collected data is subjected to instance normalization, and the value of each time series variable is normalized into a standardized form with a mean of zero and a variance of one; then the instance-normalized data is divided into a plurality of non-overlapping segments, each of which will be embedded into a hidden space for subsequent processing; each segment represents a local time series.

[0031] Further, the step S2 comprises:

[0032] In the edge device, a domain slice library is pre-stored, and the domain slices in the domain slice library are delivered to the edge device after being trained and optimized in the cloud; each domain slice contains a prefix vector P prefix of a specific scene.

[0033] Further, the step S3 comprises:

[0034] The data completion is performed based on a completion mechanism and a completion strategy of the domain slice.

[0035] Further, the completion mechanism comprises:

[0036] In the input data, if a missing is detected, the pre-trained language model uses a learnable embedding to represent the missing position, and the embedding combines context information to enable the pre-trained language model to more effectively complete the missing data.

[0037] The completion strategy comprises:

[0038] Simple completion: for simple missing cases, the pre-trained language model performs rapid completion based on statistical features in the domain slice;

[0039] Deep inference completion: for complex missing data, the pre-trained language model combines context features and deep learning methods to complete the missing part by inference; the pre-trained language model generates reasonable completion values by using the fine-tuned pre-training features:

[0040] Y t =f θ (P t ,E miss ,C t )

[0041] Wherein: f θ is a function of the completion model, defined by parameters θ, P t is the current segment, E miss is the missing embedding, and C t is the context information.

[0042] Further, the step S4 comprises:

[0043] The completed data will be input into the anomaly detection module; the sliding window strategy is adopted to judge the overall anomaly of the data; the length of the sliding window is set to w, and the time series data is operated by sliding; for the data X t in each sliding window, the pre-trained language model is used to extract deep feature representation Z t ;

[0044] Z t = F θ (X t )

[0045] Where: f θ is a pre-trained model, θ is the parameter of the model, and Z t is the feature representation; for the extracted deep feature representation Z t , the anomaly score of each time point in the window is calculated; set an anomaly score function g(·), the scheme adopts reconstruction error; for each time point x i ∈X t , the anomaly score is:

[0046] A i = g(Z t , x i )

[0047] Where: A i is the anomaly score of time point x i ; define an anomaly score threshold τ, if the anomaly score A i of time point x i is greater than the threshold, the point is marked as abnormal:

[0048]

[0049] For the entire sliding window X t , if all points are detected as abnormal, that is:

[0050]

[0051] Where: I window represents the anomaly flag of the entire window; only when all points in the window are marked as abnormal, the window is determined as abnormal.

[0052] Further, the anomaly detection module combines the contrast learning method; in contrast learning, the pre-trained language model generates different mask versions for the input data, and learns abnormal features by comparing these versions; the process is represented as:

[0053]

[0054] wherein: L contrast is a contrastive learning loss, and denote the feature representations of two different mask versions, respectively.

[0055] Further, the step S5 comprises:

[0056] After each driving task, the edge device compares the anomaly detection result with the actual driving situation, generates error feedback information, and adjusts the anomaly detection threshold value through the method of sliding window; the adjustment process is as follows:

[0057] The edge device will maintain a sliding window with a length of w, which is used to store the recent false positive rate and false negative rate; at the end of each driving task, the false positive rate and the false negative rate of this task are stored in the sliding window; the data in the sliding window is counted to calculate the average false positive rate and the average false negative rate of the last w driving tasks; the threshold value is adjusted using a simple fixed step;

[0058]

[0059] wherein τ' is the updated anomaly detection threshold value, Δ τ is a fixed step, FPR thresh。ld and FNR threshold are the threshold values of the false positive rate and the false negative rate.

[0060] Further, the step S6 comprises:

[0061] According to the data characteristics, false positive rate and false negative rate of the device, different weights w i are assigned to each group of data, and the calculation formula is as follows:

[0062]

[0063] wherein FNR i and FPR i are the false negative rate and the false positive rate of device i, and α is a smoothing coefficient for controlling the sensitivity of the weight;

[0064] Then the sum of all weights is calculated and each initial weight is normalized to obtain the final weight:

[0065]

[0066] The data uploaded by each edge device is weighted and integrated using the weighting coefficient to construct a training set;

[0067]

[0068] wherein: X i is the data uploaded by the device i, w i is the corresponding weight;

[0069] The incremental training comprises:

[0070] After the aggregation is completed, the incremental training is performed using the feedback information; the incremental training only needs to update a small number of parameters, that is, the parameters of the PLM are frozen, and the field slice is retrained.

[0071] Compared with the prior art, the present application has the following beneficial effects:

[0072] 1. The present application is based on a pre-trained language model and a cloud-edge-end collaborative architecture, and establishes an efficient data completion, anomaly detection and model optimization mechanism between edge devices and cloud servers. The system comprises an edge device module and a cloud server module, and the two modules work collaboratively through a network to jointly complete data acquisition, real-time detection and model optimization.

[0073] 2. The present application has strong data completion and noise processing capability; combined with the completion rules of the field slice, the edge device can complete the completion of missing data and the filtering of noise while performing real-time anomaly detection, ensuring that the quality of the input data is not affected by the network condition; the cloud end uses global data (the data of multiple edge devices, more rich context information) to perform fine processing on the completion, so as to improve the integrity and reliability of the data, and provide a solid data foundation for the anomaly detection of the model.

[0074] 3. The present application has a dynamic balance between individualization and global optimization; error update and field slice optimization: the error update mechanism after each driving makes the model gradually adapt to the individualized needs of the edge device. The cloud end performs incremental training on the field slice through weighted aggregation, realizes comprehensive optimization of different environments and user behaviors, and thus balances between individualization and global adaptation.

[0075] 4. The present application realizes efficient real-time updating through cloud-edge collaboration, and through the small parameter update and incremental training mechanism of the field slice, the model can maintain rapid response to environmental changes without occupying a large amount of bandwidth. This collaborative updating mode ensures the real-time performance and detection accuracy of the edge device, and effectively controls the use of edge computing resources.

[0076] 5. The present application can perform continuous detection and asynchronous updating; the asynchronous updating mechanism ensures that the detection capability of the edge device will not be affected by network problems. When the network condition recovers, the model can automatically obtain the latest field slice, realize continuous optimization of the detection capability, and ensure the stability and continuity of the system. BRIEF DESCRIPTION OF DRAWINGS

[0077] Figure 1 Workflow diagram of the cloud-edge-end time series anomaly detection system based on a pre-trained language model;

[0078] Figure 2 Result diagram of anomaly detection example 1;

[0079] Figure 3 Result diagram of anomaly detection example 2;

[0080] Figure 4 Result diagram of anomaly detection example 3;

[0081] Figure 5 Result diagram of anomaly detection example 4. DETAILED DESCRIPTION

[0082] It should be noted that the relational terms herein, such as first and second and the like, are used solely to distinguish one from another entity or action, without necessarily requiring or implying any such actual relationship or order between such entities or actions. Moreover, the terms "comprises", "comprising", or any other variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but can include other elements not expressly listed or inherent to such process, method, article, or apparatus. An element proceeded by "comprises... a" does not, without more constraints, exclude the existence of additional identical elements in the process, method, article, or apparatus that comprises the element.

[0083] The features and advantages of the present application will be further described in the following embodiments.

[0084] Embodiment One

[0085] Referring to Figure 1 The cloud-edge-end time series anomaly detection system based on a pre-trained language model specifically comprises the following modules:

[0086] A data acquisition module, a sequence completion module, an anomaly detection module, and an error collection module are arranged at the edge device end, and a weighted aggregation module, an incremental training module, and a domain slicing update module are arranged at the cloud server end;

[0087] The data acquisition module is responsible for acquiring data of various sensors in the driving process, such as acceleration, vehicle speed, GPS position, etc.

[0088] The sequence completion module completes the missing data based on a fine-tuned pre-trained language model; the pre-trained language model uses a domain slicing technology to adapt to the data characteristics in a specific scenario;

[0089] The abnormality detection module utilizes the fine-tuned pre-training language model to perform abnormality detection on the completed data; abnormal driving behaviors such as sudden braking and sudden acceleration are identified;

[0090] The error collection module records error information between the abnormality detection result and the actual situation after the driving task is completed, so as to upload to the cloud;

[0091] The weighted aggregation module receives error information from multiple edge devices, and performs weighted aggregation on the error information, so as to make full use of feedback information from different scenes;

[0092] The incremental training module optimizes the domain slice of the fine-tuned pre-training language model based on the aggregated data in an incremental training manner, so as to enhance the adaptability of the model to diversified data;

[0093] The domain slice update module distributes the updated domain slice to the edge device, so as to continuously improve the detection capability of the edge device.

[0094] The application also proposes a cloud-edge-end time series abnormality detection method based on a pre-training language model, based on the cloud-edge-end time series abnormality detection system, and specifically includes the following steps:

[0095] Step S1: data acquisition; the edge device acquires vehicle data in real time through a sensor;

[0096] Step S2: selecting a domain slice; the pre-training language model utilizes prior knowledge to select a domain slice;

[0097] Step S3: data completion; if there is missing data in the sensor data, the sequence completion module will complete the data under the prior condition of the domain slice, to ensure the integrity of the data;

[0098] Step S4: real-time abnormality detection; after the data is completed, the data is input into the abnormality detection module to perform abnormality detection on the driving behavior; in the abnormality detection, the application innovatively combines the domain slice and contrast learning; the prior knowledge provided by the domain slice and the deep features provided by the contrast learning enable the model to better distinguish between normal and abnormal data, and to exhibit higher robustness and detection accuracy in diversified scenes;

[0099] Step S5: error feedback and local update; after each driving task is completed, the abnormality detection result is compared with the actual situation, error information is generated, and the error information is uploaded to the cloud server together with the road data when the network allows;

[0100] Step S6: cloud incremental training; the cloud server aggregates the received error information by weighting, and optimizes the pre-trained language model by incremental training to update the domain slice; the application dynamically aggregates the false positive rate and the false negative rate of the edge device by the cloud; at the same time, the edge device performs incremental local update to improve the real-time adaptability to the scene and reduce the dependence on the cloud;

[0101] Step S7: domain slice distribution; the updated domain slice is distributed to the edge device to realize continuous optimization; that is, only the parameters in the domain slice are updated to avoid full training, thereby significantly reducing the computing cost.

[0102] In the embodiment, specifically, the step S1 comprises:

[0103] Step S11: sensor data collection; the edge device collects various time series data in real time through the vehicle-mounted sensor; for example, speed, acceleration, GPS position, etc.; in the actual scene, the data may have noise or missing due to environmental interference;

[0104] Step S12: sequence instance normalization and slicing; the collected data is subjected to instance normalization, and the value of each time series variable is normalized into a standardized form with a mean of zero and a variance of one; then the data subjected to instance normalization is divided into a plurality of non-overlapping segments, each segment will be embedded into a hidden space for subsequent processing; each segment represents a local time series, which can make the model more flexible to process local features of the data.

[0105] In the embodiment, it should be noted that each domain slice contains embedding information of a specific scene, and the embedding information is inserted into each layer of the model in the form of a domain prefix to guide the behavior of the model in the domain. For example, urban roads, highways and rural roads have different driving characteristics, and the superposition of Fourier coding and original data coding is mapped through a multi-layer perception (MLP) to map the superimposed embedding into prefix information.

[0106] Given the domain-specific embedding vector D embed ∈R d , the Fourier coding obtains a frequency domain representation:

[0107]

[0108] Where: ω is a frequency parameter, is the feature after Fourier coding;

[0109] The superimposed feature after Fourier coding and the original embedding feature forms a composite coding feature:

[0110]

[0111] The composite encoding features are mapped using a multi-layer perception (MLP) to obtain the final prefix information. Assuming that the weights of the MLP are W1∈R 3d×h and W2∈R h×d′ , and the activation function is ReLU, the mapping process is as follows:

[0112] H = ReLU (W1·E combined + b1)

[0113] P prefix = W2·H + b2

[0114] where H is the output of the hidden layer, P prefix ∈R d′ is the final obtained prefix information;

[0115] For each layer l of the pre-trained language model (PLM), the prefix information P prefix obtained by combining the Fourier encoding with the original encoding is inserted into the attention mechanism of the model as the (Key) and (ValUe) of each layer:

[0116] K′ l = [K l , P prefix ]

[0117] V′ l = [V l , P prefix ]

[0118] where K l and V l are the original key and value, K′ l and V′ l are the Key and Value after inserting the domain prefix;

[0119] In each layer, the prefix information P prefix and the prompt vector P init ∈R m×d′ initialized randomly are combined to generate the final prefix vector:

[0120] P l = P prefix + P init

[0121] In this embodiment, specifically, the step S2 comprises:

[0122] In the edge device, a domain slice library is pre-stored, and the domain slices in the domain slice library are trained and optimized by the cloud and then distributed to the edge device; each domain slice contains a prefix vector Pprefix The pre-trained language model discriminates the scene and selects the domain slice through the input data.

[0123] The sequence completion module in the edge device operates based on the fine-tuned pre-trained language model. The model is fine-tuned for a specific scene through the domain slice. To deal with the missing problem of time series data, the scheme designs a missing embedding mechanism;

[0124] That is, in the embodiment, specifically, the step S3 comprises:

[0125] The data completion is performed based on the completion mechanism and the completion strategy of the domain slice.

[0126] In the embodiment, specifically, the completion mechanism comprises:

[0127] In the input data, if the missing is detected, the pre-trained language model uses a learnable embedding to represent the missing position. The embedding combines the context information to enable the pre-trained language model to more effectively complete the missing data.

[0128] The completion strategy comprises:

[0129] Simple completion: for simple missing cases, the pre-trained language model performs quick completion based on the statistical features in the domain slice; for example, the missing data can be completed using historical mean, maximum value, minimum value or trend interpolation method;

[0130] Deep inference completion: for complex missing data, the pre-trained language model combines context features and deep learning methods to complete the missing part by inference; the pre-trained language model generates reasonable completion values using fine-tuned pre-trained features:

[0131] Y t =f θ (P t ,E miss ,C t )

[0132] Where: f θ is the function of the completion model, defined by the parameter θ, P t is the current segment, E miss is the missing embedding, and C t is the context information.

[0133] In the embodiment, it is also necessary to explain that the fine-tuning mechanism of the domain slice is as follows:

[0134] The pre-trained language model is fine-tuned in each specific scenario (such as urban roads, highways, and rural road sections) using different domain slices. The input data is processed using Fourier encoding to generate a set of encoding specific to the domain.

[0135] The cloud-end issued P prefix is combined with Fourier encoding, so that the input data has the superposition of domain knowledge and frequency domain features, thereby ensuring that the model can accurately distinguish between normal and abnormal behavior in different scenarios:

[0136] P combined = P prefix +F Fourier

[0137] wherein P prefix is the prefix vector trained by the cloud, and F Fourier is the Fourier encoded feature.

[0138] In this embodiment, specifically, the step S4 comprises:

[0139] The completed data will be input into the anomaly detection module; a sliding window strategy is used to make overall anomaly judgment on the data; the length of the sliding window is set to w, and the time series data is operated in a sliding manner; for the data X t in each sliding window, a deep feature representation Z t is extracted by the pre-trained language model.

[0140] Z t =F θ (X t )

[0141] wherein f θ is the pre-trained model, θ is the parameter of the model, and Z t is the feature representation; for the extracted deep feature representation Z t , the anomaly score of each time point in the window is calculated; an anomaly score function g(·) is set, and the scheme adopts reconstruction error; for each time point x i ∈X t , the anomaly score is:

[0142] A i =g(Z t , x i )

[0143] wherein A i is the anomaly score of time point x i ; an anomaly score threshold τ is defined, and if the anomaly score A i of time point x i is greater than the threshold, the point is marked as abnormal:

[0144]

[0145] For the entire sliding window X t If all points are detected as abnormal, i.e.,

[0146]

[0147] where: I window represents the abnormal flag of the entire window; only when all points in the window are marked as abnormal, the window is determined as abnormal.

[0148] In order to improve the model's ability to distinguish between normal and abnormal features, the anomaly detection module combines a contrast learning method.

[0149] In this embodiment, specifically, the anomaly detection module combines a contrast learning method; in contrast learning, the pre-trained language model generates different mask versions for the input data, and learns abnormal features by comparing these versions; this process is represented as:

[0150]

[0151] where: L contrast is the contrast learning loss, and represent the feature representations of the two different mask versions, respectively.

[0152] In this embodiment, specifically, the step S5 comprises:

[0153] After each driving task is completed, the edge device compares the anomaly detection result with the actual driving situation, generates error feedback information, and adjusts the anomaly detection threshold value through the sliding window method; the adjustment process is as follows:

[0154] The edge device will maintain a sliding window with a length of w, which is used to store the recent false positive rate (FPR) and false negative rate (FNR); at the end of each driving task, the false positive rate and false negative rate of this task are stored in the sliding window; the data in the sliding window is counted, and the average false positive rate and average false negative rate of the last w driving tasks are calculated; a simple fixed step is used to adjust the threshold value;

[0155]

[0156] where τ' is the updated anomaly detection threshold value, Δ τ is a fixed step, FPR threshold and FNR thresholdThresholds for false positive rate and false negative rate

[0157] The method can adapt to feature changes in different driving scenarios through simple addition and subtraction adjustment, reduce false positive rate and false negative rate, and thus improve detection accuracy and robustness.

[0158] The cloud server receives error feedback data and road data from multiple edge devices, and the error information of each device may come from different driving scenarios. The cloud server classifies the data according to the scenario label, for example, all data from “urban road” are aggregated together to form a unified data set.

[0159] In this embodiment, specifically, the step S6 comprises:

[0160] According to the data characteristics, false positive rate, false negative rate and other indicators of the device, different weights w are allocated to each group of data i The calculation formula is:

[0161]

[0162] Where: FNR i and FPR i are the false negative rate and false positive rate of device i, and α is a smoothing coefficient for controlling the sensitivity of the weight;

[0163] Then calculate the sum of all weights And normalize each initial weight to get the final weight:

[0164]

[0165] The data uploaded by each edge device is weighted and integrated using the weighting coefficient to construct a training set;

[0166]

[0167] Where: X i is the data uploaded by device i, and w i is the corresponding weight;

[0168] The incremental training comprises:

[0169] After aggregation, the feedback information is used for incremental training; incremental training only needs to update a small amount of parameters, i.e. freeze the parameters of the PLM and retrain the field slice, which can improve the training speed and reduce the occupation of computing resources.

[0170] In this embodiment, specifically, the optimized domain slice is transmitted to the edge device through the network. After the edge device receives the new domain slice, it directly replaces the old version, and uses the latest domain slice in the subsequent task to improve the detection accuracy.

[0171] Embodiment two

[0172] This embodiment visualizes the anomaly detection results of the model, selects four abnormal windows on different edge datasets, and obtains the corresponding anomaly scores after the score prediction process. Then, the original data points in these abnormal windows are projected onto a two-dimensional plane using t-SNE. As shown in Figs. Figure 2 、 3 , 4, 5, the method proposed in the present application successfully distinguishes the anomaly and separates the abnormal data points from the normal data points.

[0173] The above embodiments only express the specific implementation of the present application, and the description is more specific and detailed, but it cannot be understood as a limitation on the protection scope of the present application. It should be pointed out that for ordinary skilled in the art, without departing from the technical concept of the present application, a number of modifications and improvements can be made, which are all within the protection scope of the present application.

[0174] This background section is provided to generally present the context of the application, the work of the current named inventors, the work described in this background section to the extent that it is described, and aspects described in this section that are not prior art at the time of filing, are neither expressly nor impliedly admitted to be prior art of the present application.

Claims

1. A cloud-edge time series anomaly detection system based on a pre-trained language model, characterized by: include: The data acquisition module, sequence completion module, anomaly detection module, and error collection module are set up on the edge device side, and the weighted aggregation module, incremental training module, and domain slice update module are set up on the cloud server; The data acquisition module is responsible for collecting data from various sensors during driving; The sequence completion module completes the missing data based on a fine-tuned pre-trained language model; The anomaly detection module uses a fine-tuned pre-trained language model to perform anomaly detection on the completed data; The error collection module records the error information between the abnormality detection result and the actual situation after the driving task is completed; The weighted aggregation module receives error information from multiple edge devices and performs weighted aggregation on the error information to fully utilize feedback information from different scenarios; The incremental training module optimizes the domain slice of the fine-tuned pre-trained language model using incremental training based on the aggregated data; The domain slice update module sends the updated domain slice to the edge device.

2. A cloud-edge time series anomaly detection method based on a pre-trained language model, characterized by: The cloud-edge time series anomaly detection system according to claim 1 comprises: Step S1: Data collection: The edge device collects vehicle data in real time through sensors; Step S2: Select domain slices; the pre-trained language model uses its own prior knowledge to select domain slices; Step S3: Data completion: If there is any missing sensor data, the sequence completion module will complete it under the prior conditions of the domain slice to ensure the integrity of the data; Step S4: Real-time anomaly detection: After completion, the data is input into the anomaly detection module to detect anomalies in driving behavior; Step S5: Error feedback and local update: After each driving task, the abnormal detection results are compared with the actual situation, and error information is generated and uploaded to the cloud server together with the road data when the network allows; Step S6: Cloud-based incremental training: The cloud server performs weighted aggregation on the received error information, optimizes the pre-trained language model through incremental training, and updates the domain slice. Step S7: Domain slices are distributed; the updated domain slices are distributed to edge devices to achieve continuous optimization.

3. The cloud-edge time series anomaly detection method based on a pre-trained language model according to claim 2 is characterized in that: The step S1 comprises: Step S11: Sensor data collection: The edge device collects multiple time series data in real time through the vehicle-mounted sensors; Step S12: sequence instance normalization and segmentation; perform instance normalization on the collected data, normalize the value of each time series variable to a standardized form with a mean of zero and a variance of unit; then divide the instance-normalized data into multiple non-overlapping segments, each of which will be embedded in the latent space for subsequent processing; each segment represents a local time series.

4. The cloud-edge time series anomaly detection method based on a pre-trained language model according to claim 3 is characterized in that: The step S2 includes: In the edge device, a domain slice library is pre-stored. The domain slices in the domain slice library are sent to the edge device after cloud training and optimization. Each domain slice contains a prefix vector P for a specific scenario. prefix ; The pre-trained language model performs scene discrimination on the input data and then selects the domain slice.

5. The cloud-edge time series anomaly detection method based on a pre-trained language model according to claim 4 is characterized in that: The step S3 comprises: Data completion is performed based on the completion mechanism and completion strategy of domain slices.

6. The cloud-edge time series anomaly detection method based on a pre-trained language model according to claim 5 is characterized in that: The completion mechanism includes: If missing data is detected in the input data, the pre-trained language model uses a learnable embedding to represent the missing location. This embedding combined with contextual information enables the pre-trained language model to more effectively complete the missing data. The completion strategy includes: Simple completion: For simple missing cases, the pre-trained language model quickly completes the missing information based on the statistical features in the domain slice. Deep inference completion: For complex missing data, the pre-trained language model combines contextual features and deep learning methods to complete the missing parts through inference. The pre-trained language model uses fine-tuned pre-trained features to generate reasonable completion values: Y t =f θ (P t ,E miss ,C t ) Where: f θ is a function that completes the model, defined by the parameter θ, P t is the current fragment, E miss is the missing embedding, C t It is context information.

7. The cloud-edge time series anomaly detection method based on a pre-trained language model according to claim 6 is characterized in that: The step S4 comprises: The completed data will be input into the anomaly detection module; the sliding window strategy is used to judge the overall anomaly of the data; the sliding window length is set to w, and the time series data is sliding; for each data X in the sliding window t , extract deep feature representation Z through pre-trained language model t ; Z t =f θ (X t ) Where: f θ is the pre-trained model, θ is the parameter of the model, Z t is the feature representation; for the extracted deep feature representation Z t , calculate the anomaly score at each time point in the window; set an anomaly score function g(·), and use the reconstruction error; for each time point x i ∈X t , the anomaly score is: A i =g(Z t ,x i ) Among them: A i is the time point x i Anomaly score; define an anomaly score threshold τ, if the time point x i Anomaly score A i If the value is greater than the threshold, the point is marked as an anomaly: For the entire sliding window X t , if all points are detected as anomalies, that is: Among them: I window Indicates the abnormal flag of the entire window; a window is considered abnormal only when all points in the window are marked as abnormal.

8. The cloud-edge time series anomaly detection method based on a pre-trained language model according to claim 7 is characterized in that: The anomaly detection module incorporates contrastive learning, where a pre-trained language model generates different masked versions of input data and learns anomaly features by comparing these versions. The process is expressed as: Where: L contrast is the contrastive learning loss, and Represent the feature representations of two different mask versions respectively.

9. The cloud-edge time series anomaly detection method based on a pre-trained language model according to claim 8 is characterized in that: The step S5 comprises: After each driving task, the edge device compares the anomaly detection results with the actual driving situation, generates error feedback information, and adjusts the anomaly detection threshold using a sliding window method. The adjustment process is as follows: The edge device maintains a sliding window of length w to store the most recent false alarm and missed alarm rates. At the end of each driving mission, the false alarm and missed alarm rates for that mission are stored in the sliding window. Statistics are then collected from the data within the sliding window to calculate the average false alarm and missed alarm rates for the last w driving missions. The threshold is then adjusted using a simple fixed step size. Among them, τ′ is the updated anomaly detection threshold, Δτ is a fixed step size, and FPR threshold and FNR threshold is the threshold of false positive rate and false negative rate.

10. The cloud-edge time series anomaly detection method based on a pre-trained language model according to claim 9 is characterized in that: The step S6 comprises: Assign different weights w to each group of data based on the data characteristics, false alarm rate, and missed alarm rate indicators of the device. i , the calculation formula is expressed as: Among them: FNR i and FPR i are the missed alarm rate and false alarm rate of device i, respectively. α is the smoothing coefficient, which is used to control the sensitivity of the weight. Then calculate the sum of all weights And normalize each initial weight to get the final weight: Use weighting coefficients to perform weighted integration on the data uploaded by each edge device to construct a training set; Where: X i is the data uploaded by device i, w i is the corresponding weight; The incremental training comprises: After the aggregation is completed, the feedback information is used for incremental training; incremental training only needs to update a small number of parameters, that is, freeze the parameters of the PLM and retrain the domain slice.

Images