Cloud edge end time sequence anomaly detection system and method based on pre-training language model

Abstract

The invention discloses a cloud side end time sequence anomaly detection system and method based on a pre-training language model, and relates to the field of anomaly detection in a cloud side end architecture, the system comprises an edge device module and a cloud server module, and the edge device module and the cloud server module work cooperatively through a network to jointly complete data acquisition, real-time detection and model optimization. The invention further provides a detection method. According to the invention, based on the pre-training language model and the cloud edge-end collaborative architecture, an efficient data completion, anomaly detection and model optimization mechanism is established between the edge device and the cloud server.

Description

Technical Field

[0001] The present invention relates to the field of anomaly detection in a cloud-edge-device architecture, and specifically to a cloud-edge-device time series anomaly detection system and method based on a pre-trained language model. By performing real-time analysis on time series data collected by vehicle sensors, the problem of missing data is solved and the accuracy and response speed of anomaly detection are improved. Background Art

[0002] The statements in this section merely provide background information related to the present disclosure and may not constitute prior art.

[0003] Time series anomaly detection has become an important research topic in many key fields (such as traffic monitoring, industrial Internet of Things, etc.). Existing technologies mainly include statistical methods, classical machine learning methods and deep learning methods. Statistical methods such as moving average, exponential smoothing, and autoregressive integrated moving average (ARIMA) models are usually used to capture trends and cycles in time series, but they have limitations when dealing with complex, nonlinear and high-dimensional data.

[0004] Traditional machine learning methods include clustering algorithms (such as k-means and density-based methods) and classification algorithms (such as decision trees and support vector machines). Although these methods have certain anomaly detection capabilities, they often perform poorly when faced with complex high-dimensional time series data and have weak processing capabilities for noise and missing data.

[0005] In recent years, deep learning methods have shown strong advantages in time series anomaly detection. Common deep learning techniques include autoencoders, variational autoencoders (VAE), recurrent neural networks (RNN), and long short-term memory networks (LSTM). These methods can effectively learn long-term dependencies in time series data and are particularly suitable for anomaly detection of nonlinear and high-dimensional data. In addition, generative adversarial networks (GAN) and deep reinforcement learning (DRL) methods have also been applied in the field of anomaly detection. However, these methods have high requirements for computing resources and face real-time challenges when applied on edge devices.

[0006] However, the data collected by on-board sensors is easily affected by environmental conditions and is often accompanied by missing data and noise, which significantly reduces the accuracy of anomaly detection. Traditional time series analysis methods have obvious shortcomings in dealing with noise and missing data, and the combination of cloud-edge architecture provides new possibilities for data completion and noise reduction. How to efficiently complete missing data and eliminate noise at the edge, while using the powerful computing power of the cloud to optimize data and correct models, is an important technical challenge to ensure model robustness and detection results.

[0007] At the same time, in applications such as driver behavior detection and traffic accident prediction, models need to be constantly updated to cope with environmental changes and differences in data distribution. Under the cloud-edge architecture, models are usually trained and updated in the cloud to maintain high accuracy, but they also need to be able to be quickly deployed to edge devices to meet real-time requirements. However, the data characteristics of different driving environments vary, which requires the model to be robust enough to cope with diverse data scenarios. Therefore, how to achieve efficient collaborative updates of models between the cloud and edge devices, which can both ensure the real-time response of the model and balance the computing resource limitations of edge devices, is a technical difficulty that requires in-depth research.

[0008] In order to cope with application scenarios that require real-time response, such as car accident detection, combining cloud-edge-end architecture has become an effective solution. The existing cloud-edge-end architecture generally consists of edge devices and cloud servers working together, with edge devices responsible for collecting and initially processing data, while the cloud performs more complex analysis and model training. Summary of the invention

[0009] The present invention provides a cloud-edge time series anomaly detection system and method based on a pre-trained language model, which aims to achieve dynamic optimization and efficient detection through domain slicing and sequence completion; the edge device is pre-installed with basic models and domain slicing when it leaves the factory, performs local completion, noise processing and personalized local optimization, and generates error feedback after each driving; the cloud performs weighted aggregation of data from multiple edge devices, optimizes domain slicing through incremental training, and then sends the updated slicing to the edge device to ensure the global optimization and personalized capabilities of the model. This solution takes into account real-time, adaptability and computing cost, and provides solid technical support for efficient anomaly detection in complex driving scenarios.

[0010] The technical solution of the present invention is as follows:

[0011] The cloud-edge time series anomaly detection system based on the pre-trained language model includes:

[0012] The data acquisition module, sequence completion module, anomaly detection module and error collection module are arranged on the edge device side, and the weighted aggregation module, incremental training module and domain slice update module are arranged on the cloud server;

[0013] The data acquisition module is responsible for collecting data from various sensors during driving;

[0014] The sequence completion module completes the missing data based on a fine-tuned pre-trained language model;

[0015] The anomaly detection module uses a fine-tuned pre-trained language model to perform anomaly detection on the completed data;

[0016] The error collection module records the error information between the abnormal detection result and the actual situation after the driving task is completed;

[0017] The weighted aggregation module receives error information from multiple edge devices and performs weighted aggregation on the error information to fully utilize feedback information from different scenarios;

[0018] The incremental training module optimizes the domain slices of the fine-tuned pre-trained language model using incremental training based on the aggregated data;

[0019] The domain slice update module sends the updated domain slice to the edge device.

[0020] The present invention also proposes a cloud-edge-terminal time series anomaly detection method based on a pre-trained language model, based on the above-mentioned cloud-edge-terminal time series anomaly detection system, including:

[0021] Step S1: Data collection: The edge device collects vehicle data in real time through sensors;

[0022] Step S2: Select domain slices; the pre-trained language model uses its own prior knowledge to select domain slices;

[0023] Step S3: Data completion: If there is any missing sensor data, the sequence completion module will complete it under the prior conditions of the domain slice to ensure the integrity of the data;

[0024] Step S4: Real-time anomaly detection: After completion, the data is input into the anomaly detection module to detect anomalies in driving behavior;

[0025] Step S5: Error feedback and local update: After each driving task is completed, the abnormal detection result is compared with the actual situation, and error information is generated and uploaded to the cloud server together with the road data when the network allows;

[0026] Step S6: Cloud incremental training: The cloud server performs weighted aggregation on the received error information, optimizes the pre-trained language model through incremental training, and updates the domain slice;

[0027] Step S7: Send the domain slices; send the updated domain slices to the edge devices to achieve continuous optimization.

[0028] Furthermore, the step S1 comprises:

[0029] Step S11: sensor data collection: the edge device collects various time series data in real time through the vehicle-mounted sensors;

[0030] Step S12: sequence instance normalization and segmentation; perform instance normalization on the collected data, normalize the value of each time series variable to a standardized form with a mean of zero and a variance of unit; then divide the instance-normalized data into multiple non-overlapping segments, each of which will be embedded in the latent space for subsequent processing; each segment represents a local time series.

[0031] Furthermore, the step S2 comprises:

[0032] In the edge device, a domain slice library is pre-stored. The domain slices in the domain slice library are sent to the edge device after cloud training and optimization. Each domain slice contains a prefix vector P for a specific scenario. prefix ; The pre-trained language model performs scene discrimination on the input data and then selects domain slices.

[0033] Furthermore, the step S3 comprises:

[0034] Data completion is performed based on the completion mechanism and completion strategy of domain slices.

[0035] Furthermore, the completion mechanism includes:

[0036] If missing data is detected in the input data, the pre-trained language model uses a learnable embedding to represent the missing location. This embedding combined with contextual information enables the pre-trained language model to more effectively complete the missing data.

[0037] The completion strategy includes:

[0038] Simple completion: For simple missing cases, the pre-trained language model quickly completes the missing information based on the statistical features in the domain slice.

[0039] Deep inference completion: For complex missing data, the pre-trained language model combines contextual features and deep learning methods to complete the missing parts through inference; the pre-trained language model uses fine-tuned pre-trained features to generate reasonable completion values:

[0040] Y t =f θ (P t , E miss , C t )

[0041] Where: f θ is the function that completes the model, defined by the parameter θ, P t is the current segment, E miss is the missing embedding, C t It is context information.

[0042] Furthermore, the step S4 comprises:

[0043] The completed data will be input into the anomaly detection module; the sliding window strategy is used to judge the overall anomaly of the data; the sliding window length is set to w, and the time series data is slidingly operated; for each data X in the sliding window t , extract deep feature representation Z through pre-trained language model t ;

[0044] Z t =F θ (X t )

[0045] Where: f θ is the pre-trained model, θ is the parameter of the model, Z t is the feature representation; for the extracted deep feature representation Z t , calculate the anomaly score at each time point in the window; set an anomaly score function g(·), and use the reconstruction error; for each time point x i ∈X t , the anomaly score is:

[0046] A i =g(Z t , x i )

[0047] Among them: A i is time point x i Anomaly score; define an anomaly score threshold τ, if time point x i The abnormal score A i If the value is greater than the threshold, the point is marked as an anomaly:

[0048]

[0049] For the entire sliding window X t , if all points are detected as abnormal, that is:

[0050]

[0051] Where: I window Indicates the abnormal flag of the entire window; the window is considered abnormal only when all points in the window are marked as abnormal.

[0052] Furthermore, the anomaly detection module combines a contrastive learning method; in contrastive learning, the pre-trained language model generates different masked versions for the input data and learns the anomaly features by comparing these versions; this process is expressed as:

[0053]

[0054] Where: L contrast For contrastive learning loss, and Represent the feature representations of two different mask versions respectively.

[0055] Furthermore, the step S5 comprises:

[0056] After each driving task, the edge device compares the anomaly detection results with the actual driving situation, generates error feedback information, and adjusts the anomaly detection threshold through the sliding window method; the adjustment process is as follows:

[0057] The edge device maintains a sliding window of length w to store the most recent false alarm rate and missed alarm rate. At the end of each driving task, the false alarm rate and missed alarm rate of this task are stored in the sliding window. The data in the sliding window is counted to calculate the average false alarm rate and average missed alarm rate of the most recent w driving tasks. The threshold is adjusted using a simple fixed step size.

[0058]

[0059] Among them, τ′ is the updated anomaly detection threshold, Δ τ For a fixed step size, FPR thresh。ld and FNR threshold is the threshold of false positive rate and false negative rate.

[0060] Furthermore, the step S6 comprises:

[0061] Assign different weights w to each group of data according to the data characteristics, false alarm rate, and missed alarm rate indicators of the device. i , the calculation formula is expressed as:

[0062]

[0063] Among them: FNR i and FPR i are the missed alarm rate and false alarm rate of device i, respectively, and α is the smoothing coefficient, which is used to control the sensitivity of the weight;

[0064] Then calculate the sum of all weights And normalize each initial weight to get the final weight:

[0065]

[0066] The data uploaded by each edge device is weighted and integrated using the weight coefficient to construct a training set;

[0067]

[0068] Where: X i is the data uploaded by device i, w i is the corresponding weight;

[0069] The incremental training comprises:

[0070] After aggregation is completed, the feedback information is used for incremental training; incremental training only needs to update a small number of parameters, that is, freeze the parameters of PLM and retrain the domain slices.

[0071] Compared with the prior art, the present invention has the following beneficial effects:

[0072] 1. Based on the pre-trained language model and cloud-edge collaborative architecture, the present invention establishes an efficient data completion, anomaly detection and model optimization mechanism between edge devices and cloud servers. The system includes an edge device module and a cloud server module, which work together through the network to complete data collection, real-time detection and model optimization.

[0073] 2. The present invention has powerful data completion and noise processing capabilities; combined with the completion rules of domain slices, edge devices can complete the completion of missing data and noise filtering while performing anomaly detection in real time, ensuring that the quality of input data is not affected by network conditions; global data (data from multiple edge devices, richer contextual information) is used in the cloud to perform fine processing on the completion to improve data integrity and reliability, providing a solid data foundation for the model's anomaly detection.

[0074] 3. The present invention has a dynamic balance between personalization and global optimization; error update and domain slice optimization: the error update mechanism after each driving allows the model to gradually adapt to the individual needs of edge devices. The cloud performs incremental training on domain slices through weighted aggregation to achieve comprehensive optimization of different environments and user behaviors, thereby achieving a balance between personalization and global adaptation.

[0075] 4. The present invention achieves efficient real-time updates through cloud-edge collaboration. Through small parameter updates and incremental training mechanisms of domain slices, the model can maintain a rapid response to environmental changes without occupying a large amount of bandwidth. This collaborative update method ensures the real-time performance and detection accuracy of edge devices, while effectively controlling the use of edge computing resources.

[0076] 5. The present invention can perform continuous detection and asynchronous updates; the asynchronous update mechanism ensures that the detection capability of edge devices will not be affected by network problems. When the network conditions are restored, the model can automatically obtain the latest domain slices, realize continuous optimization of detection capabilities, and ensure the stability and continuity of the system. BRIEF DESCRIPTION OF THE DRAWINGS

[0077] Figure 1 This is the workflow diagram of the cloud-edge time series anomaly detection system based on the pre-trained language model;

[0078] Figure 2 This is the result diagram of anomaly detection example 1;

[0079] Figure 3 This is the result diagram of anomaly detection example 2;

[0080] Figure 4 This is the result diagram of anomaly detection example 3;

[0081] Figure 5 This is the result graph of anomaly detection example 4. DETAILED DESCRIPTION

[0082] It should be noted that relational terms such as "first" and "second" are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.

[0083] The features and performance of the present invention are further described in detail below in conjunction with the embodiments.

[0084] Embodiment 1

[0085] See also Figure 1 , a cloud-edge time series anomaly detection system based on a pre-trained language model, specifically including the following modules:

[0086] The data acquisition module, sequence completion module, anomaly detection module and error collection module are arranged on the edge device side, and the weighted aggregation module, incremental training module and domain slice update module are arranged on the cloud server;

[0087] The data acquisition module is responsible for collecting data from various sensors during driving, such as acceleration, vehicle speed, GPS location, etc.

[0088] The sequence completion module completes the missing data based on a fine-tuned pre-trained language model; the pre-trained language model uses domain slicing technology to adapt to data features in specific scenarios;

[0089] The anomaly detection module uses a fine-tuned pre-trained language model to perform anomaly detection on the completed data; identifying abnormal driving behaviors such as sudden braking and sudden acceleration;

[0090] After the driving task is completed, the error collection module records the error information between the abnormal detection result and the actual situation so as to upload it to the cloud;

[0091] The weighted aggregation module receives error information from multiple edge devices and performs weighted aggregation on the error information to fully utilize feedback information from different scenarios;

[0092] The incremental training module optimizes the domain slices of the fine-tuned pre-trained language model using incremental training based on the aggregated data, thereby enhancing the adaptability of the model to diverse data;

[0093] The domain slice update module sends the updated domain slice to the edge device to continuously improve the detection capability of the edge device.

[0094] The present invention also proposes a cloud-edge-terminal time series anomaly detection method based on a pre-trained language model. Based on the above-mentioned cloud-edge-terminal time series anomaly detection system, the method specifically includes the following steps:

[0095] Step S1: Data collection: The edge device collects vehicle data in real time through sensors;

[0096] Step S2: Select domain slices; the pre-trained language model uses its own prior knowledge to select domain slices;

[0097] Step S3: Data completion: If there is any missing sensor data, the sequence completion module will complete it under the prior conditions of the domain slice to ensure the integrity of the data;

[0098] Step S4: Real-time anomaly detection; after completion, the data is input into the anomaly detection module to detect anomalies in driving behavior; in anomaly detection, the present invention innovatively combines domain slicing and contrastive learning. The prior knowledge provided by domain slicing and the deep features provided by contrastive learning enable the model to better distinguish between normal and abnormal data, and show higher robustness and detection accuracy in diversified scenarios;

[0099] Step S5: Error feedback and local update: After each driving task is completed, the abnormal detection result is compared with the actual situation, and error information is generated and uploaded to the cloud server together with the road data when the network allows;

[0100] Step S6: incremental training in the cloud; the cloud server performs weighted aggregation on the received error information, optimizes the pre-trained language model through incremental training, and updates the domain slice; the present invention dynamically performs weighted aggregation on the false alarm rate and missed alarm rate of the edge device through the cloud; at the same time, the edge device performs incremental local updates to improve the real-time adaptability to the scene and reduce the dependence on the cloud;

[0101] Step S7: Send the domain slices; send the updated domain slices to the edge devices to achieve continuous optimization; that is, only update the parameters in the domain slices to avoid full training and significantly reduce computing costs.

[0102] In this embodiment, specifically, step S1 includes:

[0103] Step S11: sensor data collection: the edge device collects various time series data in real time through the vehicle-mounted sensors; such as speed, acceleration, GPS location, etc.; in actual scenarios, the data may be noisy or missing due to environmental interference;

[0104] Step S12: sequence instance normalization and segmentation; perform instance normalization on the collected data, normalize the value of each time series variable to a standardized form with a mean of zero and a variance of unit; then divide the instance-normalized data into multiple non-overlapping segments, each of which will be embedded in the latent space for subsequent processing; each segment represents a local time series, which allows the model to process local features of the data more flexibly.

[0105] In this embodiment, it should be noted that each domain slice contains embedded information of a specific scene, which is inserted into each layer of the model in the form of a domain prefix to guide the behavior of the model in that domain. For example, urban roads, highways, and rural roads all have different driving characteristics. The superposition of Fourier coding and original data coding is used, and a multi-layer perceptron (MLP) is used for mapping, and the superimposed embedding is mapped to prefix information.

[0106] Given a domain-specific embedding vector D embed ∈R d , Fourier encoding is used to obtain the frequency domain representation:

[0107]

[0108] Where: ω is the frequency parameter, is the feature after Fourier encoding;

[0109] The Fourier encoded features are superimposed on the original embedded features to form a composite encoded feature:

[0110]

[0111] Use a multi-layer perceptron (MLP) to map the composite coding features to obtain the final prefix information. Assume that the weight of the MLP is W1∈R 3d×h and W2∈R h×d′ , the activation function is ReLU, and the mapping process is as follows:

[0112] H = ReLU(W1·E combined +b1)

[0113] P prefix =W2·H+b2

[0114] Among them, H is the output of the hidden layer, P prefix ∈R d′ is the final prefix information;

[0115] For each layer l of the pre-trained language model (PLM), the prefix information P obtained by combining Fourier coding with the original coding is prefix Inserted into the model's attention mechanism as (Key) and (ValUe) for each layer:

[0116] K′ l =[K l , P prefix ]

[0117] V′ l =[V l , P prefix ]

[0118] Where K l and V l are the original key and value, K′ l and V′ l It is the Key and Value after inserting the domain prefix;

[0119] In each layer, the prefix information P is used prefix and a randomly initialized hint vector P init ∈R m×d′ Combine to generate the final prefix vector:

[0120] P l =P prefix +P init

[0121] In this embodiment, specifically, step S2 includes:

[0122] In the edge device, a domain slice library is pre-stored. The domain slices in the domain slice library are sent to the edge device after cloud training and optimization. Each domain slice contains a prefix vector P for a specific scenario.prefix ; The pre-trained language model performs scene discrimination on the input data and then selects domain slices.

[0123] The sequence completion module in the edge device operates based on a fine-tuned pre-trained language model. The model is fine-tuned for specific scenarios through domain slicing. In order to deal with the missing problem of time series data, the solution designs a missing embedding mechanism;

[0124] That is, in this embodiment, specifically, step S3 includes:

[0125] Data completion is performed based on the completion mechanism and completion strategy of domain slices.

[0126] In this embodiment, specifically, the completion mechanism includes:

[0127] If missing data is detected in the input data, the pre-trained language model uses a learnable embedding to represent the missing location. This embedding combined with contextual information enables the pre-trained language model to more effectively complete the missing data.

[0128] The completion strategy includes:

[0129] Simple completion: For simple missing cases, the pre-trained language model quickly completes the missing data based on the statistical features in the domain slice. For example, the missing data can be completed using the historical mean, maximum, minimum, or trend interpolation method.

[0130] Deep inference completion: For complex missing data, the pre-trained language model combines contextual features and deep learning methods to complete the missing parts through inference; the pre-trained language model uses fine-tuned pre-trained features to generate reasonable completion values:

[0131] Y t =f θ (P t , E miss , C t )

[0132] Where: f θ is the function that completes the model, defined by the parameter θ, P t is the current segment, E miss is the missing embedding, C t It is context information.

[0133] In this embodiment, it should also be noted that the fine-tuning mechanism of domain slices is as follows:

[0134] The pre-trained language model is fine-tuned using different domain slices in each specific scenario (such as urban roads, highways, and rural roads). The input data is processed using Fourier coding to generate a set of domain-specific codes.

[0135] The P sent from the cloud prefix Combined with Fourier coding, the input data has a superposition of domain knowledge and frequency domain features, thus ensuring that the model can accurately distinguish between normal and abnormal behaviors in different scenarios:

[0136] P combined =P prefix +F Fourier

[0137] Among them, P prefix is the prefix vector trained on the cloud, F Fourier is the feature after Fourier encoding.

[0138] In this embodiment, specifically, step S4 includes:

[0139] The completed data will be input into the anomaly detection module; the sliding window strategy is used to judge the overall anomaly of the data; the sliding window length is set to w, and the time series data is slidingly operated; for each data X in the sliding window t , extract deep feature representation Z through pre-trained language model t ;

[0140] Z t =F θ (X t )

[0141] Where: f θ is the pre-trained model, θ is the parameter of the model, Z t is the feature representation; for the extracted deep feature representation Z t , calculate the anomaly score at each time point in the window; set an anomaly score function g(·), and use the reconstruction error; for each time point x i ∈X t , the anomaly score is:

[0142] A i =g(Z t , x i )

[0143] Among them: A i is time point x i Anomaly score; define an anomaly score threshold τ, if time point x i The abnormal score A i If the value is greater than the threshold, the point is marked as an anomaly:

[0144]

[0145] For the entire sliding window X t , if all points are detected as abnormal, that is:

[0146]

[0147] Where: I window Indicates the abnormal flag of the entire window; the window is considered abnormal only when all points in the window are marked as abnormal.

[0148] To improve the model’s ability to distinguish between normal and abnormal features, the anomaly detection module combines contrastive learning methods.

[0149] In this embodiment, specifically, the anomaly detection module combines a contrastive learning method; in contrastive learning, the pre-trained language model generates different mask versions for the input data, and learns the anomaly features by comparing these versions; the process is expressed as:

[0150]

[0151] Where: L contrast For contrastive learning loss, and Represent the feature representations of two different mask versions respectively.

[0152] In this embodiment, specifically, step S5 includes:

[0153] After each driving task, the edge device compares the anomaly detection results with the actual driving situation, generates error feedback information, and adjusts the anomaly detection threshold through the sliding window method; the adjustment process is as follows:

[0154] The edge device maintains a sliding window of length w to store the most recent false positive rate (FPR) and false negative rate (FNR). At the end of each driving task, the false positive rate and false negative rate of this task are stored in the sliding window. The data in the sliding window is counted to calculate the average false positive rate and average false negative rate of the most recent w driving tasks. The threshold is adjusted using a simple fixed step size.

[0155]

[0156] Among them, τ′ is the updated anomaly detection threshold, Δ τ For a fixed step size, FPR threshold and FNR thresholdis the threshold of false alarm rate and missed alarm rate;

[0157] This method can adapt to feature changes in different driving scenarios through simplified addition and subtraction adjustments, reduce false alarm rates and missed alarm rates, and thus improve detection accuracy and robustness.

[0158] The cloud server receives error feedback data and road data from multiple edge devices. The error information of each device may come from different driving scenarios. The cloud server will classify the data according to the scene labels. For example, all data from "city roads" will be aggregated together to form a unified data set.

[0159] In this embodiment, specifically, step S6 includes:

[0160] Assign different weights w to each group of data based on the data characteristics, false alarm rate, missed alarm rate and other indicators of the device. i , the calculation formula is expressed as:

[0161]

[0162] Among them: FNR i and FPR i are the missed alarm rate and false alarm rate of device i, respectively, and α is the smoothing coefficient, which is used to control the sensitivity of the weight;

[0163] Then calculate the sum of all weights And normalize each initial weight to get the final weight:

[0164]

[0165] The data uploaded by each edge device is weighted and integrated using the weight coefficient to construct a training set;

[0166]

[0167] Where: X i is the data uploaded by device i, w i is the corresponding weight;

[0168] The incremental training comprises:

[0169] After the aggregation is completed, the feedback information is used for incremental training; incremental training only needs to update a small number of parameters, that is, freeze the parameters of PLM and retrain the domain slices, which can increase the training speed and reduce the use of computing resources.

[0170] In this embodiment, the optimized domain slice is transmitted to the edge device through the network. After receiving the new domain slice, the edge device directly replaces the old version with the new domain slice and uses the latest domain slice in subsequent tasks to improve detection accuracy.

[0171] Embodiment 2

[0172] This example visualizes the anomaly detection results of the model, selects four anomaly windows on different edge data sets, and obtains the corresponding anomaly scores after the score prediction process. Then, t-SNE is used to project the original data points in these anomaly windows onto a two-dimensional plane. Figure 2 , 3 As shown in Figures 4 and 5, the method proposed in the present invention successfully distinguishes the anomalies and separates the abnormal data points from the normal data points well.

[0173] The above-mentioned embodiments only express the specific implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the protection scope of the present application. It should be pointed out that, for ordinary technicians in this field, several variations and improvements can be made without departing from the technical solution concept of the present application, and these all belong to the protection scope of the present application.

[0174] This background section is provided to generally present the context of the invention, and the work of the presently named inventors, the work to the extent described in this background section, and aspects of this section that did not constitute prior art at the time of application are neither explicitly nor implicitly admitted to be prior art to the present invention.

Claims

1. A cloud-edge time series anomaly detection system based on a pre-trained language model, characterized by: include: The data acquisition module, sequence completion module, anomaly detection module and error collection module are arranged on the edge device side, and the weighted aggregation module, incremental training module and domain slice update module are arranged on the cloud server; The data acquisition module is responsible for collecting data from various sensors during driving; The sequence completion module completes the missing data based on a fine-tuned pre-trained language model; The anomaly detection module uses a fine-tuned pre-trained language model to perform anomaly detection on the completed data; The error collection module records the error information between the abnormal detection result and the actual situation after the driving task is completed; The weighted aggregation module receives error information from multiple edge devices and performs weighted aggregation on the error information to fully utilize feedback information from different scenarios; The incremental training module optimizes the domain slices of the fine-tuned pre-trained language model using incremental training based on the aggregated data; The domain slice update module sends the updated domain slice to the edge device.

2. A cloud-edge time series anomaly detection method based on a pre-trained language model, characterized in that: The cloud-edge-terminal time series anomaly detection system according to claim 1 comprises: Step S1: Data collection: The edge device collects vehicle data in real time through sensors; Step S2: Select domain slices; the pre-trained language model uses its own prior knowledge to select domain slices; Step S3: Data completion: If there is any missing sensor data, the sequence completion module will complete it under the prior conditions of the domain slice to ensure the integrity of the data; Step S4: Real-time anomaly detection: After completion, the data is input into the anomaly detection module to detect anomalies in driving behavior; Step S5: Error feedback and local update: After each driving task is completed, the abnormal detection result is compared with the actual situation, and error information is generated and uploaded to the cloud server together with the road data when the network allows; Step S6: Cloud incremental training: The cloud server performs weighted aggregation on the received error information, optimizes the pre-trained language model through incremental training, and updates the domain slice; Step S7: Send the domain slices; send the updated domain slices to the edge devices to achieve continuous optimization.

3. The cloud-edge time series anomaly detection method based on a pre-trained language model according to claim 2 is characterized in that: The step S1 comprises: Step S11: sensor data collection: the edge device collects various time series data in real time through the vehicle-mounted sensors; Step S12: sequence instance normalization and segmentation; perform instance normalization on the collected data, normalize the value of each time series variable to a standardized form with a mean of zero and a variance of unit; then divide the instance-normalized data into multiple non-overlapping segments, each of which will be embedded in the latent space for subsequent processing; each segment represents a local time series.

4. The cloud-edge time series anomaly detection method based on a pre-trained language model according to claim 3 is characterized in that: The step S2 comprises: In the edge device, a domain slice library is pre-stored. The domain slices in the domain slice library are sent to the edge device after cloud training and optimization. Each domain slice contains a prefix vector P for a specific scenario. prefix ; The pre-trained language model performs scene discrimination on the input data and then selects domain slices.

5. The cloud-edge time series anomaly detection method based on a pre-trained language model according to claim 4 is characterized in that: The step S3 comprises: Data completion is performed based on the completion mechanism and completion strategy of domain slices.

6. The cloud-edge time series anomaly detection method based on a pre-trained language model according to claim 5 is characterized in that: The completion mechanism includes: If missing data is detected in the input data, the pre-trained language model uses a learnable embedding to represent the missing location. This embedding combined with contextual information enables the pre-trained language model to more effectively complete the missing data. The completion strategy includes: Simple completion: For simple missing cases, the pre-trained language model quickly completes the missing information based on the statistical features in the domain slice. Deep inference completion: For complex missing data, the pre-trained language model combines contextual features and deep learning methods to complete the missing parts through inference; the pre-trained language model uses fine-tuned pre-trained features to generate reasonable completion values: Y t =f θ (P t ,E miss ,C t ) Where: f θ is the function that completes the model, defined by the parameter θ, P t is the current segment, E miss is the missing embedding, C t It is context information.

7. The cloud-edge time series anomaly detection method based on a pre-trained language model according to claim 6 is characterized in that: The step S4 comprises: The completed data will be input into the anomaly detection module; the sliding window strategy is used to judge the overall anomaly of the data; the sliding window length is set to w, and the time series data is slidingly operated; for each data X in the sliding window t , extract deep feature representation Z through pre-trained language model t ; Z t =f θ (X t ) Where: f θ is the pre-trained model, θ is the parameter of the model, Z t is the feature representation; for the extracted deep feature representation Z t , calculate the anomaly score at each time point in the window; set an anomaly score function g(·), and use the reconstruction error; for each time point x i ∈X t , the anomaly score is: A i =g(Z t ,x i ) Among them: A i is time point x i Anomaly score; define an anomaly score threshold τ, if time point x i The abnormal score A i If the value is greater than the threshold, the point is marked as an anomaly: For the entire sliding window X t , if all points are detected as abnormal, that is: Where: I window Indicates the abnormal flag of the entire window; the window is considered abnormal only when all points in the window are marked as abnormal.

8. The cloud-edge-end time series anomaly detection method based on a pre-trained language model according to claim 7, characterized in that: The anomaly detection module incorporates a contrastive learning approach; in contrastive learning, a pre-trained language model generates different masked versions of input data and learns anomaly features by comparing these versions; The process is expressed as: Where: L contrast For contrastive learning loss, and Represent the feature representations of two different mask versions respectively.

9. The cloud-edge-end time series anomaly detection method based on a pre-trained language model according to claim 8, characterized in that: The step S5 comprises: After each driving task, the edge device compares the anomaly detection results with the actual driving situation, generates error feedback information, and adjusts the anomaly detection threshold through the sliding window method; the adjustment process is as follows: The edge device maintains a sliding window of length w to store the most recent false alarm rate and missed alarm rate. At the end of each driving task, the false alarm rate and missed alarm rate of this task are stored in the sliding window. The data in the sliding window is counted to calculate the average false alarm rate and average missed alarm rate of the most recent w driving tasks. The threshold is adjusted using a simple fixed step size. Among them, τ′ is the updated anomaly detection threshold, Δτ is a fixed step size, and FPR threshold and FNR threshold is the threshold of false positive rate and false negative rate.

10. The cloud-edge-end time series anomaly detection method based on a pre-trained language model according to claim 9, characterized in that: The step S6 comprises: Assign different weights w to each group of data according to the data characteristics, false alarm rate, and missed alarm rate indicators of the device. i , the calculation formula is expressed as: Among them: FNR i and FPR i are the missed alarm rate and false alarm rate of device i, respectively, and α is the smoothing coefficient, which is used to control the sensitivity of the weight; Then calculate the sum of all weights And normalize each initial weight to get the final weight: The data uploaded by each edge device is weighted and integrated using the weight coefficient to construct a training set; Where: X i is the data uploaded by device i, w i is the corresponding weight; The incremental training comprises: After the aggregation is completed, the feedback information is used for incremental training; incremental training only needs to update a small number of parameters, that is, freeze the parameters of PLM and retrain the domain slices.

Images