Distributed vehicle sensor system anomaly detection method based on streaming data
By leveraging the collaborative work of a distributed vehicle sensor system, the anomaly detection of high-dimensional data streams was optimized using matrix sketching algorithm and singular value decomposition. This solved the problems of time complexity degradation and lack of distributed collaborative detection in existing technologies, achieving efficient anomaly identification and distributed expansion.
Patent Information
- Application Number
- CN202511418958.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-09-30
- Publication Date
- 2026-01-09
AI Technical Summary
In existing technologies, anomaly detection methods based on network log stream data suffer from time complexity degradation and lack of distributed collaborative detection mechanisms in high-dimensional data streams.
A distributed vehicle sensor system is adopted, which optimizes the time complexity and achieves distributed expansion by coordinating the work of a coordinator and multiple sensor sites, utilizing a matrix sketch algorithm and a sliding window update mechanism, combined with singular value decomposition and synchronous power iteration.
It achieves efficient identification of abnormal behavior in vehicle sensor systems, reduces the amortized time complexity of each update to O(dl log l), and has distributed scalability.
Smart Images
Figure CN121309075A_ABST
Abstract
Description
Technical Field
[0001] This disclosure relates to the field of computers, and more specifically, to an anomaly detection method for a distributed vehicle sensor system based on streaming data. Background Technology
[0002] In existing technologies, the patent "A Method for Detecting Anomaly Behavior in Network Space Based on Network Log Stream Data" discloses an innovative method for detecting anomaly behavior in network logs, specifically designed for efficiently identifying anomalies in computer networks such as distributed denial-of-service (DDoS) attacks or internal data leaks. This method processes high-dimensional data streams through five consecutive steps: First, it initializes a reference window matrix sketch and calculates the principal component of the earliest input vector; second, for each subsequent input vector, it updates the test window matrix sketch using an optimal matrix sketch algorithm on a sliding window. This algorithm achieves theoretically optimal space complexity through a multi-layered structure (including primary and auxiliary sketches) and dynamic management of historical snapshot queues. The method optimizes resource utilization. The third step utilizes MGKL divergence as a change scoring function to quantify the probability density estimate difference score between the reference window and the test window, specifically calculated by estimating the mean vector and covariance matrix. The fourth step sets a dynamic confidence threshold based on hypothesis testing (e.g., pH detection). If the difference score exceeds the threshold, an anomaly is reported and the reference window is adaptively updated. Simultaneously, the threshold increases or decreases with false positive events to improve accuracy. Finally, new inputs are continuously processed, and the output includes anomaly judgment, confidence score, matrix sketch, and probability distribution parameters, facilitating the discovery of anomaly attributes and characteristic causes. The innovation lies in improving the matrix sketch algorithm, overcoming the limitations of large space overhead and fixed reference windows in existing technologies, achieving a trade-off between computational accuracy and storage efficiency in resource-constrained scenarios. This method has the following two drawbacks:
[0003] 1. For a data stream that generates vectors of dimension d, given an adjustable parameter l, the amortized time complexity of each update is O(dl+l). 3 For high-dimensional data streams, this typically requires l << d. If the parameter l is chosen to be close to d, the amortized time complexity will degenerate to O(d). 3 ).
[0004] 2. The anomaly detection algorithm proposed in this patent is limited to a centralized computing paradigm and does not involve a collaborative detection mechanism under a distributed architecture.
[0005] Therefore, it is necessary to propose an anomaly detection method for distributed vehicle sensor systems based on streaming data to address the performance loss caused by the degradation of time complexity in existing algorithms, while also possessing distributed scalability. Summary of the Invention
[0006] The purpose of this disclosure is to provide an anomaly detection method for a distributed vehicle sensor system based on streaming data, so as to solve the problem that existing methods do not involve a collaborative detection mechanism under a distributed architecture.
[0007] In one general aspect, an anomaly detection method for a distributed vehicle sensor system based on streaming data is provided, consisting of a coordinator Cord and sensor systems of m vehicles;
[0008] Each vehicle's sensor system continuously generates log data streams, which are then processed at a dedicated station denoted as S1, S2, ..., S. m The i-th station S i The data stream being processed is a i1 ,a i2 ..., where each data vector a it Let S be a vector of dimension d; a channel is established between the coordinator and each station for distributed communication, and a configurable hyperparameter l is set; each station S i It is necessary to maintain the local cumulative Frobenius norm F. i Global cumulative Frobenius norm A local matrix sketch D and a matrix sketch mEH maintained by an exponential histogram, site S i Process the vector a arriving in the data stream it The process includes five steps:
[0009] Step 1: Cold start phase. For a given window size W, the coordinator Cord collects a matrix consisting of the earliest W input vectors from all distributed sites via channels as a reference window W. r =[x1,x2,…,x w The principal components are calculated by singular value decomposition, and the reference window matrix K1 composed of the first 21 singular values and singular vectors is retained.
[0010] Step 2: Maintain the local cumulative Frobenius norm F i and global cumulative Frobenius norm
[0011] Step 3: Update the local matrix sketch D using the matrix sketch algorithm on a distributed sliding window, and return two matrices, denoted as Z and ZCC;
[0012] Step 4: Send matrices Z and ZCC to the coordinator Cord via the channel to notify the coordinator Cord to update the global test window matrix sketch K2;
[0013] Step 5: Convert the vector a itUsed to update the index histogram matrix sketch mEH, if the timestamp of the element e with the smallest timestamp in the updated mEH is e. t The time is less than the current time minus the window size W, and the site is S. i The element e is sent to the coordinator Cord via the channel, notifying the coordinator to perform an expired discard operation on the element; when K2 maintained by the coordinator is updated, the output is a judgment on whether an abnormal behavior event of the vehicle sensor system has occurred.
[0014] The maintenance of the local cumulative Frobenius norm F i and global cumulative Frobenius norm The specific method is as follows:
[0015] Step 2.1: For the currently arriving vector a in the data stream it Calculate its vector 2-norm ||a it ||2, Update variable F i for
[0016] Step 2.2: If the updated Site S i Send F through the channel to the coordinator Cord j Notify the coordinator Cord to update the global [data / data]. Then F i Set to 0;
[0017] Step 2.3: If a synchronization update is detected from the coordinator If the broadcast message has the value, then update
[0018] The method for updating the local matrix sketch D using the optimal matrix sketch algorithm on a sliding window is as follows:
[0019] Step 3.1: Transfer vector a it Insert into the row of all zeros in the two-dimensional array D;
[0020] Step 3.2: If there are no rows of all zeros in D, perform a Fast Frequent Directions operation, that is, decompose the matrix D into singular values to obtain U,Σ=diag(σ1,σ2,…,σ 2l+1 ),V, will Σ 2 Through the l-th largest singular value σ l The difference between the squares of the two matrices is used to truncate the new diagonal matrix Σ', which is... And update the new matrix to D = Σ'V;
[0021] Step 3.3: Obtain matrix Z by applying the synchronous exponential iteration method to matrix D;
[0022] Step 3.4: Update the matrix
[0023] Step 3.5: Return the matrix Z obtained in Step 3.3 and the calculated matrix.
[0024] The specific process of the synchronous power iteration method is as follows:
[0025] a) Given parameters: low-rank approximation rank k, relative error coefficient ε SI Initialize the power exponent. And a random Gaussian matrix Π of l×d, wherein each element of matrix Π is independently sampled from a random Gaussian distribution with mean 0 and variance 1;
[0026] b) Calculate the matrix
[0027] c) Perform QR decomposition on matrix G to obtain matrices Q and R;
[0028] d) Calculate the matrix Performing eigenvalue decomposition on a positive definite matrix M yields an eigenvalue diagonal matrix. and eigenvector matrix U;
[0029] e) Calculate and return the matrix Z = QU.
[0030] The communication protocol used by the channel of the coordinator Cord is as follows:
[0031] If you receive site S i Send F via channel to coordinator Cord j Notify the coordinator to update the global [data / data]. If a request message is received, the following processing steps are performed: First, update the global Frobenius norm. Simultaneously, increment the message counter #msg by 1; secondly, if #msg ≥ m, where m is the number of stations, broadcast a synchronization update to all stations. The message with the value of #msg is then reset to 0;
[0032] If you receive site S i If the matrix Z and ZCC are sent to the coordinator Cord via the channel, requesting the coordinator to update the global test window matrix sketch K2, the following processing steps are executed: First, calculate the matrix... Secondly, eigenvalue decomposition is performed on the positive definite matrix covK to obtain the eigenvalue diagonal matrix Σ and the eigenvector matrix V; finally, Σ... 2 Through the l-th largest singular value σ lThe difference between the squares of the two matrices is used to truncate the new diagonal matrix Σ', which is... Then update K2 = Σ'V;
[0033] If site S i If the element e is sent to the coordinator Cord via the channel, requesting the coordinator to perform an expired and discarded operation on the element, the following processing steps are executed: First, calculate the matrix. (× represents the cross product operation); secondly, perform eigenvalue decomposition on the positive definite matrix covK to obtain the eigenvalue diagonal matrix Σ and the eigenvector matrix V; finally, update K2 = Σ'V.
[0034] When K2 maintained by the coordinator is updated, the following steps are performed:
[0035] Step 1: Estimate the probability density of the reference window K1 by varying the scoring function MGKL divergence. Probability density estimation of K2 with the test window Calculate the quantitative difference score c.
[0036] Step 2: Quantify the confidence threshold θ for the occurrence of network anomalous behavior events by hypothesis testing to determine whether the data stream distribution has changed. If c KL If the value is greater than the threshold θ, an anomaly detection event is reported, and the matrix sketch K2 of the current test window is used as the new reference window matrix sketch K1.
[0037] Step 3: Output the judgment of whether an abnormal behavior event has occurred, and the confidence level m of the hypothesis test to make this judgment. t Reference window W r The corresponding matrix sketch K1 and the given probability density estimate Test window W t The corresponding matrix sketch K2 and the given probability density estimate Quantify the difference score c.
[0038] The difference score is calculated by using a matrix sketch to estimate the probability density distribution. The parameters include the mean vector μ1, covariance matrix K1, and probability density distribution estimation. The parameters are the mean vector μ2, the covariance matrix K2, and the specific calculation form of the MGKL difference score is as follows:
[0039]
[0040] The method for determining the confidence threshold in step 2 is as follows: taking "the data in the reference window K1 and the test window K2 come from the same distribution" as the null hypothesis, and using PH detection to calculate the confidence level of this hypothesis; that is, maintaining a cumulative variable m. t :
[0041]
[0042] Where c t' The difference score is calculated for the MGKL divergence at the current time. The moving average of the difference scores between the reference window K1 and the test window K2 over a period of time is used, and δ is the tolerance of the detection system to change, set to a value close to 0; when When an anomaly is detected, an anomaly detection event is reported, and the matrix sketch K2 of the current test window is used as the new reference window matrix sketch K1.
[0043] The technical effects to be achieved by the embodiments of the present invention are as follows:
[0044] Efficiently identify anomalous behavior in the distributed data stream of a vehicle sensor system, reducing the amortized time complexity of each update from O(dl+l) 3 It is optimized to O(dl log l) and has distributed scalability. Attached Figure Description
[0045] The above and other objects and features of this disclosure will become clearer from the following description taken in conjunction with the accompanying drawings.
[0046] Figure 1 This is a schematic diagram illustrating the architecture of an anomaly detection method for a distributed vehicle sensor system according to an embodiment of the present disclosure;
[0047] Figure 2 This is a schematic diagram illustrating a site workflow according to an embodiment of the present disclosure;
[0048] Figure 3 This is a schematic diagram illustrating the coordinator workflow according to an embodiment of the present disclosure. Detailed Implementation
[0049] The following detailed embodiments are provided to aid the reader in gaining a comprehensive understanding of the methods, apparatus, and / or systems described herein. However, various changes, modifications, and equivalents of the methods, apparatus, and / or systems described herein will become apparent upon understanding this disclosure. For example, the order of operations described herein is merely illustrative and is not limited to those orders set forth herein, but may be changed as will become clear upon understanding this disclosure, except for operations that must occur in a specific order. Furthermore, for clarity and conciseness, descriptions of features known in the art may be omitted.
[0050] The features described herein may be implemented in different forms and should not be construed as limited to the examples described herein. Rather, the examples described herein are provided only to illustrate some of the many feasible ways of implementing the methods, apparatus, and / or systems described herein, which will become clear upon understanding the disclosure of this application.
[0051] As used herein, the term “and / or” includes any one of the associated listed items and any combination of any two or more.
[0052] Although terms such as “first,” “second,” and “third” may be used herein to describe various components, assemblies, regions, layers, or parts, these components, assemblies, regions, layers, or parts should not be limited by these terms. Rather, these terms are used only to distinguish one component, assembly, region, layer, or part from another. Thus, without departing from the teaching of the examples described herein, the first component, first assembly, first region, first layer, or first part referred to as the first component, first assembly, first region, first layer, or first part may also be referred to as the second component, second assembly, second region, second layer, or second part.
[0053] In the specification, when an element (such as a layer, region, or substrate) is described as being "on" another element, "connected to," or "bonded to" another element, the element may be directly "on" another element, directly "connected to," or "bonded to" the other element, or one or more other elements may be present in between. Conversely, when an element is described as being "directly on" another element, "directly connected to," or "directly bonded to" another element, no other elements may be present in between.
[0054] The terminology used herein is for the purpose of describing various examples only and is not intended to limit disclosure. Unless the context clearly indicates otherwise, the singular form is intended to include the plural form as well. The terms “comprising,” “including,” and “having” indicate the presence of the described features, quantities, operations, components, elements, and / or combinations thereof, but do not preclude the presence or addition of one or more other features, quantities, operations, components, elements, and / or combinations thereof.
[0055] Unless otherwise defined, all terms used herein (including technical and scientific terms) shall have the same meaning as commonly understood by one of ordinary skill in the art to which this disclosure pertains upon understanding this disclosure. Unless expressly defined herein, terms (such as those defined in a general dictionary) shall be interpreted as having a meaning consistent with their meaning in the context of the relevant field and in this disclosure, and shall not be interpreted in an idealized or overly formalistic manner.
[0056] Furthermore, in the description of the examples, detailed descriptions of well-known related structures or functions will be omitted when it is believed that such detailed descriptions would lead to a vague interpretation of this disclosure.
[0057] Figure 1 This is a schematic diagram illustrating an anomaly detection method for a distributed vehicle sensor system based on streaming data according to an embodiment of the present disclosure.
[0058] To achieve the aforementioned objectives, the present invention employs the following technical framework: Figure 1 As shown.
[0059] The purpose of this disclosure is to provide an anomaly detection method for a distributed vehicle sensor system based on a stream data matrix sketch.
[0060] In general, this paper provides an anomaly detection method for a distributed vehicle sensor system based on streaming data. The distributed system consists of a coordinator Cord and sensor systems from multiple vehicles, totaling m vehicles. The log data streams continuously generated by the sensor systems of each vehicle are assigned to a dedicated station for processing, denoted as S1, S2, ..., S... m The i-th station S i The data stream being processed is a i1 ,a i2 ..., where each line is a vector of dimension d. Channels are established between the coordinator and each station for distributed communication. An optional hyperparameter l is also available.
[0061] For each site S i It is necessary to maintain the local cumulative Frobenius norm F. i (Initial value is 0), Global cumulative Frobenius norm (Initial value 0), a local matrix sketch D (a two-dimensional array of dimension 2l×d, with each element initially valued at 0), and a matrix sketch mEH maintained by an exponential histogram. Site S i Process the vector a arriving in the data stream it The process includes five steps:
[0062] Step 1: Cold Start Phase: For a given window size W, the coordinator Cord needs to collect a matrix consisting of the earliest W input vectors from all distributed sites via channels as a reference window W. r =[x1,x2,…,x w The principal components are calculated through singular value decomposition, and a reference window matrix sketch K1 composed of the first 2l singular values and singular vectors is retained. The cold start phase ends, and subsequent operations do not require the execution of the cold start phase.
[0063] Step 2: Maintain the local cumulative Frobenius norm F i and global cumulative Frobenius norm
[0064] Step 3: Update the local matrix sketch D using the matrix sketch algorithm on a distributed sliding window. This step returns two matrices, denoted as Z and ZCC.
[0065] Step 4: Send matrices Z and ZCC to the coordinator Cord via the channel to notify the coordinator to update the global test window matrix sketch k2.
[0066] Step 5: Transfer vector a it Used to update the index histogram matrix sketch mEH. If the updated mEH contains the element with the smallest timestamp e, then the timestamp e is... t The time is less than the current time minus the window size W, and the site is S. i The element e is sent to the coordinator Cord via the channel, notifying the coordinator to perform an expired and discarded operation on the element.
[0067] In step 2, the local cumulative Frobenius norm F is maintained. i and global cumulative Frobenius norm Including steps 2.1-2.2:
[0068] Step 2.1: For the currently arriving vector a in the data stream it Calculate its vector 2-norm ||a it ||2, Update variable F i for
[0069] Step 2.2: If the updated Site S i Send F via channel to coordinator Cord j Notify the coordinator to update the global [data / data]. Then F i Set to 0.
[0070] Step 2.3: If a synchronization update is detected from the coordinator If the broadcast message has the value, then update
[0071] The method for updating the local matrix sketch D using the optimal matrix sketch algorithm on the sliding window in step 3 includes steps 3.1-3.5:
[0072] Step 3.1: Place a it Insert into the row of all zeros in the two-dimensional array D;
[0073] Step 3.2: If there are no rows of all zeros in D, perform a Fast Frequent Directions operation, that is, decompose the matrix D into singular values to obtain U,Σ=diag(σ1,σ2,…,σ 2l+1 ),V, will Σ 2 Through the l-th largest singular value σ l The difference between the squares of the two matrices is used to truncate the new diagonal matrix Σ', which is... And update the new matrix to D = Σ'V;
[0074] Step 3.3: Obtain matrix Z by applying the synchronous exponential iteration method to matrix D;
[0075] Step 3.4: Update the matrix
[0076] Step 3.5: Return the matrix Z obtained in Step 3.3 and the calculated matrix.
[0077] The synchronous power iteration method in step 3.3 is specifically as follows:
[0078] a) Given parameters: the low-rank approximate rank k (based on l and The value of ε is determined by the doubling detection method, and the relative error coefficient ε is also determined. SI Initialize the power exponent. Given an l×d random Gaussian matrix Π, where each element is independently sampled from a random Gaussian distribution with mean 0 and variance 1.
[0079] b) Calculate the matrix
[0080] c) Perform QR decomposition on matrix G to obtain matrices Q and R;
[0081] d) Calculate the matrix Performing eigenvalue decomposition on a positive definite matrix M yields an eigenvalue diagonal matrix. And the eigenvector matrix U.
[0082] e) Calculate and return the matrix Z = QU.
[0083] The coordinator listens for messages from all stations via a channel. The communication protocol design for the coordinator's processing and response to different message types is as follows:
[0084] 1. If you receive a message from site S i Send F via channel to coordinator Cord j Notify the coordinator to update the global [data / data]. If a request message is received, the following processing steps will be performed:
[0085] a) Update the global Frobenius norm At the same time, increment the message counter #msg by 1.
[0086] b) If #msg ≥ m, where m is the number of sites, then broadcast a synchronization update to all sites. The message contains the value of #msg, which is then reset to 0.
[0087] 2. If you receive a message from site S i If the matrix Z and ZCC are sent to the coordinator Cord via the channel, requesting the coordinator to update the global test window matrix sketch k2, then the following processing steps will be executed:
[0088] a) Calculate the matrix
[0089] b) Perform eigenvalue decomposition on the positive definite matrix covK to obtain the eigenvalue diagonal matrix Σ and the eigenvector matrix V.
[0090] c) Σ 2 Through the l-th largest singular value σ l The difference between the squares of the two matrices is used to truncate the new diagonal matrix Σ', which is... Then update K2 = Σ'V.
[0091] 3. If site S i If the element e is sent to the coordinator Cord via the channel, notifying the coordinator of the request to perform an expired discard operation on the element, the following processing steps will be executed:
[0092] a) Calculate the matrix (× represents the vector outer product operation)
[0093] b) Perform eigenvalue decomposition on the positive definite matrix covK to obtain the eigenvalue diagonal matrix Σ and the eigenvector matrix V.
[0094] c) Update K2 = Σ'V
[0095] When K2, maintained by the coordinator, is updated, the following steps are performed:
[0096] Step 1: Estimate the probability density of the reference window K1 by varying the scoring function MGKL divergence. Probability density estimation of K2 with the test window Calculate the quantitative difference score c.
[0097] Step 2: Quantify the confidence threshold θ for the occurrence of network anomalous behavior events by hypothesis testing to determine whether the data stream distribution has changed. If c KLIf the value is greater than the threshold θ, an anomaly detection event is reported, and the matrix sketch K2 of the current test window is used as the new reference window matrix sketch K1.
[0098] Step 3: Output the judgment of whether an abnormal behavior event has occurred, and the confidence level m of the hypothesis test to make this judgment. t Reference window W r The corresponding matrix sketch K1 and the given probability density estimate Test window W t The corresponding matrix sketch K2 and the given probability density estimate Quantify the difference score c.
[0099] The method for calculating the difference score in step 1 is to use a matrix sketch to estimate the probability density distribution. The parameters include the mean vector μ1, covariance matrix K1, and probability density distribution estimation. The parameters are the mean vector μ2, the covariance matrix K2, and the specific calculation form of the MGKL difference score is as follows:
[0100]
[0101] The method for determining the confidence threshold in step 2 is as follows: taking "the data in the reference window K1 and the test window K2 come from the same distribution" as the null hypothesis, and using PH detection to calculate the confidence level of this hypothesis; that is, maintaining a cumulative variable m. t :
[0102]
[0103] Where c t' The difference score is calculated for the MGKL divergence at the current time. The moving average of the difference scores between the reference window K1 and the test window K2 over a period of time is used, and δ is the tolerance of the detection system to change, set to a value close to 0; when When an anomaly is detected, an anomaly detection event is reported, and the matrix sketch K2 of the current test window is used as the new reference window matrix sketch K1.
[0104] In step 2, the confidence threshold θ is set as a dynamic threshold. When the detected abnormal event is a false positive, the threshold θ is doubled, that is, the new threshold θ' = 2θ; at other times, the threshold θ will gradually decrease over time.
[0105] While some embodiments of this disclosure have been shown and described, those skilled in the art will understand that modifications may be made to these embodiments without departing from the principles and spirit of this disclosure, which are defined by the claims and their equivalents.
Claims
1. An anomaly detection method for a distributed vehicle sensor system based on streaming data, characterized in that, It consists of a coordinator Cord and a sensor system for m vehicles; Each vehicle's sensor system continuously generates log data streams, which are then processed at a dedicated station denoted as S1, S2, ..., S. m The i-th station S i The data stream being processed is a i1 ,a i2 ..., where each data vector a it Let S be a vector of dimension d; a channel is established between the coordinator and each station for distributed communication, and a configurable hyperparameter l is set; each station S i It is necessary to maintain the local cumulative Frobenius norm F. i Global cumulative Frobenius norm A local matrix sketch D and a matrix sketch mEH maintained by an exponential histogram, site S i Process the vector a arriving in the data stream it The process includes five steps: Step 1: Cold start phase. For a given window size W, the coordinator Cord collects a matrix consisting of the earliest W input vectors from all distributed sites via channels as a reference window W. r =[x1,x2,…,x w The principal components are calculated by singular value decomposition, and the reference window matrix K1 composed of the first 21 singular values and singular vectors is retained. Step 2: Maintain the local cumulative Frobenius norm F i and global cumulative Frobenius norm Step 3: Update the local matrix sketch D using the matrix sketch algorithm on a distributed sliding window, and return two matrices, denoted as Z and ZCC; Step 4: Send matrices Z and ZCC to the coordinator Cord via the channel to notify the coordinator Cord to update the global test window matrix sketch K2; Step 5: Convert the vector a it Used to update the index histogram matrix sketch mEH, if the timestamp of the element e with the smallest timestamp in the updated mEH is e. t The time is less than the current time minus the window size W, and the site is S. i The element e is sent to the coordinator Cord via the channel, notifying the coordinator to perform an expired discard operation on the element; when K2 maintained by the coordinator is updated, the output is a judgment on whether an abnormal behavior event of the vehicle sensor system has occurred.
2. The anomaly detection method for a distributed vehicle sensor system based on streaming data as described in claim 1, characterized in that, The maintenance of the local cumulative Frobenius norm F i and global cumulative Frobenius norm The specific method is as follows: Step 2.1: For the currently arriving vector a in the data stream it Calculate its vector 2-norm ||a it ||2, Update variable F i for Step 2.2: If the updated Site S i Send F through the channel to the coordinator Cord j Notify the coordinator Cord to update the global [data / data]. Then F i Set to 0; Step 2.3: If a synchronization update is detected from the coordinator If the broadcast message has the value, then update 3. The anomaly detection method for a distributed vehicle sensor system based on streaming data as described in claim 2, characterized in that, The method for updating the local matrix sketch D using the optimal matrix sketch algorithm on a sliding window is as follows: Step 3.1: Transfer vector a it Insert into the row of all zeros in the two-dimensional array D; Step 3.2: If there are no rows of all zeros in D, perform a Fast Frequent Directions operation, that is, decompose the matrix D into singular values to obtain U,Σ=diag(σ1,σ2,…,σ 2l+1 ),V, will Σ 2 Through the l-th largest singular value σ l The difference between the squares of the two matrices is used to truncate the new diagonal matrix Σ', which is... And update the new matrix to D = Σ'V; Step 3.3: Obtain matrix Z by applying the synchronous exponential iteration method to matrix D; Step 3.4: Update matrix D = D - DZZ T ; Step 3.5: Return the matrix Z obtained in Step 3.3 and the calculated matrix ZDD = Z. T D T D.
4. The anomaly detection method for a distributed vehicle sensor system based on streaming data as described in claim 1, characterized in that, The specific process of the synchronous power iteration method is as follows: a) Given parameters: low-rank approximation rank k, relative error coefficient ε SI Initialize the power exponent. And a random Gaussian matrix Π of l×d, wherein each element of matrix Π is independently sampled from a random Gaussian distribution with mean 0 and variance 1; b) Calculate matrix G = (DD) T ) q DΠ; c) Perform QR decomposition on matrix G to obtain matrices Q and R; d) Calculate matrix M = Q T DD T Q. Perform eigenvalue decomposition on a positive definite matrix M to obtain an eigenvalue diagonal matrix. and eigenvector matrix U; e) Calculate and return the matrix Z = QU.
5. The anomaly detection method for a distributed vehicle sensor system based on streaming data as described in claim 1, characterized in that, The communication protocol used by the channel of the coordinator Cord is as follows: If you receive site S i Send F via channel to coordinator Cord j Notify the coordinator to update the global [data / data]. If a request message is received, the following processing steps are performed: First, update the global Frobenius norm. Simultaneously, increment the message counter #msg by 1; secondly, if #msg ≥ m, where m is the number of stations, broadcast a synchronization update to all stations. The message with the value of #msg is then reset to 0; If you receive site S i If the matrix Z and ZCC are sent to the coordinator Cord via the channel, requesting the coordinator to update the global test window matrix sketch K2, the following processing steps are executed: First, calculate the matrix... Secondly, eigenvalue decomposition is performed on the positive definite matrix covK to obtain the eigenvalue diagonal matrix Σ and the eigenvector matrix V; finally, Σ... 2 Through the l-th largest singular value σ l The difference between the squares of the two matrices is used to truncate the new diagonal matrix Σ', which is... Then update K2 = Σ'V; If site S i If the element e is sent to the coordinator Cord via the channel, requesting the coordinator to perform an expired and discarded operation on the element, the following processing steps are executed: First, calculate the matrix. (× represents the cross product operation); secondly, perform eigenvalue decomposition on the positive definite matrix covK to obtain the eigenvalue diagonal matrix Σ and the eigenvector matrix V; finally, update K2 = Σ'V.
6. The anomaly detection method for a distributed vehicle sensor system based on streaming data as described in claim 1, characterized in that, When K2 maintained by the coordinator is updated, the specific steps are as follows: Step 1: Estimate the probability density of the reference window K1 by varying the scoring function MGKL divergence. Probability density estimation of K2 with the test window Calculate the quantitative difference score c. Step 2: Quantify the confidence threshold θ for the occurrence of network anomalous behavior events by hypothesis testing to determine whether the data stream distribution has changed. If c KL If the value is greater than the threshold θ, an anomaly detection event is reported, and the matrix sketch K2 of the current test window is used as the new reference window matrix sketch K1. Step 3: Output the judgment of whether an abnormal behavior event has occurred, and the confidence level m of the hypothesis test to make this judgment. t Reference window W r The corresponding matrix sketch K1 and the given probability density estimate Test window W t The corresponding matrix sketch K2 and the given probability density estimate Quantify the difference score c.
7. The anomaly detection method for a distributed vehicle sensor system based on streaming data as described in claim 6, characterized in that, The difference score is calculated by using a matrix sketch to estimate the probability density distribution. The parameters include the mean vector μ1, covariance matrix K1, and probability density distribution estimation. The parameters are the mean vector μ2, the covariance matrix K2, and the specific calculation form of the MGKL difference score is as follows: The method for determining the confidence threshold in step 2 is as follows: taking "the data in the reference window K1 and the test window K2 come from the same distribution" as the null hypothesis, and using PH detection to calculate the confidence level of this hypothesis; that is, maintaining a cumulative variable m. t : Where c t' The difference score is calculated for the MGKL divergence at the current time. The moving average of the difference scores between the reference window K1 and the test window K2 over a period of time is used, and δ is the tolerance of the detection system to change, set to a value close to 0; when When an anomaly is detected, an anomaly detection event is reported, and the matrix sketch K2 of the current test window is used as the new reference window matrix sketch K1.