Method for detecting rare abnormal signals in ellipsoid family observation background environment

Abstract

The invention discloses a method for detecting rare abnormal signals in an ellipsoid family observation background environment. The method comprises the following steps: collecting historical observation data in a working environment, forming an observation data matrix, and extracting three-dimensional features of the observation data matrix to obtain a feature matrix; a Wasserstein adversarial generative network is utilized to generate a simulated anomaly observation feature matrix; training a detection model by using a recurrent neural network; inputting the observation characteristic matrix during the pure noise period and the observation characteristic matrix during the abnormal occurrence period as training data, and training a 0 / 1 detection model; the detection model is used for detecting whether observation data is abnormal or not. Compared with an existing rare signal detection method, the method has the advantages that on the basis of ellipsoid family distribution characteristics of data, robust multi-dimensional features in a distribution family category are considered, multi-dimensional feature information is simply and efficiently utilized by means of a deep learning technology, features of rare signals are fully mined, and the efficient and robust rare abnormal signal detection method is created.

Description

Technical Field

[0001] The invention relates to the technical field of signal detection, and in particular to a method for detecting rare abnormal signals in an ellipsoid family observation background environment. Background Art

[0002] Important events (signals) such as financial fraud and enemy reconnaissance have a low frequency of occurrence and are likely to appear in a variety of application environments. If such signals are not discovered in time, they will bring great harm to society. Robust and effective detection algorithms can help related fields to detect anomalies in a timely manner, take timely measures, and thus avoid risks. Therefore, the research on robust rare signal detection methods is crucial. The low frequency of occurrence of signals, that is, the rarity of signals, makes detection more difficult; the complexity of the observation data collection environment makes the observation data usually not meet the characteristics of Gaussian distribution, and the distribution rate of data may appear in a larger range, so the detection algorithm needs to have a high robustness.

[0003] For the detection of rare abnormal signals, current results mainly focus on the research of detection methods in Gaussian observation environments. Although they can be directly applied to non-Gaussian observation environments, the detection performance is poor and unstable. When designing detection methods, existing results extract one-dimensional features of observation data and design detection methods based on these features, such as HigherCriticism. However, when the abnormal signal is a weak signal, that is, similar to the background signal of the observation environment, the one-dimensional feature has limited information, and existing detection is likely to identify it as a background signal, resulting in missed detection. Summary of the invention

[0004] The purpose of the present invention is to provide a method for detecting rare abnormal signals in an ellipsoid family observation background environment in order to solve the above-mentioned existing problems, so as to realize effective detection of various abnormal changes in different types of observation environments.

[0005] The technical solution of the present invention is as follows:

[0006] A method for detecting rare abnormal signals in an ellipsoid family observation background environment comprises the following steps:

[0007] Collect historical observation data in the working environment to form an observation data matrix, extract the three-dimensional features of the observation data matrix, and obtain a feature matrix; the three-dimensional features include energy features, uniformity features, and dispersion features;

[0008] The Wasserstein generative adversarial network is used to generate a simulated abnormal observation feature matrix. The observation feature matrix during the period when the abnormal signal appears is used as the input sample, and the WGAN is used to generate an observation feature matrix that is difficult to distinguish between true and false, so as to increase the sample size and diversity of the training samples.

[0009] The detection model is trained using a recurrent neural network; the observation feature matrix during the pure noise period and the observation feature matrix during the abnormality period are used as training data input to train the 0 / 1 detection model;

[0010] The detection model is used to detect whether the observed data is abnormal or not.

[0011] Furthermore, the calculation of the feature matrix includes the following steps:

[0012] Calculate the observation data matrix V i The energy eigenvector α=(α1,α2,…,α n ) T ,in:

[0013]

[0014] Calculate the observation data matrix V using the direction vector of the observation data i The uniformity eigenvector β=(β1,β2,…,β n ) T , where β i is the number of vectors whose distance from the i-th observation direction vector is less than or equal to r among all observation direction vectors;

[0015] Remember v i is the observation data matrix V i The direction vector of the i-th row of observation data, that is, the normalized vector is the set of direction vectors of all observation data Then β i That is the collection The number of elements in in:

[0016]

[0017] And ||·||2 is the vector 2-norm operator, |·| is the operator of the number of elements in a set, and r is a known adjustable parameter;

[0018] Calculate its discrete eigenvector γ=(γ1,γ2,…,γ n ) T , where γ i For collection The vector and the observation direction v i The ratio of the cosine value of the smallest angle to the cosine value of the largest angle, that is:

[0019]

[0020] And <·,·> is the inner product operator, min{·} is the operation of taking the minimum value in the set, and max{·} is the operation of taking the maximum value in the set; the observation data matrix V i The feature matrix W i for

[0021] W i =(α,β,γ).

[0022] Furthermore, the method of generating a simulated abnormal observation feature matrix using a Wasserstein adversarial generative network includes the following steps:

[0023] The observation matrix during the period when the abnormal signal appears As observations, use true samples to input the discriminant network f ω (·);

[0024] Input the random signal into the sample generation network g θ (·), as a simulated observation fake sample input to the discriminant network f ω (·) Perform training optimization;

[0025] Using the training set {X j}, i = 1, ..., m for adversarial training, and then obtain the feature matrix X generated by random noise ε j Generative network X j =g θ (ε), where θ is the network parameter.

[0026] Furthermore, the discriminant network f ω The parameter ω of (·) is updated by stochastic gradient ascent with the gradient:

[0027]

[0028] Where M is the number of samples, is the sample, ε i is a Gaussian white noise signal sample.

[0029] Furthermore, the sample generation network g θ The parameters θ of (·) are updated by stochastic gradient ascent with the gradient:

[0030]

[0031] Furthermore, the use of a recurrent neural network to train a detection model includes the following steps:

[0032] The feature matrix of pure noise observation {W1,W2,…,W k} and the observed feature matrix {X1,X2,…,X m+l} as two types of samples input into the recurrent neural network to train the detection function;

[0033] Take the measurement matrix to be tested for anomalies and calculate the characteristic matrix of the abnormal measurement matrix;

[0034] The feature matrix to be detected for abnormality is input into the detection network to obtain the detection result.

[0035] Compared with the prior art, the present invention has the following beneficial effects:

[0036] 1. A detection method for rare abnormal signals in an ellipsoid family observation background environment, considering the scene of observation data of ellipsoid family distribution, and multivariate Gaussian distribution, heavy-tailed multivariate t distribution, multivariate Laplace distribution, etc. all belong to the set of ellipsoid family distribution. In other words, the detection technology of the present application is applicable to different data scenarios;

[0037] 2. A detection method for rare abnormal signals in an ellipsoidal observation background environment. In data feature analysis, based on the ellipsoidal distribution characteristics of the data, the energy information and direction information of the observation vector are considered at the same time. In addition to the energy feature, two new feature metrics are proposed from the perspective of observation direction information: uniformity metric and discreteness metric. The use of multi-dimensional features helps to fully mine the abnormal signal information;

[0038] 3. A detection method for rare abnormal signals in an ellipsoid family observation background environment. The detection scheme is designed from the field of deep learning technology. Since the focus is on the detection technology of rare abnormal signals, and rare anomalies bring about changes in the distribution of observation features, the Wasserstein distance is specifically selected to measure the difference between the observation feature matrices, that is, WGAN is selected to generate simulated abnormal observation features; since the observation features are time series data, the RNN technology for time series data is selected to train the detection model. The introduction of deep learning technology avoids the design difficulty of traditional detectors that introduce multiple feature information at the same time, and ensures the comparability of the detector. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] Figure 1 This is a flow chart for calculating the observation feature matrix of the method of this application.

[0040] Figure 2 Flowchart for calculating the sample size and diversity of the abnormal observation features that supplement the method of this application.

[0041] Figure 3 This is a flow chart of a robust detection method for rare abnormal signals of the detection method of the present application. DETAILED DESCRIPTION

[0042] It should be noted that relational terms such as "first" and "second" are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.

[0043] The features and performance of the present invention are further described in detail below in conjunction with the embodiments.

[0044] See also Figure 1-3 , a detection method for rare abnormal signals in the ellipsoid family observation background environment, such as Figure 3 As shown, the following steps are included:

[0045] After collecting historical observation data in the working environment, the data is divided into two categories: one is a matrix of multiple observation data obtained in a pure noise observation environment The other type is a matrix of multiple observation data during the period of abnormality Where n is the total number of observation moments and p is the number of observation antennas;

[0046] Extract the three-dimensional features of the observation data matrix, where the energy feature is to calculate the energy value of each observation; the uniformity feature is to propose a new metric to measure the distribution uniformity of the observation direction; the discreteness feature is to propose a new metric to measure the distribution discreteness of the observation direction, that is, the three-dimensional features: energy feature, uniformity feature and discreteness feature constitute a feature matrix;

[0047] The Wasserstein Generative Adversarial Network (WGAN) is used to generate a simulated abnormal observation feature matrix (the observation feature matrix during the period when the abnormal signal appears); the observation feature matrix during the period when the abnormal signal appears is used as the input sample, and the WGAN is used to generate an observation feature matrix that is difficult to distinguish between true and false, so as to increase the sample size and diversity of the training samples;

[0048] The detection model is trained using a recurrent neural network; the observation feature matrix during the pure noise period and the observation feature matrix during the abnormality period are used as training data input to train the 0 / 1 detection model;

[0049] The detection model is used to detect whether the observed data is abnormal or not.

[0050] Attached Figure 1 Take a pure noise observation data matrix V i As an example, its characteristic matrix W is given i The calculation flow chart of this method is used to calculate all observation data matrices V i ,Y j ,i=1,…k,j=1,…,m, and the characteristic matrix of the observation matrix Z (the observation matrix to be detected for abnormal signals). j Y j The characteristic matrix of the abnormal observation matrix. The specific operations are as follows:

[0051] Calculate the observation data matrix V respectively i ,Y j ,i=1,…k,j=1,…,m’s characteristic matrix, V i =(v ij ) n×p represents a pure noise observation data matrix, Y j =(y ij ) n×p Represents an abnormal observation data matrix. The calculation of the feature matrix includes the following steps:

[0052] Calculate the observation data matrix V i The energy eigenvector α=(α1,α2,…,α n ) T ,in:

[0053]

[0054] Calculate the observation data matrix V using the direction vector of the observation data i The uniformity eigenvector β=(β1,β2,…,β n ) T , where β i is the number of vectors whose distance from the i-th observation direction vector is less than or equal to r among all observation direction vectors;

[0055] Remember v i is the observation data matrix V i The direction vector of the i-th row of observation data (the observation data at the i-th moment during the observation period), that is, the normalized vector is the set of direction vectors of all observation data Then β i That is the collection The number of elements in in:

[0056]

[0057] And ||·||2 is the vector 2-norm operator, |·| is the operator of the number of elements in a set, and r is a known adjustable parameter;

[0058] Calculate its discrete eigenvector γ=(γ1,γ2,…,γ n ) T , where γ i For collection The vector and the observation direction v i The ratio of the cosine value of the smallest angle to the cosine value of the largest angle, that is:

[0059]

[0060] And <·,·> is the inner product operator, min{·} is the operation of taking the minimum value in the set, and max{·} is the operation of taking the maximum value in the set; the observation data matrix V i The feature matrix W i for

[0061] W i =(α,β,γ).

[0062] As attached Figure 2 As shown, the data feature matrix X1, X2, …, X1 during the abnormal period is generated. m Feature matrix data X that is difficult to distinguish true from false m+1 ,X m+2 ,…,X m+l , to increase the diversity of data collected during the occurrence of anomalies. Using the Wasserstein adversarial generative network to generate a simulated anomaly observation feature matrix includes the following steps:

[0063] The observation matrix during the period when the abnormal signal appears As observations, use true samples to input the discriminant network f ω (·);

[0064] Input the random signal into the sample generation network g θ (·), as a simulated observation fake sample input to the discriminant network f ω (·) Perform training optimization;

[0065] Using the training set {X j}, i = 1, ..., m for adversarial training, and then obtain the feature matrix X generated by random noise ε j Generative network X j =g θ (ε), where θ is the network parameter. The specific algorithm process is shown in Algorithm 1:

[0066] Algorithm 1

[0067]

[0068] Training a detection model using a recurrent neural network consists of the following steps:

[0069] The feature matrix of pure noise observation {W1,W2,…,W k} and the observed feature matrix {X1,X2,…,X m+l} as two types of samples input into the recurrent neural network to train the detection function;

[0070] Take the measurement matrix to be tested for anomalies and calculate the characteristic matrix of the abnormal measurement matrix;

[0071] The feature matrix to be detected for abnormality is input into the detection network to obtain the detection result.

[0072] The above-mentioned embodiments only express the specific implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the protection scope of the present application. It should be pointed out that, for ordinary technicians in this field, several variations and improvements can be made without departing from the technical solution concept of the present application, and these all belong to the protection scope of the present application.

Claims

1. A method for detecting rare abnormal signals in an ellipsoid family observation background environment, characterized in that: The following steps are involved: Collect historical observation data in the working environment to form an observation data matrix, extract the three-dimensional features of the observation data matrix, and obtain a feature matrix; The three-dimensional features include energy features, uniformity features and dispersion features; Use Wasserstein adversarial generative network to generate simulated abnormal observation feature matrix; Taking the observation feature matrix during the period when the abnormal signal appears as the input sample, WGAN is used to generate an observation feature matrix that is difficult to distinguish between true and false, so as to increase the sample size and diversity of training samples; The detection model is trained using a recurrent neural network; the observation feature matrix during the pure noise period and the observation feature matrix during the abnormality period are used as training data input to train the 0 / 1 detection model; The detection model is used to detect whether the observed data is abnormal or not.

2. The method for detecting rare abnormal signals in an ellipsoid family observation background environment according to claim 1, characterized in that: The energy signature is calculated as follows: Calculate the observation data matrix V i The energy eigenvector α=(α1,α2,…,α n ) T ,in:

3. The method for detecting rare abnormal signals in an ellipsoid family observation background environment according to claim 1, characterized in that: The uniformity characteristic is calculated as follows: Calculate the observation data matrix V using the direction vector of the observation data i The uniformity eigenvector β=(β1,β2,…,β n ) T , where β i is the number of vectors whose distance from the i-th observation direction vector is less than or equal to r among all observation direction vectors; Remember v i is the observation data matrix V i The direction vector of the i-th row of observation data, that is, the normalized vector is the set of direction vectors of all observation data Then β i That is the collection The number of elements in in: And ||·||2 is the vector 2-norm operator, |·| is the operator of the number of elements in a set, and r is a known adjustable parameter.

4. The method for detecting rare abnormal signals in an ellipsoid family observation background environment according to claim 1, characterized in that: The calculation of the discreteness feature is as follows: Calculate its discrete eigenvector γ=(γ1,γ2,…,γ n ) T , where γ i For collection The vector and the observation direction v i The ratio of the cosine value of the smallest angle to the cosine value of the largest angle, that is: And <·,·> is the inner product operator, min{·} is the operation of taking the minimum value in the set, and max{·} is the operation of taking the maximum value in the set; the observation data matrix V i The feature matrix W i W i =(α,β,γ).

5. The method for detecting rare abnormal signals in an ellipsoid family observation background environment according to claim 1, characterized in that: The method of using the Wasserstein adversarial generative network to generate a simulated abnormal observation feature matrix comprises the following steps: The observation matrix during the period when the abnormal signal appears As observations, use true samples to input the discriminant network f ω (·); Input the random signal into the sample generation network g θ (·), as a simulated observation fake sample input to the discriminant network f ω (·) Perform training optimization; Using the training set {X j }, i = 1, ..., m for adversarial training, and then obtain the feature matrix X generated by random noise ε j Generative network X j =g θ (ε), where θ is the network parameter; Discriminant network f ω The parameter ω of (·) is updated by stochastic gradient ascent with the gradient: Where M is the number of samples, is the sample, ε i is a Gaussian white noise signal sample; Sample generation network g θ The parameters θ of (·) are updated by stochastic gradient ascent with the gradient:

6. The method for detecting rare abnormal signals in an ellipsoid family observation background environment according to claim 1, characterized in that: The steps of using a recurrent neural network to train the detection model are as follows: The feature matrix of pure noise observation {W1,W2,…,W k } and the observed feature matrix {X1,X2,…,X m+l } as two types of samples input into the recurrent neural network to train the detection function; Take the measurement matrix to be tested for anomalies and calculate the characteristic matrix of the abnormal measurement matrix; The feature matrix to be detected for abnormality is input into the detection network to obtain the detection result.

Images