Latent space optimized autoencoder anomaly detection method and system

Abstract

The application discloses a hidden space optimization self-encoder anomaly detection method and system, and is used for satellite telemetry data anomaly detection. The method comprises the following steps: inputting collected telemetry data into a pre-established and trained anomaly detection model to identify whether the telemetry data is abnormal; wherein the anomaly detection model comprises a hidden space optimization self-encoder and an anomaly discriminator; the training of the anomaly detection model is performed through the acquired low-dimensional hidden space features for adversarial interpolation, and then a more compact feature distribution is obtained by using a constructed regularization evaluation network, so that the anomaly discriminator is easy to realize anomaly detection. The detection method can acquire rich and effective feature information, and effectively improves the anomaly detection result. Research results show that the method has good performance for different data sets, can be practically applied to satellite anomaly detection, and improves the safety of satellite on-orbit operation.

Description

Technical Field

[0001] This invention belongs to the field of satellite anomaly detection technology, and particularly relates to an anomaly detection method and system of autoencoder with latent space optimization. Background Technology

[0002] During satellite operation, on-orbit malfunctions frequently occur due to factors such as the extremely harsh space environment, component aging, and system design problems. These malfunctions can even threaten the safety of the spacecraft and the success of the satellite's mission. Satellite telemetry data reflects the operational status of equipment and is a crucial basis for ground control personnel to assess the satellite's operational status and performance. Anomaly detection in telemetry parameters is the most effective method for identifying abnormalities or unexpected events in orbit and is becoming a hot research topic in the aerospace field.

[0003] Anomalies are generally also referred to as outliers, abnormalities, or deviations. A common definition of anomalies is: "In a dataset, anomalies are those data points that deviate from the majority of the data, and whose patterns differ from the majority of the data." Anomaly detection involves separating anomalous data (behaviors) from normal data (behaviors) in a dataset. In the spacecraft field, according to the U.S. National Research Council Convention, spacecraft anomalies are defined as "unexpected or unplanned events occurring on a spacecraft." Anomalies exhibited by satellites in orbit are diverse. Anomalies in satellite telemetry data can generally be categorized into point anomalies, contextual anomalies, clustering anomalies, and correlation anomalies. This application primarily focuses on correlation anomalies. Furthermore, during satellite anomaly detection, some anomaly types may be known, but over time, due to changes in environmental factors and command sequences, new anomaly types are continuously discovered. Therefore, the scenario we handle is semi-supervised, where only normal pattern sample data is available during the training phase. In a semi-supervised environment, the model is designed to construct normal profiles using a large amount of normal sample sequence data. Behaviors that deviate significantly from normal patterns are detected as anomalies. Numerous scholars have published literature and patents related to semi-supervised anomaly detection methods.

[0004] Over the past decade, while anomaly detection has achieved considerable success in many areas, the field of anomaly detection for complex high-dimensional data continues to attract significant research interest. In high-dimensional data spaces, due to the "curse of dimensionality," as the dimensionality increases, the differences in distance between anomalous and normal samples, as well as the differences in distance between normal samples themselves, become less pronounced. Furthermore, as the data dimensionality increases, density estimation of the original data space becomes difficult because noise and irrelevant features have a greater negative impact on density estimation. Therefore, many researchers have developed two-stage anomaly detection algorithms: (a) reducing the data dimensionality to obtain high-level feature representations, and (b) constructing anomaly scores based on the learned features. For example, some researchers have proposed a two-stage BotCensor network detection system and anomaly detection based on Impression Extractor (IE-Net) and expert networks. In addition, some researchers have employed spectral anomaly detection and alternative dimensionality reduction techniques to find low-dimensional representations of the original high-dimensional data, hoping to distinguish between anomalous and normal instances. While it has been shown that using two-stage anomaly detection algorithms to handle high-dimensional data is feasible, unfortunately, there are still some limitations to simply estimating the density of the low-dimensional space to effectively identify anomalies in high-dimensional data.

[0005] In recent years, with the development of deep learning, a series of deep learning-based anomaly detection methods have been proposed, with autoencoders and adversarial neural networks (ANNs) and their variants attracting particular attention. As one of the most popular unsupervised learning algorithms, AEs are also favored by scholars in the field of anomaly detection, such as deep autoencoders, variational autoencoders (VAEs), and stacked autoencoders (SAEs). The core idea of ​​these methods is to encode the input data into a low-dimensional representation, and then decode the low-dimensional representation back to the original data space by minimizing the reconstruction error. Generative Adversarial Networks (GANs) have the greatest advantage of a strong ability to fit the distribution of training data. Applying them to the field of anomaly detection can effectively and accurately identify anomalies, with typical examples including DCGAN, BiGAN, and GANomaly. These methods aim to capture the features of training samples to the greatest extent possible with the generator, hoping to make the generated samples so realistic that they can deceive the discriminator network. The discriminator's purpose is to compare the two, distinguish the authenticity of the input data as much as possible, and improve itself by continuously learning to reduce the deviation between positive and negative samples. In addition to these, many researchers have proposed other deep learning anomaly detection models. Ruff et al.'s DSVDD trains a classifier model by minimizing the radius of a hypersphere, enclosing all training samples in the representation space. Although this method addresses the curse of dimensionality by projecting all training samples into the representation space, it may overfit the training data, especially when the training set is small or contaminated with anomalous samples. Schlegl et al. proposed AnoGAN, a deep adversarial network model based on unsupervised learning. GANs only learn the distribution of positive examples to identify anomalous images by reconstructing the residual between the image and the feature distribution of the input image. While AnoGAN exhibits good anomaly detection performance, in practical applications, it requires continuous iterative optimization when dealing with high-dimensional data, inevitably consuming a lot of time and demonstrating low computational efficiency. DAGMM combines deep autoencoders and Gaussian mixture models (GMMs) for anomaly detection. However, real-world data may not only have high dimensionality but also lack a clearly defined distribution. Modeling the density distribution of the input data in GMMs requires manual parameter adjustment, which severely impacts detection performance. Furthermore, while autoencoder models can separate outliers from normal points in low-dimensional representations, the distribution of normal data may be arbitrary rather than a prior distribution (such as a GMM). On the other hand, some outliers may exhibit dense clustered distributions, and in addition, there are always some normal points discretely distributed near normally dense clusters in space. These factors also pose significant challenges to anomaly detection in large-scale high-dimensional data. Summary of the Invention

[0006] Due to the high dimensionality of satellite telemetry data and the complexity of the correlation between parameters, simply using the feature distribution extracted by the autoencoder without proper constraints will not be conducive to maintaining the density distribution of the original high-dimensional data and the extraction of effective information. This will lead to the overlap of normal and abnormal samples in the feature space, making it difficult to distinguish them, and thus the anomaly detection effect will not be ideal.

[0007] The purpose of this invention is to overcome the shortcomings of the prior art and to propose an autoencoder anomaly detection method and system with latent space optimization, so as to obtain rich and effective feature information in the feature space.

[0008] To achieve the above objectives, this invention proposes a latent space-optimized autoencoder anomaly detection method for satellite telemetry data anomaly detection, the method comprising:

[0009] The collected telemetry data is input into a pre-established and trained anomaly detection model to identify whether the telemetry data is abnormal; among which,

[0010] The anomaly detection model includes a latent space-optimized autoencoder and an anomaly discriminator. The training of the anomaly detection model involves adversarial interpolation of the acquired low-dimensional latent space features, followed by the use of a constructed regularized evaluation network to obtain a more compact feature distribution, thereby making the anomaly discriminator easier to implement anomaly detection.

[0011] As an improvement to the above method, the latent space-optimized autoencoder includes an encoder and a decoder; the processing includes:

[0012] The input vector x passes through an encoder z = g parameterized by θ. θ (x) yields the latent variable z, which is obtained through... Parameterized Generate a vector that is approximately reconstructed from the input vector x.

[0013] As an improvement to the above method, the processing of the anomaly discriminator includes: taking the reconstruction error obtained using the autoencoder and the reconstructed features obtained by relative Euclidean distance and cosine similarity as input, and taking the anomaly score constructed by using a probability density-aware strategy as output, wherein an anomaly score of 0 indicates that the data is normal, and an anomaly score of 1 indicates that the data is abnormal.

[0014] As an improvement to the above method, the method further includes a training step for the anomaly detection model; specifically including:

[0015] Build an algorithmic framework for an anomaly detection model;

[0016] Establish training and testing sets;

[0017] The training set data is sequentially input into the algorithm framework of the anomaly detection model for training until the training requirements are met, resulting in a well-trained anomaly detection model.

[0018] The test set is then sequentially input into the trained anomaly detection model for verification testing.

[0019] As an improvement to the above method, the algorithmic framework for building the anomaly detection model includes:

[0020] An autoencoder with the same structure as the anomaly detection model is added, and an evaluation network is connected after the added autoencoder. A latent feature space constraint component is added between the added autoencoder and the anomaly detection model's autoencoder; where,

[0021] The latent feature space constraint component is used to perform a convex combination of the latent space interpolation of the added autoencoder and the autoencoder of the anomaly detection model through the interpolation constraint coefficient α and input it into the evaluation network.

[0022] The evaluation network is used to perform regularized evaluation and optimization of the latent space interpolation effect, thereby predicting the interpolation constraint coefficient α.

[0023] As an improvement to the above method, the evaluation network s ξ The loss of the confrontation ξ The following optimizations have been made:

[0024]

[0025] Where, x α To increase the output of the autoencoder, μ is a hyperparameter, x1 is the input of the added autoencoder, which is one training data, x2 is the input of the autoencoder of the anomaly detection model, which is another training data, and α∈[0,0.5].

[0026] As an improvement to the above method, the establishment of the training set and test set specifically includes:

[0027] The dataset Micius was constructed using three publicly available benchmark datasets, Thyroid, Satellite, and Landsat, as well as real telemetry parameters from a quantum science experimental satellite. A portion of the data was used to build the training set, and another portion was used to build the test set.

[0028] On the other hand, this invention proposes a latent space-optimized autoencoder anomaly detection system for satellite telemetry data anomaly detection, characterized in that the system comprises:

[0029] The acquisition and detection module is used to input the acquired telemetry data into a pre-established and trained anomaly detection model to identify whether the telemetry data is abnormal; among which,

[0030] The anomaly detection model includes a latent space-optimized autoencoder and an anomaly discriminator. The training of the anomaly detection model involves adversarial interpolation of the acquired low-dimensional latent space features, followed by the use of a constructed regularized evaluation network to obtain a more compact feature distribution, thereby making the anomaly discriminator easier to implement anomaly detection.

[0031] Compared with the prior art, the advantages of the present invention are:

[0032] 1. The latent space optimized autoencoder anomaly detection method of the present invention performs adversarial interpolation on the acquired low-dimensional latent space features and obtains a more compact feature distribution during the regularization process of the evaluation network, making it easier to distinguish between normal and abnormal samples in the feature space.

[0033] 2. This invention employs a probability density-aware strategy (KDE) to construct anomaly scores. During training, it learns the probability density distribution of high-dimensional data, which can effectively detect anomalous objects in the test.

[0034] 3. This invention utilizes real satellite telemetry parameter data and three publicly available datasets—Thyroid, Satellite, and Landsat—to study the anomaly detection problem in satellite telemetry data. It addresses the challenge of obtaining the complete density distribution of the original high-dimensional data due to the high dimensionality and complex correlations between parameters in current anomaly detection methods. The detection method of this invention acquires rich and effective feature information, significantly improving anomaly detection results. Research results show that the experimental model performs well in anomaly detection across the four datasets in terms of AUC and F1 score, demonstrating its practical applicability to satellite anomaly detection and enhancing the safety of satellite operation in orbit. Attached Figure Description

[0035] Figure 1 This invention provides a latent space optimized autoencoder anomaly detection algorithm framework.

[0036] Figure 2 This is the main structure of the evaluation network of this invention;

[0037] Figure 3(a) shows the variation curves of the AUC values ​​of the method of the present invention and the benchmark algorithm due to the contamination rate of the training dataset;

[0038] Figure 3(b) shows the variation curves of the F1 value of the method of the present invention and the benchmark algorithm due to the contamination rate of the training dataset;

[0039] Figure 4(a) shows the latent space distribution of the test data using the method of the present invention, Figure 4(b) shows the latent space distribution of the test data using AEGAN-KED, Figure 4(c) shows the latent space distribution of the test data using DSVDD, and Figure 4(d) shows the latent space distribution of the test data using DAGMM. Detailed Implementation

[0040] For the reasons mentioned above, this paper proposes a latent space interpolation autoencoder (LSIA-AD) model for satellite anomaly detection, which addresses the challenges of anomaly detection in high-dimensional datasets. The model consists of two parts: one part comprises an autoencoder and an anomaly discriminator. The autoencoder performs convex combination interpolation on the low-dimensional space of the sample data, simultaneously acquiring the data's feature representation. The anomaly discriminator then selects a kernel function to appropriately simulate the data's probability density distribution. The other part is an evaluation network with a regularization module, which evaluates and optimizes the latent space interpolation effect, thereby predicting the interpolation constraint coefficients. Unlike standard autoencoders, the latent space interpolation-constrained autoencoder not only considers reconstruction errors but also optimizes the data distribution in the latent data space. Therefore, the density distribution and key features of high-dimensional data are preserved in the low-dimensional representation. Data with high density values ​​are more likely to be normal objects, while those with low density values ​​are considered anomalous objects, thus constructing a more distinctive profile for normal data.

[0041] The technical solution of the present invention will be described in detail below with reference to the accompanying drawings and embodiments.

[0042] Example 1

[0043] Embodiment 1 of this invention proposes an autoencoder anomaly detection method based on latent space optimization. It uses a framework of adversarial interpolation in the latent feature space to learn the original attributes and features of data samples, and is validated on three public datasets and one real-world task dataset. The specific technical solution is as follows: First, the overall network structure is introduced. Inspired by related work, the overall network structure consists of four parts: encoder-decoder, latent feature space constraint component, evaluation network, and anomaly discriminator. Then, the various parts of the loss function are described, and finally, a description of the specific implementation method is given.

[0044] (1) Problem Definition

[0045] Anomaly detection is formalized into a semi-supervised scenario, where the training dataset contains only normal data samples. The goal of anomaly detection is to train and build a detector on the training set to determine whether new samples are anomaly. We use... Denotes a training dataset containing only normal samples, where xi ∈R d The goal of anomaly detection methods is to create a model from random sampling that can identify whether a sample is normal or abnormal. The test set contains normal and abnormal samples, defined as follows: Where y i =0 indicates normal, y i =0 indicates an anomaly.

[0046] (2) Introduction to self-encoders

[0047] An autoencoder is an unsupervised learning neural network that learns the inherent patterns and features of data by reconstructing input data. Its input and output dimensions are the same. The weights are adjusted using the backpropagation algorithm, and the network is trained by minimizing the reconstruction error between the input and output data, making them as close as possible to each other, thus learning the internal representation of samples with the same data distribution characteristics. A classic autoencoder consists of two parts: an encoder and a decoder. Generally, the input vector... Through an encoder z = g with θ parameterization θ (x) is used to obtain the latent variable z, and then the latent variable z is obtained through a method by Parameterized decoder To generate a vector that is approximately reconstructed from the input vector x. Where g θ and It is implemented using a multi-layer neural network. The encoder and decoder are trained simultaneously to minimize the difference between the input vector x and the output vector x, typically choosing l2 = ||x||. 2 As a measure of distance difference.

[0048] Encoding process:

[0049] z = g θ (W1x+b1),

[0050] Decoding process:

[0051]

[0052] (3) Construction of anomaly detection algorithm framework

[0053] In the latent space, feature differences between similar samples are removed as redundant information, and only their core features are retained, which is beneficial for feature extraction and full utilization of the data. However, due to the complexity of high-dimensional data, simply acquiring feature information from the latent space without constraints will result in inaccurate capture of feature information and data distribution among data points. Therefore, this paper proposes to generate new samples through latent space interpolation, thereby optimizing the latent space to make its data distribution more compact and dense, enabling the extraction of richer and more effective feature information from the latent space. Figure 1As shown, the interpolation process involves obtaining two sets of latent variables in the latent space through the encoder, and then decoding a mixture of these two sets of latent variables through the decoder. Typically, the mixed latent variables are formed by convex combination, therefore interpolation is equivalent to calculating... Where α∈[0,1], z1=g θ (x1) and z2 = g θ (x2). As the parameter α changes from 0 to 1, a series of interpolated data points will be generated, providing a basis for subsequent training of the classifier.

[0054] like Figure 1 As shown, a portion of the LSIA-AD model consists of an autoencoder and an anomaly discriminator (i.e., Figure 1 The kernel density estimation model consists of a training data set. Transformation from the sample space to the latent space z = g θ (x), then the decoder reconstructs the image from the latent space, and obtains the reconstructed features as input to the classifier for anomaly detection; another part is an evaluation module. Its function is to evaluate the quality of interpolation, thereby predicting the interpolation constraint function α∈[0,1]. z1 and z2 have encoder g. θ (·)get.

[0055] As mentioned above, a well-performing latent variable interpolation should possess two characteristics: first, the interpolated intermediate points are indistinguishable from the real data, appearing more realistic; second, the intermediate points provide a smooth semantic transition between endpoints, preserving data information. Therefore, we propose a regularizer to constrain interpolation, encouraging the interpolated data points to appear realistic, or more specifically, indistinguishable from the reconstruction of real data points.

[0056] To improve the interpolation results, this paper introduces an evaluation network for constraint, which functions somewhat like an adversarial neural network. The main structure of the evaluation network is as follows: Figure 2 As shown, it is similar to the architecture of an encoder. Figure 2 Chinese: x α The input to the evaluation network is represented by , Reshape represents data dimensionality upscaling, Linear represents the linear layer in the neural network, loss represents the loss function, and α represents the interpolation coefficients predicted by the evaluation network.

[0057] The input to the evaluation network is the reconstructed data x after interpolation of the existing data points. α The purpose is to predict convex combination parameters through interpolation points. The data points x obtained after interpolation... αWhen used as input to the evaluation network, since the parameters α and 1-α have some ambiguity during prediction, and the data z1 and z2 have symmetry, the range of α is set to [0, 0.5]. In this case, the autoencoder is essentially trained to deceive the evaluation network, making it believe that the value of parameter α is 0. This is achieved by adding an extra term to the loss of the autoencoder to optimize its parameters and thus fool the evaluation network. Using s ξ Let the evaluation network be represented, then the adversarial loss of the evaluation network is optimized as follows:

[0058]

[0059] in, μ is a hyperparameter. In the above equation, the first term is used to train the evaluation network to predict from it, and the second term acts as a regularizer, which has two functions: firstly, it forces the evaluation network to always output 0 for non-interpolated data input; secondly, it uses x and μ in the data space to determine the hyperparameter. Interpolation is performed between these terms to ensure the commenter is exposed to real data, even if the autoencoder's reconstruction is poor. We found the second term is not critical to our method but helps stabilize the adversarial learning process. The autoencoder's loss function is modified by adding a regularization term:

[0060]

[0061] Here, ζ is a scalar hyperparameter that controls the weight of the regularization term. Note that the regularization term effectively attempts to make the evaluation network's output zero regardless of the value of α, thus "tricking" the evaluation network into believing that the input with interpolated data is uninterpolated (i.e., α = 0). Utilizing the idea of ​​the GAN framework, the parameters θ and... Relative to l g,h (This allows the autoencoder to establish a connection with the evaluation network), ξ is about the loss function l ξ Optimization.

[0062] (4) Construction of outlier scores

[0063] In addition to adversarial interpolation optimization of the latent space and low-dimensional representation of the input data, we also consider the reconstruction error of this method to facilitate subsequent anomaly detection. Therefore, the data feature information is extracted as follows:

[0064] η = [l2, c],

[0065] c=[rec_euclidean,rec_cosine],

[0066] Here, η represents the reconstruction error feature, which can be multidimensional, taking into account multiple distance metrics such as relative Euclidean distance, absolute Euclidean distance, and cosine similarity. The latent space optimization method proposed in this paper uses relative Euclidean distance and cosine similarity as reconstruction features c. Finally, we use the extracted data feature information as input to an anomaly discriminator, which obtains the density distribution of the training data to determine the data anomaly score.

[0067] In statistics, random sampling is the simplest method for estimating probability density functions, while kernel density functions represent another form of random sampling. The basic steps involve generating a uniform random sample. In random sampling, each sample point has a weight. The kernel function describes the distribution of these weights. Generally, most weights are distributed within a region near the sample point. Summing the kernel density estimates of these sample points yields the kernel density function for the entire sample set. The general definition of kernel density estimation is as follows.

[0068] According to the anomaly discriminator, for the feed source from the encoder, X1, X2, X3, ... X n The population density distribution function of the data at any point x is defined as:

[0069]

[0070] Where K is the kernel function (usually non-negative), and h is the bandwidth. A Gaussian kernel function is typically chosen. The kernel density estimation function can outline the data distribution, providing a clearer visual representation of the data. It can also be extended to multivariate kernel density functions.

[0071] (5) Data Source

[0072] Table 1. Statistical information of the dataset

[0073]

[0074] This experiment uses the Micius dataset, which is constructed from three publicly available benchmark datasets: Thyroid, Satellite, and Landsat, as well as real telemetry parameters from a quantum science experimental satellite. 50% of the normal data samples are randomly selected as the training set, and the remaining 50% of the samples, along with all abnormal samples, are used as the test set to complete the training of the model.

[0075] Micius: This dataset comes from telemetry data of a Chinese scientific experimental satellite and contains 19 attributes related to satellite operation modes, spanning from January 2017 to February 2019. The dataset includes four operation modes; mode 4 is considered a rare mode and is classified as an anomaly, while the other modes are classified as normal.

[0076] Landsat: This dataset comes from the UCI Machine Learning Repository and belongs to the US Landsat satellite system for exploring Earth's resources and environment. Its main tasks are to investigate various surface resources, assess and forecast severe natural disasters, and create thematic maps. The dataset contains a total of 42 feature attributes, divided into normal and abnormal categories. After removing meaningless categorical attributes, 36 attributes were ultimately selected.

[0077] Satellite: This dataset is derived from LandsatSatellite, a multi-class dataset in the UCI Machine Learning Library. The training and test data from the LandsatSatellite dataset are combined, with the three smallest classes (2, 4, and 5) merged to form the outlier class, and all other classes merged to form a normal class.

[0078] Thyroid: This dataset comes from the ODDS data repository and belongs to the medical thyroid lesion dataset. The original dataset contains a total of 3 attribute features. In this task, the superfunctional class is a minority class, therefore it is considered an abnormal class, and the other two classes are considered normal classes.

[0079] (6) Verification Results

[0080] (a) Introduction to the benchmark algorithm

[0081] OC-SVM: One-class Support Vector Machine (OC-SVM) is a common kernel-based anomaly detection method. It uses kernel functions to map data to a feature space, thereby constructing a hyperplane decision boundary for anomaly detection. In our experiments, we used the widely adopted Radial Basis Function (RBF) kernel in all tasks.

[0082] IFOREST: Isolation Forest is a tree-based anomaly detection algorithm that constructs a decision tree based on the results of separating each data point in the dataset to effectively discover anomalous samples.

[0083] LOF: Local Outlier Factor is a density-based anomaly detection algorithm that determines whether a sample is abnormal by calculating the density correlation between each sample and its surrounding samples.

[0084] Deep SVDD: Deep Support Vector Data Description utilizes neural network training to minimize the hypersphere that divides the feature space of the computational samples, and then determines whether a sample point is an anomaly based on the distance between the center of the sphere and the test sample point.

[0085] DAGMM (Deep Autoencoding Gaussian Mixture Model) is one of the most influential anomaly detection algorithms, consisting of a compression network and an estimation network. It utilizes a deep autoencoder to generate a low-dimensional representation and reconstruction error for each input data point, which are then used as input to the Gaussian mixture model for anomaly detection.

[0086] GAOD: GAOD is a classification-based anomaly detection method that does not require domain knowledge of the data and achieves anomaly detection in general data by training a classifier.

[0087] (b) Experimental evaluation indicators

[0088] Anomaly detection is essentially a classification task. Generally, positive examples are considered anomalous samples, and negative examples are considered normal samples. Since anomalies are the target of detection, and the number of anomalous samples must be less than the number of normal samples, anomalies are treated as positive examples. This paper uses F1 score and AUC as intuitive methods to compare anomaly detection performance.

[0089]

[0090]

[0091] in, TP represents the number of abnormal samples detected; FN represents the number of abnormal samples detected as normal; FP represents the number of normal samples detected as abnormal; TN represents the number of normal samples detected as normal. i∈positiveclass Only add up the indices of the positive samples, then rank. i The index of the i-th positive sample is represented (abnormal scores are sorted in ascending order); G and R represent the number of positive and negative samples, respectively.

[0092] (c) Algorithm effectiveness

[0093] In the first set of experiments, we used completely clean training data and used the mean and standard deviation of the evaluation metrics from 20 experiments as anomaly detection performance measures. Based on the proportion of abnormal samples shown in Table 1, we determined the anomaly score threshold to identify anomalies. Table 2 shows the performance of LSIA-AD and its comparison algorithm on four benchmark datasets, with the best results highlighted in bold.

[0094] Table 2. Average AUC and F1 score (%) of LSIA-AD and benchmark algorithms

[0095]

[0096] As shown in the table above, the proposed method LSIA-AD outperforms baseline methods in terms of AUC and F1 score across all datasets. On the Micius dataset, LSIA-AD improves the F1 score by 5.8% and 11.6% compared to the best baseline algorithms LOF and OC-SVM, respectively, and the AUC by 5.6% and 13.1%, respectively. For DAGMM, while it works quite well on the other three datasets, considering both latent representation and reconstruction errors in energy modeling, it exhibits some bias in density estimation across different datasets, resulting in poor algorithm stability. Our proposed method achieves an F1 score of 78.4% and an AUC of 99.2% on the classic Thyroid dataset, also surpassing the baseline algorithms. For the Satellite and Landsat datasets, LSIA-AD outperforms the suboptimal method LOF by 4.8% and 3.3% in F1 score, respectively. Regarding standard deviation, although our proposed method is not the most stable on every dataset, its overall stability is better. LSIA-AD also outperforms the benchmark algorithm on the classic dataset Thyroid. Although the stability of our proposed method is not optimal on all datasets, it performs well overall. OC-SVM performs poorly in anomaly detection in high-dimensional data. This is because the core idea of ​​OC-SVM is to use normal data to find boundaries in high-dimensional space, but when there are too many attributes in the data, irrelevant redundant attributes may have a significant impact on the results of OC-SVM. The reason why LSIA-AD is better than DAGMM may be: (1) because the data distribution is more compact due to interpolation optimization in the latent space, it is better than autoencoders in learning low-dimensional representations that maintain the original data distribution; (2) it uses kernel density estimation to simulate the probability density distribution of the data instead of Gaussian mixture models. KDE is better than Gaussian mixture models because GMM is a parametric estimation, which refers to the process of estimating the parameters of the selected distribution using sample data, while KDE is a nonparametric estimation that allows obtaining a functional form that fits the data without any theoretical guidance or constraints. For DSVDD, the data is placed on a hypersphere with a predefined center and a minimum radius. Test samples falling inside the hypersphere are classified as normal, while those outside are classified as anomalous. Although the hard boundaries of SVDD are strong, they can lead to overfitting of the model to the training data—a problem that has been addressed with soft-bounded SVDD, but it still overfits due to a lack of sufficient generalization constraints, as all training samples are projected onto a single point in the representation space, resulting in poor anomaly detection.

[0097] Another potential reason why our method outperforms all baselines is that we employ a latent space optimization strategy, which only requires interpolation of the feature space of the input data. This strategy helps our method to make the number distribution in the latent space more compact, effectively separating densely distributed anomalies in the latent data space. Table 2 also shows that LSIA-AD significantly outperforms its variant (AEGAN-KDE) on four datasets, indicating that optimized interpolation of the latent space better preserves key information and learns the data distribution in the latent data space, thus improving anomaly detection performance.

[0098] To further quantitatively verify the superiority of the proposed method, we used the Wilcoxon rank-sum test to statistically evaluate the proposed method against all benchmark methods on four datasets. More specifically, we need to test the proposed method against each benchmark method one by one, evaluating the following null hypothesis H0 and alternative hypothesis H1 for each pair of methods:

[0099] H0: X≈Y, H1: X>Y,

[0100] Where X represents the result of LSIA-AD on a specific dataset, and Y represents the result of a specific benchmark method on the corresponding dataset. We calculate the p-value for each test and test the hypothesis at the significance level of α = 0.001. The statistical evaluation results of AUC and F1 are shown in the table below.

[0101] Table 3. Wilcoxon rank-sum test p-values ​​for AUC values

[0102]

[0103] Table 4. Wilcoxon rank-sum test p-values ​​for F1 values

[0104]

[0105] As shown in Tables 3 and 4, at a significance level of α = 0.01, except for the DAGMM dataset where the p-value of the AUC is greater than α, the Wilcoxon rank-sum test results for the AUC and F1 values ​​of our proposed method are significant on all other datasets. Therefore, we can reject the null hypothesis H0 and accept the alternative hypothesis H1. In other words, the performance of our proposed method is significantly better than all baselines. In summary, this experiment confirms that the improvement of our proposed method in anomaly detection is statistically significant.

[0106] (d) Algorithm robustness

[0107] In practical applications, training data may be mixed with noise, or abnormal samples may be incorrectly labeled as normal samples during the construction of the training dataset. Therefore, to verify the algorithm's robustness, we used the Micius dataset with different levels of contamination to validate the algorithm. In this set of experiments, we investigated how LSIA-AD responds to contaminated training data. In each run, we randomly sampled and retained 50% of the normal data for testing. The remaining 50% of the normal data was mixed with c% (c = [0,1,2,3,4,5]) of samples drawn from the abnormal data and then used for model training to test the algorithm's robustness. The experimental results are shown in Figure 3.

[0108] Figures 3(a) and 3(b) show the average AUC and F1 scores of LSIA-AD and all benchmark algorithms after running 20 times on the Micius dataset with added contamination data. Figure 3(a) shows the AUC values ​​of the proposed method and benchmark algorithms as a function of the contamination rate of the training dataset; Figure 3(b) shows the F1 scores of the proposed method and benchmark algorithms as a function of the contamination rate of the training dataset. When the contamination ratio c increases from 1% to 5%, the AUC and F1 scores of all methods fluctuate, but different algorithms show varying sensitivities to contamination data. We note that our method maintains good detection accuracy with 5% contamination data and outperforms the baseline on all contamination data, demonstrating good robustness to contamination on the Micius dataset. For the three selected deep learning benchmark methods, although their AUC and F1 scores decrease with increasing contamination data ratio, the data fluctuations are not significant. In contrast, for traditional methods, their AUC and F1 scores fluctuate more significantly and are less stable with increasing contamination data ratio. In particular, with OC-SVM, using the same parameter settings as in our experiments with clean training data, we observed that OC-SVM was more sensitive to contamination. To achieve better detection accuracy, it is crucial to train the model with high-quality data (i.e., clean data or data with the lowest possible contamination rate).

[0109] (e) Visualization of latent space sample distribution

[0110] In the experiment, to demonstrate the effect of adversarial interpolation on latent space optimization, we selected three methods—AEGAN-KED, DSVDD, and DAGMM—to compare the distribution of the latent space of the test samples in the Thyroid dataset. The results are visualized using t-SNE as shown in Figure 4. Figure 4(a) shows the latent space distribution of the test data using the method of this invention, Figure 4(b) shows the latent space distribution of the test data using AEGAN-KED, Figure 4(c) shows the latent space distribution of the test data using DSVDD, and Figure 4(d) shows the latent space distribution of the test data using DAGMM. First, as can be seen from Figure 4(a), in the learned latent space, although LSIA-AD has a small overlap between normal and abnormal samples, it can generally better separate abnormal and normal samples. In contrast, AEGAN-KED, DSVDD, and DAGMM show more overlap between abnormal and normal samples in the learned low-dimensional space. Secondly, compared to AEGAN-KDE, DSVDD, and DAGMM, our proposed method appears to achieve a more compact representation of normal samples in the latent space, with fewer outliers appearing in the normal clusters. This indicates that our proposed method performs better in latent space optimization. Simultaneously, we also observed that for deep autoencoders with the same structure (Fig. 4(b) AEGAN-KDE), adversarial interpolation optimizes the latent space better, resulting in a more compact data distribution. In other words, adversarial interpolation in the latent space extracts sufficient core feature information during training, and the latent space-optimized autoencoder and evaluation network mutually improve each other's performance. The regularization introduced by the evaluation network helps the deep autoencoder escape less attractive local optima to achieve better compression, while the autoencoder provides the evaluation network with more meaningful low-dimensional representations for robust density estimation.

[0111] In summary, the anomaly detection method optimized by latent space achieves state-of-the-art accuracy on public benchmark datasets, providing a promising alternative for semi-supervised anomaly detection.

[0112] Example 2

[0113] Embodiment 2 of the present invention proposes a latent space optimized autoencoder anomaly detection system for satellite telemetry data anomaly detection, implemented based on the method of Embodiment 1. The system includes:

[0114] The acquisition and detection module is used to input the acquired telemetry data into a pre-established and trained anomaly detection model to identify whether the telemetry data is abnormal; among which,

[0115] The anomaly detection model includes a latent space-optimized autoencoder and an anomaly discriminator. The training of the anomaly detection model involves adversarial interpolation of the acquired low-dimensional latent space features, followed by the use of a constructed regularized evaluation network to obtain a more compact feature distribution, thereby making the anomaly discriminator easier to implement anomaly detection.

[0116] Overview:

[0117] In this application, we propose a latent space optimized autoencoder semi-supervised model (LSIA-AD) for anomaly detection in high-dimensional satellite telemetry data. The LSIA-AD method consists of two main parts: a deep learning network and an anomaly discriminator. The deep learning network projects samples into a low-dimensional space under latent space interpolation constraints, preserving key information for anomaly detection. The anomaly discriminator feeds low-dimensional representations and reconstructed error features as feeds and learns the probability density distribution of training samples to identify anomalies in the samples to be tested. In experimental studies, LSIA-AD demonstrates superior performance to existing techniques on public benchmark datasets. These experiments ensure the accuracy of the model and provide a reference for the future expansion and application of the proposed anomaly detection framework.

[0118] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to the embodiments, those skilled in the art should understand that modifications or equivalent substitutions to the technical solutions of the present invention do not depart from the spirit and scope of the technical solutions of the present invention, and all such modifications or substitutions should be covered within the scope of the claims of the present invention.

Claims

1. A latent space-optimized autoencoder anomaly detection method for satellite telemetry data anomaly detection, the method comprising: The collected telemetry data is input into a pre-established and trained anomaly detection model to identify whether the telemetry data is abnormal; among which, The anomaly detection model includes a latent space optimized autoencoder and an anomaly discriminator. The training of the anomaly detection model involves adversarial interpolation of the acquired low-dimensional latent space features, followed by the use of a constructed regularized evaluation network to obtain a more compact feature distribution, thereby making the anomaly discriminator easier to implement anomaly detection. The processing of the anomaly discriminator includes: taking the reconstruction error obtained using the autoencoder and the reconstructed features obtained by relative Euclidean distance and cosine similarity as input, and taking the anomaly score constructed by using a probability density-aware strategy as output, where an anomaly score of 0 indicates that the data is normal, and an anomaly score of 1 indicates that the data is abnormal.

2. The latent space optimized autoencoder anomaly detection method according to claim 1, characterized in that, The latent space-optimized autoencoder includes an encoder and a decoder; the processing includes: Input vector By Parameterized encoder The latent variable z is obtained by... Parameterized decoder , generate with input vector Approximate reconstructed vector .

3. The latent space optimized autoencoder anomaly detection method according to claim 2, characterized in that, The method also includes a training step for the anomaly detection model; specifically including: Build an algorithmic framework for an anomaly detection model; Establish training and testing sets; The training set data is sequentially input into the algorithm framework of the anomaly detection model for training until the training requirements are met, resulting in a well-trained anomaly detection model. The test set is then sequentially input into the trained anomaly detection model for verification testing.

4. The latent space optimized autoencoder anomaly detection method according to claim 3, characterized in that, The algorithmic framework for building the anomaly detection model includes: An autoencoder with the same structure as the anomaly detection model is added, and an evaluation network is connected after the added autoencoder. A latent feature space constraint component is added between the added autoencoder and the anomaly detection model's autoencoder; where, The latent feature space constraint component is used to constrain the interpolation coefficients. The latent space interpolation of the added autoencoder and the anomaly detection model is convexly combined and input into the evaluation network. The evaluation network is used to perform regularized evaluation and optimization of the latent space interpolation effect, thereby predicting the interpolation constraint coefficients. .

5. The latent space optimized autoencoder anomaly detection method according to claim 4, characterized in that, The evaluation network The losses of the confrontation The following optimizations have been made: ， in, To increase the output of the autoencoder, , For hyperparameters, The input to the autoencoder is given as training data. This serves as the input to the autoencoder of the anomaly detection model, and is another set of training data. .

6. The latent space optimized autoencoder anomaly detection method according to claim 4, characterized in that, The establishment of the training set and test set specifically includes: The dataset Micius was constructed using three publicly available benchmark datasets, Thyroid, Satellite, and Landsat, as well as real telemetry parameters from a quantum science experimental satellite. A portion of the data was used to build the training set, and another portion was used to build the test set.

7. A latent space-optimized autoencoder anomaly detection system for detecting anomalies in satellite telemetry data, characterized in that, The system includes: The acquisition and detection module is used to input the acquired telemetry data into a pre-established and trained anomaly detection model to identify whether the telemetry data is abnormal; among which, The anomaly detection model includes a latent space optimized autoencoder and an anomaly discriminator. The training of the anomaly detection model involves adversarial interpolation of the acquired low-dimensional latent space features, followed by the use of a constructed regularized evaluation network to obtain a more compact feature distribution, thereby making the anomaly discriminator easier to implement anomaly detection. The processing of the anomaly discriminator includes: taking the reconstruction error obtained using the autoencoder and the reconstructed features obtained by relative Euclidean distance and cosine similarity as input, and taking the anomaly score constructed by using a probability density-aware strategy as output, where an anomaly score of 0 indicates that the data is normal, and an anomaly score of 1 indicates that the data is abnormal.

Images