An abnormal value-containing uncertain data target classification method and system

By using anomaly attribute detection and correction models to handle outliers in uncertain data, the problem of unknown anomaly locations is solved, thereby improving the accuracy of target recognition and the reliability of data.

CN116821824BActive Publication Date: 2025-12-30709TH RESEARCH INSTITUTE CHINA STATE SHIPBUILDING CORP LTD
View PDF 2 Cites 0 Cited by

Patent Information

Application Number
CN202310603432.0
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2023-05-25
Publication Date
2025-12-30
Estimated Expiration
2043-05-25

AI Technical Summary

Technical Problem

When the specific location of anomalies in uncertain data is unknown, existing technologies struggle to effectively process such data, resulting in low target recognition accuracy.

Method used

The abnormal attribute detection model detects the location of abnormal attributes, replaces the outliers with a mask matrix and a random noise matrix, generates corrected data by combining the abnormal attribute correction model, and inputs it into the target classifier for classification.

Benefits of technology

It improves the accuracy of target recognition, enhances the authenticity and reliability of data by detecting and correcting abnormal attribute values, and strengthens the classification ability of the classifier.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN116821824B_ABST
    Figure CN116821824B_ABST
Patent Text Reader

Abstract

The application provides an abnormal value-containing uncertain data target classification method and system, which comprises the following steps: obtaining abnormal value-containing uncertain data; inputting the uncertain data into an abnormal attribute detection model to obtain a first generation matrix, and obtaining the positions of abnormal attributes in the uncertain data according to the reconstruction errors between each generation value in the first generation matrix and the corresponding attribute value and a set threshold, and generating a mask matrix; replacing the abnormal values in the uncertain data by using the mask matrix and a random noise matrix to obtain replacement data, inputting the replacement data into an abnormal attribute correction model to obtain a second generation matrix; replacing the abnormal values in the uncertain data by using the mask matrix and the second generation matrix to obtain correction data, and inputting the correction data into a target classifier to obtain a target classification result corresponding to the uncertain data. The application realizes the detection and correction of specific abnormal attribute values in the uncertain data, and improves the accuracy of target recognition.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This invention belongs to the field of target recognition technology, and more specifically, relates to a method and system for classifying targets in uncertain data containing outliers. Background Technology

[0002] Target recognition technology plays a crucial role in military and civilian fields such as situational analysis and environmental monitoring. However, due to the influence of external conditions, the data transmitted by sensors may contain errors, and some attribute values ​​may be biased. If data containing abnormal attributes is not processed beforehand and is directly used for recognition, the target recognition performance will be severely affected, resulting in a very low accuracy rate.

[0003] Current incomplete data classification methods impute missing values ​​before classifying the data, assuming the exact location of each missing value is known. However, in many applications, the location of errors in the data is random, making it unknown beforehand which dimension will be incorrect. This prevents current incomplete data classification methods from directly handling anomalous data. Therefore, how to process uncertain data and further classify it to improve the accuracy of target recognition has become a crucial issue that urgently needs to be addressed in the field of target recognition. Summary of the Invention

[0004] To address the shortcomings of existing technologies, the present invention aims to provide a method and system for classifying targets in uncertain data containing outliers. This method addresses the problem of how to process uncertain data and perform further classification when the specific location of outliers in uncertain data is unknown, thereby improving the accuracy of target recognition.

[0005] To achieve the above objectives, in a first aspect, the present invention provides a method for classifying uncertain data targets containing outliers, comprising:

[0006] S101 Obtain uncertain data containing outliers; the uncertain data includes attribute values ​​for each attribute dimension;

[0007] S102 The uncertain data is input into the abnormal attribute detection model to obtain the first generation matrix corresponding to the uncertain data. The position of the abnormal attribute in the uncertain data is obtained according to the reconstruction error between each generated value and the corresponding attribute value in the first generation matrix and the set threshold. A mask matrix is ​​generated according to the position of the abnormal attribute in the uncertain data.

[0008] S103 uses the mask matrix and random noise matrix to replace outliers in uncertain data to obtain replacement data, and inputs the replacement data into the outlier attribute correction model to obtain the second generator matrix corresponding to the uncertain data.

[0009] S104 uses the mask matrix and the second generator matrix to replace outliers in the uncertain data to obtain corrected data, and inputs the corrected data into the target classifier to obtain the target classification result corresponding to the uncertain data.

[0010] In an optional example, the loss function value of the anomaly attribute detection model is determined based on the weighted mean square error between the input and generated values ​​of each attribute dimension in the unlabeled sample, and the KL divergence between the prior and posterior Gaussian distributions corresponding to the unlabeled sample.

[0011] In an optional example, the anomaly attribute correction model is trained based on labeled samples, a joint discriminator, and a classifier;

[0012] The loss function value of the abnormal attribute correction model is determined based on the KL divergence loss, reconstruction error loss, discriminator loss, L2 regularization term, and cross-entropy loss of the classifier corresponding to the labeled samples.

[0013] In an optional example, the anomaly detection model consists of a first variational autoencoder and a first bidirectional long short-term memory network; the anomaly correction model consists of a second variational autoencoder and a second bidirectional long short-term memory network.

[0014] Secondly, the present invention provides a classification system for uncertain data targets containing outliers, comprising:

[0015] The data acquisition module is used to acquire uncertain data containing outliers; the uncertain data includes attribute values ​​of each attribute dimension;

[0016] An abnormal attribute detection module is used to input the uncertain data into an abnormal attribute detection model, obtain a first generation matrix corresponding to the uncertain data, obtain the position of the abnormal attribute in the uncertain data based on the reconstruction error between each generated value and the corresponding attribute value in the first generation matrix and a set threshold, and generate a mask matrix based on the position of the abnormal attribute in the uncertain data.

[0017] An anomaly attribute correction module is used to replace outliers in uncertain data using the mask matrix and random noise matrix to obtain replacement data, and input the replacement data into the anomaly attribute correction model to obtain the second generator matrix corresponding to the uncertain data.

[0018] The target classification module is used to replace outliers in uncertain data using the mask matrix and the second generator matrix to obtain corrected data, and then inputs the corrected data into the target classifier to obtain the target classification result corresponding to the uncertain data.

[0019] In an optional example, the loss function value of the anomaly attribute detection model applied in the anomaly attribute detection module is determined based on the weighted mean square error between the input and generated values ​​of each attribute dimension in the unlabeled sample, and the KL divergence between the prior Gaussian distribution and the posterior Gaussian distribution corresponding to the unlabeled sample.

[0020] In an optional example, the anomaly attribute correction model applied in the anomaly attribute correction module is trained based on labeled samples, a joint discriminator, and a classifier.

[0021] The loss function value of the abnormal attribute correction model is determined based on the KL divergence loss, reconstruction error loss, discriminator loss, L2 regularization term, and cross-entropy loss of the classifier corresponding to the labeled samples.

[0022] In an optional example, the abnormal attribute detection model used in the abnormal attribute detection module consists of a first variational autoencoder and a first bidirectional long short-term memory network; the abnormal attribute correction model used in the abnormal attribute correction module consists of a second variational autoencoder and a second bidirectional long short-term memory network.

[0023] In summary, the technical solutions conceived by this invention have the following beneficial effects compared with the prior art:

[0024] This invention provides a method and system for classifying targets in uncertain data containing outliers. By first using an anomaly attribute detection model to detect anomaly attributes in uncertain data, and then using an anomaly attribute correction model to correct the anomaly attributes, the method achieves the detection and correction of specific anomaly attribute values, thereby improving the authenticity and reliability of the data. On this basis, a target classifier is then used to classify the corrected data, which effectively improves the classifier's ability to classify anomaly data and increases the accuracy of target recognition. Attached Figure Description

[0025] Figure 1 This is one of the flowcharts of the uncertain data target classification method provided by the present invention;

[0026] Figure 2 This is a schematic diagram of the structure of the abnormal attribute detection model provided by the present invention;

[0027] Figure 3 This is a training diagram of the abnormal attribute correction model provided by the present invention;

[0028] Figure 4 This is the second flowchart of the uncertain data target classification method provided by the present invention;

[0029] Figure 5 This is the network framework of the uncertain data anomaly detection and correction method provided by the present invention;

[0030] Figure 6 This is an architecture diagram of the uncertain data target classification system provided by the present invention. Detailed Implementation

[0031] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.

[0032] Most common anomaly detection methods can detect the entire data entry containing an anomaly, but they neglect the detection of specific anomalous attribute values, meaning they cannot determine the location of the specific anomalous attribute. Therefore, it is necessary to perform anomaly attribute detection and correction on the anomaly data before further classification processing to improve the accuracy of target recognition.

[0033] To address this issue, the present invention provides a target classification method for uncertain data containing outliers, thereby solving the problem of target classification errors caused by abnormal errors in target feature data during target identification. Figure 1 This is one of the flowcharts illustrating the uncertain data target classification method provided by the present invention, such as... Figure 1 As shown, the method includes:

[0034] Step S101: Obtain uncertain data containing outliers; uncertain data includes attribute values ​​for each attribute dimension;

[0035] Step S102: Input uncertain data into the abnormal attribute detection model to obtain the first generation matrix corresponding to the uncertain data, and obtain the position of the abnormal attribute in the uncertain data based on the reconstruction error between each generated value and the corresponding attribute value in the first generation matrix and the set threshold, and generate a mask matrix based on the position of the abnormal attribute in the uncertain data.

[0036] Step S103: Use the mask matrix and random noise matrix to replace outliers in the uncertain data to obtain replacement data, and input the replacement data into the outlier attribute correction model to obtain the second generator matrix corresponding to the uncertain data.

[0037] Step S104: Use the mask matrix and the second generator matrix to replace outliers in the uncertain data to obtain corrected data, and input the corrected data into the target classifier to obtain the target classification result corresponding to the uncertain data.

[0038] Here, the mask matrix can be a binary matrix generated based on the location of anomalous attributes in the uncertain data. Using the mask matrix and the second generated matrix from the anomalous attribute correction model, outliers in the original uncertain data are replaced, allowing the final data to retain the original, anomalous data information, further improving the authenticity and reliability of the final data. The model structure of the target classifier can be set according to the specific classification task. For example, for a classification task targeting time-series data, a Bidirectional Long Short Term Memory Network (BiLSTM) can be used.

[0039] In addition, prior to the above steps, an anomaly detection model can be pre-trained using an unlabeled dataset, and an anomaly correction model can be pre-trained using a labeled dataset. Here, both the anomaly detection model and the anomaly correction model can be generative models.

[0040] It should be noted that training the anomaly detection model using an unlabeled dataset allows the model to effectively learn the feature information contained in the dataset. When anomaly attribute values ​​in the input data contain errors, outlier detection is achieved by comparing the reconstruction error between the generated and input values ​​with a set threshold. Training the anomaly correction model using a labeled dataset allows this model to generate new data information that is as close as possible to the original data distribution. Based on this, the trained anomaly correction model can be used to correct uncertain data containing outliers.

[0041] The method provided in this invention first uses an anomaly attribute detection model to detect anomaly attributes in uncertain data information, and then uses an anomaly attribute correction model to correct the anomaly attributes. This achieves the detection and correction of specific anomaly attribute values, improving the authenticity and reliability of the data. On this basis, a target classifier is then used to classify the corrected data, effectively improving the classifier's ability to classify anomaly data and increasing the accuracy of target recognition.

[0042] Based on the above embodiments, the loss function value of the abnormal attribute detection model is determined based on the weighted mean square error between the input and generated values ​​of each attribute dimension in the unlabeled sample, and the KL divergence between the prior Gaussian distribution and the posterior Gaussian distribution corresponding to the unlabeled sample.

[0043] Furthermore, in training the abnormal attribute detection model, the training set uses normal unlabeled samples, and the overall loss function consists of the attribute loss function and the KL divergence loss, denoted as:

[0044] Ldet =L dim +L KL

[0045] Among them, the attribute loss function L dim The weighted mean square error between the input and generated values ​​of each attribute dimension in the unlabeled samples is determined by adding constraint information to the original mean square error loss term to reduce the influence of inter-dimensional coupling, and is expressed as:

[0046]

[0047] Where, x i The original input value. ω represents the generated value of the model. i b represents the constraint weights for each attribute dimension in the sample. i ω represents the offset of each attribute dimension in the data. i and b i All are obtained through network training, where n is the total number of attribute dimensions.

[0048] L KL The KL divergence between the prior network Gaussian distribution and the generator network Gaussian distribution in the anomaly attribute detection model is expressed as:

[0049]

[0050] Where, p θ (z) is the Gaussian distribution corresponding to the prior network of the latent variable z, reflecting the distribution characteristics of the original data. It is the Gaussian distribution corresponding to the posterior network of the latent variable z, reflecting the distribution characteristics of the reconstructed data, D KL This represents the KL divergence.

[0051] Based on any of the above embodiments, the abnormal attribute correction model is obtained by training a joint discriminator and classifier based on labeled samples.

[0052] The loss function value of the anomaly attribute correction model is determined based on the KL divergence loss, reconstruction error loss, discriminator loss, L2 regularization term, and cross-entropy loss of the classifier corresponding to the labeled samples.

[0053] It should be noted that, in order to further improve the accuracy of the model-generated data and enhance the model's stability, a discriminator was added to the abnormal attribute correction model, and the overall generation performance of the model was improved through adversarial training. In addition, to further constrain the data generation process, a BiLSTM classifier was added, and the network module was trained using the cross-entropy loss function. The L2 regularization term can enhance the model's generalization ability and adjust the model's resistance to perturbations.

[0054] Furthermore, in training the abnormal attribute correction model, the training set uses normal labeled samples. The overall loss function consists of KL divergence loss, reconstruction error loss, discriminator loss, L2 regularization term, and classifier cross-entropy loss. The overall network loss function expression is as follows:

[0055]

[0056] The KL divergence loss is similar to the KL divergence loss in the detection model, specifically:

[0057]

[0058] The reconstruction error loss, or reconstruction probability error, is:

[0059]

[0060] The loss function of the discriminator is:

[0061]

[0062] The cross-entropy loss function of the classifier is:

[0063]

[0064] Where E represents the expected distribution, p θ (X|Z) is the probability that the original input sample is reconstructed, X i For the original input value, G(Z) i () represents the generated value of the model, D represents the discriminator, and Y represents the true label value of the sample. λ is the generated label value of the classifier, λ is the penalty factor used to control the degree of weight decay of the regularization term, N is the total number of samples, and γ is the trainable parameter.

[0065] Based on any of the above embodiments, in order to further improve the accuracy of the data generated by the abnormal attribute detection model and thus improve the accuracy of abnormal attribute detection, the abnormal attribute detection model in this embodiment of the invention consists of a first variational autoencoder (VAE) and a first bidirectional long short-term memory network; in order to further improve the accuracy of the data generated by the abnormal attribute correction model and thus improve the accuracy of abnormal attribute correction, the abnormal attribute correction model in this embodiment of the invention consists of a second variational autoencoder and a second bidirectional long short-term memory network.

[0066] This invention discloses an anomaly attribute value correction classification method based on variational autoencoder and bidirectional long short-term memory network. The proposed anomaly attribute value correction classification method can effectively detect abnormal data and effectively improve the classifier's ability to classify abnormal data.

[0067] Based on any of the above embodiments, the method provided by the present invention includes the following steps:

[0068] Step 1: Collect target data. Obtain normal labeled data as the training set and data containing outliers as the test set.

[0069] Step 2 is to build an abnormal attribute value detection network module, namely the abnormal attribute detection model in step S102. Figure 2 This is a schematic diagram of the structure of the abnormal attribute detection model provided by the present invention, as shown below. Figure 2 As shown, this module consists of a variational autoencoder and a bidirectional long short-term memory network. By adding constraints between various attribute dimensions during network training, it can detect abnormal attribute information.

[0070] Step 3 is to build the outlier generation correction network module, which is the outlier attribute correction model in step S103. Figure 3 This is a training diagram of the abnormal attribute correction model provided by the present invention, as shown below. Figure 3 As shown, this module consists of another set of variational autoencoders and bidirectional long short-term memory networks, namely the second variational autoencoder and the second bidirectional long short-term memory network. Furthermore, to further improve the accuracy of the model-generated data and enhance its stability, a discriminator is added to the VAE-BiLSTM network, using adversarial training to improve the overall generation performance of the model.

[0071] Step 4: Train the network using normal data.

[0072] 4.1 In training the abnormal attribute value detection network module, the loss function includes KL divergence error loss and attribute error loss, and the gradient descent method is used to optimize the network. Specifically, the abnormal attribute value detection network module is trained using normal data to obtain the distribution characteristics of the original data and the reconstructed data. The KL divergence error loss is calculated based on the relationship between these two distribution characteristics. The mean squared error of the reconstructed data and the original data is calculated, and constraint information is added to the mean squared error loss term to reduce the influence of inter-dimensional coupling, thus obtaining the attribute error loss.

[0073] 4.2 In the training of the outlier generation correction network module, the loss function includes reconstruction error loss, KL divergence error loss, discriminator error loss, classifier error loss, and a regularized L2 weight decay term. The module is trained using labeled normal data. For the BiLSTM classifier, a cross-entropy loss function is used to train the network module to further constrain the data generation process. Utilizing the idea of ​​Generative Adversarial Interpolation (GAIN), a discriminator is added to the module network, which, along with the generator, trains the module through mutual constraints. The L2 regularization term enhances the model's generalization ability and adjusts its robustness against perturbations.

[0074] Step 5 yields a mask matrix for uncertain data containing outliers. Figure 4 This is the second flowchart of the uncertain data target classification method provided by the present invention. Figure 5 This is the network framework of the uncertain data anomaly detection and correction method provided by the present invention, such as... Figure 4 and Figure 5 As shown, the network inputs uncertain data X containing outliers, and the generated values ​​of the uncertain data are obtained through the outlier attribute value detection network module. The mean squared error between the input value and the mean squared error is calculated, and then compared with a set threshold to obtain the mask matrix M corresponding to the input data. Here, each row of X represents a data point in the input, and each column represents an attribute dimension.

[0075] Step 6 corrects outliers in the uncertain data containing outliers. The obtained mask matrix M, uncertain data X, and random noise matrix H are combined (M*X + (1-M)*H) to obtain new data information, i.e., replacement data X′, which is then input into the outlier generation and correction module. The new data obtained from this module... By combining the mask matrix with the original data, that is... Obtain the corrected uncertain data

[0076] Step 7 outputs the final classification result. The corrected uncertain data is input into the classifier, i.e., the target classifier in step S104, to obtain the final classification result, thus completing the classification of uncertain data containing outliers.

[0077] This invention achieves higher-precision classification of uncertain input data samples containing anomalous attribute values ​​by constructing an anomaly attribute value detection network module and an anomaly value generation and correction network module. First, the model is trained using normal sample data. When experimenting with uncertain data samples, the anomaly attribute value detection network module obtains the mask matrix corresponding to the uncertain input data. Then, a missing value imputation method is used to obtain new data information that approximates the original data distribution. Finally, the mask matrix is ​​used to replace the anomalous attribute values.

[0078] Based on any of the above embodiments, the method provided by the present invention includes the following steps:

[0079] S1. Acquire high-dimensional data information. Utilize existing sensors to acquire a large amount of high-dimensional data information that requires outlier detection and correction.

[0080] S2. Data Standardization. For the acquired high-dimensional data, the different dimensions of the various feature attributes result in significant differences in attribute values. Directly inputting this data with different dimensions into the network for calculation would cause substantial bias. To eliminate the influence of dimensions, data standardization is necessary before the experiment. The standardization formula is as follows:

[0081]

[0082] Where μ is the mean of the sample data and σ is the standard deviation of the sample data. For each dimension of the data sample, the mean and standard deviation of the data in that dimension are calculated, and then the formula is used to standardize each feature value of each data point.

[0083] S3. Train the network using normal data. Train the outlier detection network module using an unlabeled dataset, enabling it to effectively learn the features contained in the dataset. When an outlier value appears in the input data, it detects the outlier by comparing the reconstruction error between the generated and input values ​​with a set threshold. Simultaneously, train the outlier generative correction network module using a labeled dataset. This module generates new data information that closely approximates the original data distribution, and the trained correction network module is used to correct the test data containing outliers.

[0084] S4. Input test data to obtain corrected data. Input the unlabeled dataset into the trained network to obtain the mask matrix corresponding to the dataset containing outliers and the generated data information. Replace the generated data with the original data using the mask matrix and the original data, so that the final data retains the original data information without outliers, further improving the authenticity and reliability of the final data. In addition, to ensure the reliability of the experiment and reduce the impact of random errors, each experiment is repeated multiple times, and the final experimental results are obtained by calculating the average value. When selecting the threshold in the outlier attribute detection network, a certain threshold size is usually selected for each dimension of attribute to detect attribute anomalies in each dimension.

[0085] S5. The corrected data is then re-input into the classifier for classification and identification. The classifier is used to obtain the final classification result of the original data containing outliers.

[0086] Verification and implementation:

[0087] For the confusion matrix, one of the evaluation metrics for anomaly detection models, P is typically used to represent data predicted as an anomaly, N to represent data predicted as normal, T to represent data predicted as true, and F to represent data predicted as false. The confusion matrix is ​​shown in Table 1.

[0088] Table 1. Outlier Detection Confusion Matrix

[0089] Predicting positive examples Predict negative examples True positive examples TP FN True negatives FP TN

[0090] To verify the accuracy of anomaly detection, three commonly used anomaly detection evaluation metrics—precision (P), recall (R), and weighted harmonic mean F1—were used to analyze the experimental results.

[0091] Accuracy:

[0092] Recall rate:

[0093] F1:

[0094] The aforementioned evaluation metrics are commonly used when performing anomaly detection on the entire dataset. However, these metrics cannot reflect the detection capability of the proposed method for outlier attribute values. To further accurately evaluate model performance, the following anomaly attribute value detection evaluation metrics are also proposed for the anomaly attribute value detection network proposed in this invention. (The last sentence appears to be incomplete and possibly refers to a different dataset.) k In a dataset containing data, for the i-th data item, the actual number of anomalous attribute dimensions it contains is represented by d. iThe number of abnormal attribute dimensions detected by the network is represented as d′. i Then the attribute dimension detection accuracy A d The calculation formula is defined as follows:

[0095]

[0096] Based on any of the above embodiments, the present invention provides a target classification system for uncertain data containing outliers. Figure 6 This is an architecture diagram of the uncertain data target classification system provided by the present invention, such as... Figure 6 As shown, the system includes:

[0097] The data acquisition module 610 is used to acquire uncertain data containing outliers; the uncertain data includes attribute values ​​of each attribute dimension.

[0098] An abnormal attribute detection module 620 is used to input uncertain data into an abnormal attribute detection model, obtain a first generation matrix corresponding to the uncertain data, obtain the position of the abnormal attribute in the uncertain data based on the reconstruction error between each generated value and the corresponding attribute value in the first generation matrix and a set threshold, and generate a mask matrix based on the position of the abnormal attribute in the uncertain data.

[0099] The abnormal attribute correction module 630 is used to replace outliers in uncertain data using a mask matrix and a random noise matrix to obtain replacement data, and inputs the replacement data into the abnormal attribute correction model to obtain the second generator matrix corresponding to the uncertain data.

[0100] The target classification module 640 is used to replace outliers in uncertain data using a mask matrix and a second generator matrix to obtain corrected data, and then inputs the corrected data into the target classifier to obtain the target classification result corresponding to the uncertain data.

[0101] The system provided in this embodiment of the invention first uses an anomaly attribute detection model to detect anomaly attributes in uncertain data information, and then uses an anomaly attribute correction model to correct the anomaly attributes. This achieves the detection and correction of specific anomaly attribute values, improving the authenticity and reliability of the data. On this basis, a target classifier is used to classify the corrected data, effectively improving the classifier's ability to classify anomaly data and increasing the accuracy of target recognition.

[0102] It is understood that the detailed functional implementation of each of the above modules can be found in the description of the aforementioned method embodiments, and will not be repeated here.

[0103] In addition, embodiments of the present invention provide another classification device for uncertain data targets containing outliers, which includes: a memory and a processor;

[0104] The memory is used to store computer programs;

[0105] The processor is configured to implement the methods described in the above embodiments when executing the computer program.

[0106] In addition, the present invention provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the methods described in the above embodiments.

[0107] Based on the methods in the above embodiments, this embodiment of the invention provides a computer program product that, when run on a processor, causes the processor to execute the methods in the above embodiments.

[0108] Those skilled in the art will readily understand that the above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. An abnormal value-containing uncertainty data-oriented target classification method, characterized in that, The method comprises the following steps: S101: acquiring uncertain data containing abnormal values; the uncertain data comprises attribute values of each attribute dimension; S102: inputting the uncertain data into an abnormal attribute detection model to obtain a first generation matrix corresponding to the uncertain data, and obtaining the positions of abnormal attributes in the uncertain data according to reconstruction errors between each generation value in the first generation matrix and the corresponding attribute value and a set threshold, and generating a mask matrix according to the positions of the abnormal attributes in the uncertain data; S103: replacing abnormal values in the uncertain data by using the mask matrix and a random noise matrix to obtain replacement data, and inputting the replacement data into an abnormal attribute correction model to obtain a second generation matrix corresponding to the uncertain data; S104: replacing abnormal values in the uncertain data by using the mask matrix and the second generation matrix to obtain corrected data, and inputting the corrected data into a target classifier to obtain a target classification result corresponding to the uncertain data; wherein the uncertain data containing abnormal values is acquired by a sensor; the abnormal attribute correction model is trained based on labeled samples, a discriminator and a classifier; a loss function value of the abnormal attribute correction model is determined based on a KL divergence loss, a reconstruction error loss, a discriminator loss, an L2 regularization term and a cross-entropy loss of the classifier corresponding to the labeled samples; the abnormal attribute detection model is composed of a first variational autoencoder and a first bidirectional long short-term memory network; and the abnormal attribute correction model is composed of a second variational autoencoder and a second bidirectional long short-term memory network.

2. The method of claim 1, wherein, The loss function value of the abnormal attribute detection model is determined based on a weighted mean square error between input values of each attribute dimension in the unlabeled sample and generation values, and a KL divergence between a priori Gaussian distribution and posteriori Gaussian distribution corresponding to the unlabeled sample.

3. An outlier-containing uncertain data target classification system, characterized in that, The method comprises the following steps: A data acquisition module is configured to acquire uncertain data containing abnormal values; the uncertain data comprises attribute values of each attribute dimension; An abnormal attribute detection module is configured to input the uncertain data into an abnormal attribute detection model to obtain a first generation matrix corresponding to the uncertain data, and obtain the positions of abnormal attributes in the uncertain data according to reconstruction errors between each generation value in the first generation matrix and the corresponding attribute value and a set threshold, and generate a mask matrix according to the positions of the abnormal attributes in the uncertain data; An abnormal attribute correction module is configured to replace abnormal values in the uncertain data by using the mask matrix and a random noise matrix to obtain replacement data, and input the replacement data into an abnormal attribute correction model to obtain a second generation matrix corresponding to the uncertain data; A target classification module is configured to replace abnormal values in the uncertain data by using the mask matrix and the second generation matrix to obtain corrected data, and input the corrected data into a target classifier to obtain a target classification result corresponding to the uncertain data; wherein the uncertain data containing abnormal values is acquired by a sensor; the abnormal attribute correction model applied in the abnormal attribute correction module is trained based on labeled samples, a discriminator and a classifier; The loss function value of the abnormal attribute correction model is determined based on a KL divergence loss corresponding to the labeled sample, a reconstruction error loss, a discriminator loss, an L2 regularization term, and a cross-entropy loss of the classifier. The abnormal attribute detection model applied in the abnormal attribute detection module is composed of a first variational autoencoder and a first bidirectional long short-term memory network; and the abnormal attribute correction model applied in the abnormal attribute correction module is composed of a second variational autoencoder and a second bidirectional long short-term memory network.

4. The system of claim 3, wherein, The loss function value of the abnormal attribute detection model applied in the abnormal attribute detection module is determined based on a weighted mean square error between input values and generated values of each attribute dimension in the unlabeled sample, and a KL divergence between a prior Gaussian distribution and a posterior Gaussian distribution corresponding to the unlabeled sample.

Citation Information

Patent Citations

  • Dirt process data restoration method and system, electronic equipment and storage medium

    CN115270961A

  • Anomaly detection method for large-scale multivariate time series data in cloud environment

    WO2022160902A1