An open set semi-supervised image out-of-distribution anomaly detection method and system

Abstract

The application discloses an open set semi-supervised image distribution out-of-distribution anomaly detection method and system. The method comprises the following steps: performing weak enhancement on collected labeled samples, simultaneously performing weak enhancement and strong enhancement on collected unlabeled samples, extracting high-dimensional features from the enhanced samples, and performing classification prediction on the high-dimensional features to obtain a semi-supervised classification loss and an anomaly score regression loss obtained by anomaly detection; fusing the semi-supervised classification loss, the anomaly score regression loss, a multi-level negative refining loss and a clustering bidirectional refining loss obtained by calculation to obtain a loss function; training an anomaly detection model by using the loss function and the collected samples; and detecting to-be-detected samples by using the trained anomaly detection model. The application can simultaneously improve the in-distribution classification performance and the out-of-distribution anomaly detection capability.

Description

Technical Field

[0001] This invention relates to the fields of computer vision and artificial intelligence, and in particular to a method and system for detecting out-of-distribution anomalies in open-set semi-supervised images. Background Technology

[0002] With the widespread application of computer vision technology in high-risk fields such as autonomous driving, medical diagnosis, and industrial defect detection, the safety and reliability of the system have become a major concern. In practical deployments, models often face out-of-distribution (OOD) anomalies that have not appeared in the training set. However, existing deep learning models generally suffer from overconfidence, misidentifying unknown anomalies as known categories (such as misclassifying road debris as vehicles), which poses a serious safety hazard in open scenarios. Furthermore, in industrial deployments, acquiring large-scale, high-quality labeled data is extremely expensive and time-consuming, while the massive amounts of unlabeled data collected cheaply often contain a large number of unknown anomalies. This open-set semi-supervised learning (OSSL) scenario not only requires the model to learn classification using a small number of labeled samples, but also requires the model to effectively distinguish between in-distribution samples and out-of-distribution anomalies from mixed unlabeled data.

[0003] To address the aforementioned challenges, existing open-set semi-supervised out-of-distribution anomaly detection methods suffer from two main bottlenecks: First, current methods lack sufficient semantic representation of features, focusing excessively on the logical output of the classifier while neglecting semantic constraints at the feature level. This results in a lack of discriminative power when faced with semantically similar but strongly interfering anomalies. Second, existing research often employs a passive filtering strategy of detection and filtering, treating selected out-of-distribution samples as interference noise and discarding them directly. This approach ignores the potential value of these samples for actively optimizing the decision boundaries between in-distribution and out-of-distribution samples, preventing the effective widening of the gap between in-distribution and out-of-distribution samples in the feature space.

[0004] Therefore, there is an urgent need for a new technical solution that utilizes anomalous samples to actively separate the inner and outer boundaries of the distribution, thereby effectively reshaping the feature space structure and simultaneously improving the classification performance within the distribution and the anomaly detection capability outside the distribution. Summary of the Invention

[0005] The purpose of this invention is to provide a method and system for detecting out-of-distribution anomalies in open-set semi-supervised images, thereby overcoming the aforementioned limitations of the prior art. The various technical effects of the preferred solutions among the many technical solutions provided by this invention are detailed below.

[0006] To achieve the above objectives, the present invention provides the following technical solution: This invention provides a method for detecting out-of-distribution anomalies in open-set semi-supervised images, comprising: Weak enhancement is performed on the collected labeled samples, and both weak and strong enhancement are performed on the collected unlabeled samples. High-dimensional features are extracted from the enhanced samples, and classification prediction is performed on the high-dimensional features to obtain semi-supervised classification loss and anomaly detection to obtain anomaly score regression loss. Based on the statistical distribution of the set of anomaly detection scores of labeled samples and anomaly detection scores of unlabeled samples, positive thresholds and negative thresholds are generated, and positive and negative filter masks are generated based on the positive and negative thresholds. The similarity between each sample feature and multiple cluster centers of the collected samples is iteratively optimized to obtain a soft-assignment weight matrix. The semantic relationship weights between the collected samples are constructed based on the soft-assignment weight matrix, and the cluster-aware relationship scaling factor between the collected samples is calculated based on the semantic relationship weights. The collected samples are filtered according to the negative filtering mask, and the prediction entropy is maximized for the filtered abnormal samples to obtain multi-level negative refinement loss. The classification semantic consistency matrix between unlabeled samples is calculated based on the classification prediction values ​​of the weakly enhanced unlabeled samples. The forward refinement weighting matrix between unlabeled samples is calculated based on the forward filtering mask and the classification semantic consistency matrix. The clustering bidirectional refinement loss is calculated based on the forward refinement weighting matrix, the scaling factor corresponding to the unlabeled sample, and the sample feature similarity between unlabeled samples. The loss function is obtained by fusing the semi-supervised classification loss, anomaly score regression loss, multi-level negative refinement loss, and cluster bidirectional refinement loss. The anomaly detection model is trained using the loss function and the collected samples. The trained anomaly detection model is then used to detect the samples to be tested.

[0007] In some embodiments, the weak enhancement includes random horizontal flipping and random cropping, and the strong enhancement adds a random enhancement strategy on top of the weak enhancement.

[0008] In some embodiments, the semi-supervised classification loss obtained by classifying and predicting high-dimensional features includes: Calculate the supervised loss between the true label of the labeled sample and its classification prediction value; obtain the pseudo label after prediction of the weakly enhanced unlabeled sample, and calculate the consistency loss between the pseudo label and the classification prediction value of the corresponding strongly enhanced unlabeled sample; fuse the supervised loss and the consistency loss to obtain the semi-supervised classification loss.

[0009] In some embodiments, the semi-supervised classification loss is obtained by fusing the supervised loss and the consistency loss, including: The consistency loss between the pseudo-label with a prediction confidence of not less than the threshold and the classification prediction value of its strongly enhanced unlabeled sample is calculated, and the consistency loss between the pseudo-label with a prediction confidence of less than the threshold and the classification prediction value of its strongly enhanced unlabeled sample is recorded as zero. The average value of the supervision loss is fused with the average value of the consistency loss to obtain the semi-supervised classification loss.

[0010] In some embodiments, the abnormality score regression loss obtained from the high-dimensional feature anomaly detection includes: An anomaly score soft target is calculated based on the transfer scores of weakly enhanced unlabeled samples and multiple cluster centers, and the classification confidence of weakly enhanced unlabeled samples. The anomaly score regression loss is then calculated based on the anomaly score soft target, the anomaly detection scores of weakly enhanced labeled samples, and the anomaly detection scores of strongly enhanced unlabeled samples.

[0011] In some embodiments, the classification confidence of the weakly enhanced unlabeled sample is obtained based on the prediction confidence and classification entropy.

[0012] In some embodiments, the scaling factor of the selected abnormal samples is forced to be set to the maximum rejection constant.

[0013] In some embodiments, the plurality of cluster centers are multiple cluster centers obtained by iteratively updating the sample feature vectors of the labeled samples and the unlabeled samples that are forward filtered by a forward filtering mask, using momentum update coefficients.

[0014] In some embodiments, the step of using a trained anomaly detection model to detect the sample to be tested includes: Input the sample to be tested into the trained anomaly detection model to obtain the in-distribution class probability and anomaly score; The outlier score of the sample to be tested is compared with the outlier score threshold. When the score is higher than the outlier score threshold, it is identified as an in-distribution sample; when the score is not higher than the outlier score threshold, it is identified as an out-of-distribution outlier sample. For the selected samples within the distribution, the category with the highest probability within the distribution is considered as its predicted category.

[0015] According to another aspect of the present invention, an open-set semi-supervised image distribution out-of-distribution anomaly detection system is also provided. The open-set semi-supervised image distribution out-of-distribution anomaly detection system is trained using the open-set semi-supervised image distribution out-of-distribution anomaly detection method described above, and then the system is used to detect test samples. The open-set semi-supervised image distribution out-of-distribution anomaly detection system includes a feature extractor, a classification prediction head, an anomaly score head, and a soft clustering module all connected to the feature extractor. The anomaly score head is connected to the soft clustering module. The feature extractor is used to extract high-dimensional features of the samples, the classification prediction head is used to output the probability within the K-class distribution based on the high-dimensional features, the anomaly score head is used to output the confidence score that the sample belongs to the normal sample within the distribution based on the high-dimensional features, and the soft clustering module is used to statistically distribute and output the semantic relationship between samples based on the high-dimensional features and the anomaly scores output by the anomaly score head. The semi-supervised classification loss is obtained through the classification prediction head, the outlier score regression loss is obtained through the outlier score head, the multi-level negative refinement loss is obtained through the classification prediction head and the outlier score head, and the clustering bidirectional refinement loss is obtained through the soft clustering module.

[0016] Implementing one of the above-described technical solutions of the present invention has the following advantages or beneficial effects: This invention employs an adaptive bidirectional threshold screening mechanism to identify high-confidence samples within a distribution and high-confidence outlier samples outside a distribution from mixed unlabeled data. Through soft clustering semantic modeling, cluster-aware relationship scaling factors, and bidirectional refinement loss, it promotes the aggregation of similar samples within the distribution and the separation of samples with different semantic meanings in the feature space, while actively using outlier samples to widen the boundaries between the distribution and its inlier / outlier. This breaks away from the traditional pure classification perspective, proactively mining and fully utilizing potential open-set outlier sample information in unlabeled data.

[0017] During the model training phase, an adaptive bidirectional threshold filtering mechanism is used to accurately distinguish high-confidence in-distribution samples from unlabeled data from outlier samples. Then, combined with a soft clustering mechanism, positive semantic aggregation of in-distribution samples and negative semantic separation of outlier samples are collaboratively performed in the feature space. This method effectively reshapes the feature space structure, thereby simultaneously improving in-distribution classification performance and out-of-distribution anomaly detection capabilities. Attached Figure Description

[0018] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort. In the drawings: Figure 1This is a flowchart of an open-set semi-supervised image distribution anomaly detection method according to an embodiment of the present invention; Figure 2 This is a schematic diagram of the structure of an open-set semi-supervised image distribution anomaly detection system according to an embodiment of the present invention. Detailed Implementation

[0019] To make the objectives, technical solutions, and advantages of the present invention clearer, various exemplary embodiments described below will be referenced to the accompanying drawings, which form part of the exemplary embodiments, illustrating various exemplary embodiments that may be used to implement the present invention. Unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this disclosure. It should be understood that they are merely examples of processes, methods, and apparatuses consistent with some aspects of the present invention disclosed as detailed in the appended claims, and other embodiments may be used, or structural and functional modifications may be made to the embodiments listed herein without departing from the scope and spirit of the present invention.

[0020] In the description of this invention, it should be understood that the terms "center," "longitudinal," "lateral," etc., indicate the orientation or positional relationship based on the accompanying drawings, and are only for the convenience of describing the invention and simplifying the description, and do not indicate or imply that the referred element must have a specific orientation, or be constructed and operated in a specific orientation. The terms "first," "second," etc., are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. The term "multiple" means two or more. The terms "connected" and "linked" should be interpreted broadly, for example, they can be fixed connections, detachable connections, integral connections, mechanical connections, electrical connections, communication connections, direct connections, indirect connections through an intermediate medium, and can be the internal connection of two elements or the interaction relationship between two elements. The term "and / or" includes any and all combinations of one or more of the related listed items. Those skilled in the art can understand the specific meaning of the above terms in this invention according to the specific circumstances.

[0021] To illustrate the technical solution described in this invention, specific embodiments are described below, showing only the parts related to the embodiments of this invention.

[0022] Example 1: like Figure 1 As shown in this embodiment, a method for detecting out-of-distribution anomalies in open-set semi-supervised images includes: S100. Perform weak enhancement on the collected labeled samples, and perform both weak and strong enhancement on the collected unlabeled samples. Extract high-dimensional features from the enhanced samples, and perform classification prediction on the high-dimensional features to obtain semi-supervised classification loss and anomaly detection to obtain anomaly score regression loss.

[0023] In a specific embodiment, image data is collected for real-world scenarios, and the dataset contains a small number of labeled samples and a large number of unlabeled samples. For example, in the CIFAR-100 dataset, each of the 55 in-distribution classes contains 500 training image samples, 50 of which are labeled, and the remaining 450 are unlabeled. Let... For a labeled distribution of samples, This is an unlabeled sample set, which contains both in-distribution and out-of-distribution samples. The number of samples (or batch size) within the labeled distribution. >1 represents the ratio of unlabeled to labeled samples.

[0024] Furthermore, for the image data collected from the database, a small number of in-distribution category samples are labeled to obtain a labeled sample set. and unlabeled sample set Within each batch, according to : The image is sampled according to a certain ratio, and weak enhancement is applied to labeled samples. Weak enhancement is applied to unlabeled samples simultaneously. and enhance Among them, weak enhancement Includes random cropping and horizontal flipping, with strong enhancement. The RandAugment (random augmentation) strategy was added to the weak augmentation.

[0025] Understandably, data augmentation strategies can also include AutoAugment, TrivialAugment, Cutout, MixUp, CutMix, or dedicated augmentation strategies for industrial, medical, and remote sensing images; the combination of weak and strong augmentation can be adjusted according to the image source, number of categories, and noise intensity.

[0026] In some embodiments, a semi-supervised classification loss is obtained by classifying and predicting high-dimensional features, including: Calculate the supervised loss between the true label of a labeled sample and its classification prediction value; obtain the pseudo label after prediction of a weakly augmented unlabeled sample, and calculate the consistency loss between the pseudo label and the classification prediction value of the corresponding strongly augmented unlabeled sample; fuse the supervised loss and the consistency loss to obtain the semi-supervised classification loss.

[0027] Furthermore, by fusing the supervised loss and the consistency loss, a semi-supervised classification loss is obtained, which includes: Calculate the consistency loss between the pseudo-label with a prediction confidence of not less than the threshold and the classification prediction value of its strongly enhanced unlabeled sample; the consistency loss between the pseudo-label with a prediction confidence of less than the threshold and the classification prediction value of its strongly enhanced unlabeled sample is recorded as zero; the average value of the supervised loss and the average value of the consistency loss are fused to obtain the semi-supervised classification loss.

[0028] In a specific embodiment, before calculating the semi-supervised classification loss, a neural network model can be constructed that includes a feature extractor, a classification prediction head, and an anomaly score head. Feature Extractor Wide ResNet-28-2 is used to extract high-dimensional features; classification prediction head. Output the probability within the K-class distribution; outlier score header. The output sample represents the confidence score of a normal sample within the distribution; the overall framework can be found in [reference needed]. Figure 2 As shown in the figure. Simultaneously, K cluster centers are randomly initialized.

[0029] For labeled samples Based on the prediction results and real labels Calculate the standard cross-entropy supervised loss; for unlabeled samples , , It is a classification prediction head The class prediction probability is obtained by using weakly augmented samples. Get pseudo tags Then calculate the prediction of strongly enhanced samples. The consistency loss is calculated, where the consistency loss is obtained through cross-entropy. The semi-supervised classification loss is then calculated by combining the two classes of samples. : ; in, ( The cross-entropy between two probability distributions is represented by ), and B is the batch size. It is an indicator function, when The value is 1 if the prediction confidence is high, and 0 otherwise. This process ensures that the prediction confidence is only affected when the prediction confidence level is high. Exceeding the threshold Only when the threshold is exceeded is the calculation performed. The consistency loss between the corresponding pseudo-label and its strongly enhanced unlabeled sample classification prediction value is used, i.e., the pseudo-label is used for model training.

[0030] In some embodiments, the anomaly score regression loss obtained from high-dimensional feature anomaly detection includes: An anomaly score soft objective is calculated based on the transport scores of weakly enhanced unlabeled samples and multiple cluster centers, and the classification confidence of weakly enhanced unlabeled samples. Anomaly score regression loss is then calculated based on the anomaly score soft objective, the anomaly detection scores of weakly enhanced labeled samples, and the anomaly detection scores of strongly enhanced unlabeled samples. The classification confidence of weakly enhanced unlabeled samples is obtained from the prediction confidence and classification entropy.

[0031] In a specific embodiment, an anomaly score calculation and prediction module can be set up during model training. The anomaly score calculation is used to generate a target anomaly score value, representing the probability that an unlabeled sample belongs to the in-distribution sample, based on the association relationship between unlabeled samples and multiple cluster centers. This is based on unlabeled samples. With the k-th in-distribution semantic cluster center The degree of matching between them is determined by introducing partial optimal transmission constraints to calculate the unlabeled samples. Transfer scores to each cluster center : ; Where K is the total number of categories within the distribution. These are unlabeled samples (unsupervised samples). Transmitted to the kth cluster center The partial optimal transmission rate can be obtained by solving the partial optimal transmission problem.

[0032] Understandably, the anomaly score objective can be generated by weighting a partial optimal transfer score with classification confidence, or it can be further integrated with energy score, maximum class probability, feature distance, Mahalanobis distance, or uncertainty estimation results.

[0033] Next, the predicted value of the i-th unlabeled sample is used. Calculate the classification confidence score for this sample. : ; The predicted value for the i-th weakly enhanced unlabeled sample The corresponding prediction confidence level, for Information entropy is used to characterize the uncertainty of the model's classification result for the unlabeled sample. The larger the entropy value, the higher the classification uncertainty.

[0034] Then, by combining the two, the anomaly score soft target of the i-th unlabeled sample is obtained. : .

[0035] The Sigmoid function is a sigmoid smooth saturation function.

[0036] Finally, the outlier score prediction module, based on the aforementioned soft objective, uses outlier score regression loss. For abnormal score headers Regression training is performed so that the model gradually learns the ability to distinguish between in-distribution samples and out-of-distribution abnormal samples during the training phase: ; in, Indicates the abnormal score head pair of samples The output score is calculated. The first term of the formula forces the score of samples within the labeled distribution to approach 1, and the second term forces the score of unlabeled, strongly augmented samples to approximate the calculated soft target score. .

[0037] S200: Generate positive and negative thresholds based on the statistical distribution of the set of anomaly detection scores for labeled and unlabeled samples. Then, generate positive and negative filter masks based on the positive and negative thresholds. The anomaly detection scores are obtained through anomaly detection (the anomaly score header is directly output).

[0038] To avoid the problem of unstable sample selection caused by using a fixed threshold during open set semi-supervised training, this embodiment introduces an adaptive bidirectional threshold selection mechanism during the training phase.

[0039] In a specific embodiment, the set of anomaly detection scores for the current batch of samples is statistically analyzed. The distribution of the positive threshold is adaptively generated. and negative threshold : ; in, This represents the operation of taking the q-quantile of the set. Then, based on a positive threshold... and negative threshold Generate a high-confidence positively distributed sample mask, i.e., a positive selection mask. And a mask for outlier samples from a high-confidence negative distribution, i.e., a negative screening mask. .

[0040] when When determining unsupervised samples For samples within a high-confidence distribution, i.e., a forward mask. ;when When an outlier is identified as an outsider sample in the high-confidence distribution, it is identified as a negative mask. The specific formula is as follows: .

[0041] For ease of understanding, the above anomaly detection score calculation is used to generate an anomaly score target value that characterizes the probability that an unlabeled sample belongs to a sample within the distribution, based on the association between the unlabeled sample and multiple cluster centers.

[0042] It should be noted that the positive and negative thresholds can be batch quantiles, moving quantiles, or EMA smoothing quantiles.

[0043] S300. Iteratively optimize the similarity between each sample feature and multiple cluster centers of the collected samples to obtain a soft-assignment weight matrix. Construct semantic relationship weights between the collected samples based on the soft-assignment weight matrix, and calculate the cluster-aware relationship scaling factor between the collected samples based on the semantic relationship weights. Each sample feature of the collected samples can be extracted using a feature extractor.

[0044] In a specific embodiment, the cosine similarity between each feature of the sample composed of labeled and unlabeled samples and the cluster center is calculated, such as calculating the first... Individual sample features With the kth cluster center The cosine similarity is calculated, and the Sinkhorn-Knopp algorithm is used for iterative optimization to obtain the soft-assigned weight matrix. : .

[0045] Understandably, soft clustering normalization can employ the Sinkhorn-Knopp algorithm, or other differentiable normalization, optimal transport approximation, or temperature-based softmax allocation methods.

[0046] Furthermore, the multiple cluster centers are obtained by iteratively updating the sample feature vectors in the union of labeled samples and unlabeled samples that are forward filtered by the forward filtering mask, using momentum update coefficients.

[0047] In a specific embodiment, an exponential moving average strategy is used to update the multiple cluster centers. Correspondingly, the k-th cluster center is updated in the t-th iteration. The formula is: ; in, This represents the k-th cluster center after the t-th iteration; For momentum update coefficients; The set of samples participating in the update consists of labeled samples and samples that satisfy... The union of unlabeled samples; The soft weights assigned to sample i by center k; Let i be the feature vector of sample i.

[0048] Understandably, cluster centers can be updated online using EMA, or statistically updated using mini-batch means, prototype memory, or sliding window.

[0049] Based on soft-assigned weights Model the semantic relationship weights between samples i and j in a set consisting of labeled and unlabeled samples. : ; in, Let i be the soft assignment vector for sample i. Indicates cluster determinism, This represents the similarity of the assignment distributions. Here, the soft assignment vector for sample i is the i-th row of the Q matrix.

[0050] Based on semantic relation weights With boundary threshold Calculate the cluster-aware relation scaling factor : .

[0051] S400. The collected samples are filtered according to the negative filtering mask. The prediction entropy is maximized for the filtered abnormal samples to obtain the multi-level negative refinement loss.

[0052] For samples identified as out-of-distribution anomalies with high confidence, this embodiment introduces a multi-level negative refinement mechanism during training. The boundary is optimized using the selected anomaly samples at the decision level and feature level, so that the anomaly samples are kept away from the in-distribution samples during semantic modeling.

[0053] In a specific embodiment, for the selected abnormal samples ( Maximizing the prediction entropy and calculating the multi-stage negative refining loss. : ; in, This represents the number of samples identified as abnormal in the current batch. For the collected samples Information entropy for predicting probability distributions.

[0054] Furthermore, for samples deemed anomalous, their scaling factor is forcibly set to the maximum exclusion constant (e.g., 1): .

[0055] To put it simply, by introducing a maximum repulsion constant into the contrastive clustering loss, the model can proactively push outlier samples away from the currently constructed semantic clusters within the distribution. This not only protects the semantic purity of known categories but also reserves representation space for unknown categories in the feature distribution.

[0056] S500: Calculate the classification semantic consistency matrix among unlabeled samples based on the classification prediction values ​​of the weakly enhanced unlabeled samples. Calculate the forward refinement weighting matrix among unlabeled samples based on the forward filtering mask and the classification semantic consistency matrix. Calculate the clustering bidirectional refinement loss based on the forward refinement weighting matrix, the scaling factor corresponding to the unlabeled samples, and the sample feature similarity among the unlabeled samples. The classification prediction values ​​of the weakly enhanced unlabeled samples are obtained through classification prediction (directly output by the classification prediction head).

[0057] In a specific embodiment, after obtaining the screening results of samples within the high-confidence distribution and high-confidence outlier samples, this embodiment further optimizes the semantic structure within the distribution through a clustering bidirectional refinement mechanism. Based on the unsupervised sample classification prediction results, according to the predicted values ​​of the i-th weakly enhanced sample and the j-th weakly enhanced sample selected from the unlabeled samples, the classification semantic consistency matrix between each sample and other samples is calculated. Then, combined with a positive filtering mask. Calculate the positive refined weighting matrix for sample i and sample j. : .

[0058] Finally, the clustering bidirectional refining loss is calculated. : ; in, This represents the cosine similarity function used to calculate the similarity of sample features between labeled samples; The temperature coefficient is used for comparative learning.

[0059] S600: The loss function is obtained by fusing the semi-supervised classification loss, anomaly score regression loss, multi-level negative refinement loss, and cluster bidirectional refinement loss. The anomaly detection model is trained using the loss function and the collected samples. The trained anomaly detection model is then used to detect the samples to be tested.

[0060] In a specific embodiment, the loss function obtained by fusion is: ; in, , and These are the balance weight coefficients for outlier score loss, cluster bidirectional refinement loss, and multi-level negative refinement loss, respectively.

[0061] Based on the above embodiments, after the model loss function is calculated, backpropagation uses gradient descent to update the model parameters. After multiple iterations of updating the model using the iterator learning rate, the trained model parameters are saved.

[0062] In some embodiments, a trained anomaly detection model is used to detect the sample to be tested, including: Input the sample to be tested into the trained anomaly detection model to obtain the in-distribution class probability and anomaly score; The outlier score of the sample to be tested is compared with the outlier score threshold. When the score is higher than the outlier score threshold, it is identified as an in-distribution sample; when the score is not higher than the outlier score threshold, it is identified as an out-of-distribution outlier sample. For the selected samples within the distribution, the category with the highest probability within the distribution is considered as its predicted category.

[0063] In summary, this embodiment improves the stability of unknown anomaly identification through soft targets with anomaly scores; reduces sample selection fluctuations caused by fixed thresholds through adaptive bidirectional thresholds; optimizes the feature space structure through soft clustering semantic relationship modeling; and transforms anomalous samples from passive filtering objects into training signals for actively optimizing decision boundaries through multi-level negative refinement. Furthermore, it can simultaneously improve in-distribution classification accuracy and out-of-distribution anomaly detection capability under conditions of a small number of labeled samples and a large number of mixed unlabeled samples.

[0064] Example 2: This embodiment provides a comparative analysis of an open-set semi-supervised image distribution anomaly detection method (this method) based on Embodiment 1 with cutting-edge methods (such as Fixmatch, MTCF, OpenMatch, T2T, SSB, IOMatch, SCOMatch, POT-OSSL), thereby demonstrating the effectiveness and advancement of this method.

[0065] In this embodiment, the CIFAR-100 dataset is used, containing 100 categories, with each category containing 500 training images and 100 test images. For in-distribution and out-of-distribution data, a 55 / 45 split is used between known in-distribution classes and unknown out-of-distribution classes. The area under the receiver operating characteristic curve (AUROC) is used as the evaluation metric for out-of-distribution anomaly detection; a higher value indicates better out-of-distribution anomaly detection performance. Accuracy is used as the evaluation metric for in-distribution sample classification; a higher value indicates better in-distribution classification detection performance.

[0066] During training, the number of training iterations (epochs) was set to 512, the number of batches per iteration was set to 64, and the temperature coefficient was set to [missing information]. =0.1, cluster center number K=55, weight coefficients of the loss function are set to =0.5, =2 and =0.1, boundary threshold set to =0.2.

[0067] Table 1. Summary of experimental results comparing our method with cutting-edge methods. Table 1 shows the comparative experimental results of the semantic refinement contrastive clustering method proposed in this invention (the proposed method is listed in the table) and existing state-of-the-art open-set semi-supervised learning methods, under the settings of 25 sample labels per class and 50 labels per class within the distribution. Regardless of whether the labels are scarce or relatively abundant, the proposed method achieves the best results in both Accuracy and AUROC, the two key metrics. This fully verifies the effectiveness and advancement of the soft clustering semantic relationship mining and bidirectional refinement strategy in solving the out-of-distribution anomaly detection task in open-set semi-supervised learning.

[0068] Example 3: like Figure 2 As shown, this embodiment provides an open-set semi-supervised image out-of-distribution anomaly detection system. The system is trained using the open-set semi-supervised image out-of-distribution anomaly detection method described in Embodiment 1, and then used to detect test samples. The open-set semi-supervised image out-of-distribution anomaly detection system includes a feature extractor, a classification prediction head, an anomaly score head, and a soft clustering module, all connected to the feature extractor. The anomaly score head is connected to the soft clustering module.

[0069] Furthermore, the feature extractor is used to extract high-dimensional features of the samples, the classification prediction head is used to output the probability within the K-class distribution based on the high-dimensional features, the anomaly score head is used to output the confidence score of the sample belonging to the normal sample within the distribution based on the high-dimensional features, and the soft clustering module is used to output the semantic relationship between samples (such as semantic relationship weights and scaling factors) based on the statistical distribution of the anomaly scores output by the high-dimensional features and the anomaly scores output by the anomaly score head.

[0070] Furthermore, the semi-supervised classification loss is obtained through the classification prediction head, the outlier score regression loss is obtained through the outlier score head, the multi-level negative refinement loss is obtained through the classification prediction head and the outlier score head, and the clustering bidirectional refinement loss is obtained through the soft clustering module.

[0071] In some embodiments, the feature extractor is Wide ResNet-28-2. Of course, the feature extractor can be replaced by ResNet, DenseNet, EfficientNet, ConvNeXt, VisionTransformer, Swing Transformer, or a lightweight mobile network, depending on the application scenario, as long as it can output feature vectors for classification and semantic clustering.

[0072] Furthermore, this embodiment includes the following steps in the image anomaly detection and classification process after model training: Step 1: Anomaly score and prediction probability extraction during the testing phase. In the testing phase, for the sample to be tested... Features are extracted using a feature extractor. Then, the classification prediction heads are used respectively. Output class probabilities within the distribution and abnormal score header Output abnormal scores .

[0073] Step 2: Identification of outlier samples. The outlier score of the sample to be tested... With abnormal score threshold Comparison, when the score is above the threshold When the score is below the threshold, it is considered an in-distribution sample; when the score is below the threshold, it is considered an in-distribution sample. When that happens, it is identified as an out-of-distribution anomalous sample.

[0074] Step 3: In-distribution sample classification. For in-distribution samples with scores higher than the threshold, classify them based on the in-distribution class probabilities. The category with the highest probability is considered the category to which the sample belongs.

[0075] It should be noted that the method in this embodiment is the same as that in Embodiment 1, and you can refer to Embodiment 1 for details.

[0076] It should be understood that although the steps in the flowcharts of the above embodiments are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the processes of the above embodiments may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages of other steps.

[0077] The above description is merely a preferred embodiment of the present invention. Those skilled in the art will understand that various changes or equivalent substitutions can be made to these features and embodiments without departing from the spirit and scope of the present invention. Furthermore, under the teachings of the present invention, these features and embodiments can be modified to adapt to specific situations and materials without departing from the spirit and scope of the present invention. Therefore, the present invention is not limited to the specific embodiments disclosed herein, and all embodiments falling within the scope of the claims of this application are within the protection scope of the present invention.

Claims

1. A method for detecting out-of-distribution anomalies in open-set semi-supervised images, characterized in that, include: Weak enhancement is performed on the collected labeled samples, and both weak and strong enhancement are performed on the collected unlabeled samples. High-dimensional features are extracted from the enhanced samples, and classification prediction is performed on the high-dimensional features to obtain semi-supervised classification loss and anomaly detection to obtain anomaly score regression loss. Based on the statistical distribution of the set of anomaly detection scores of labeled samples and anomaly detection scores of unlabeled samples, positive thresholds and negative thresholds are generated, and positive and negative filter masks are generated based on the positive and negative thresholds. The similarity between each sample feature and multiple cluster centers of the collected samples is iteratively optimized to obtain a soft-assignment weight matrix. The semantic relationship weights between the collected samples are constructed based on the soft-assignment weight matrix, and the cluster-aware relationship scaling factor between the collected samples is calculated based on the semantic relationship weights. The collected samples are filtered according to the negative filtering mask, and the prediction entropy is maximized for the filtered abnormal samples to obtain multi-level negative refinement loss. The classification semantic consistency matrix between unlabeled samples is calculated based on the classification prediction values ​​of the weakly enhanced unlabeled samples. The forward refinement weighting matrix between unlabeled samples is calculated based on the forward filtering mask and the classification semantic consistency matrix. The clustering bidirectional refinement loss is calculated based on the forward refinement weighting matrix, the scaling factor corresponding to the unlabeled sample, and the sample feature similarity between unlabeled samples. The loss function is obtained by fusing the semi-supervised classification loss, anomaly score regression loss, multi-level negative refinement loss, and cluster bidirectional refinement loss. The anomaly detection model is trained using the loss function and the collected samples. The trained anomaly detection model is then used to detect the samples to be tested.

2. The method for detecting out-of-distribution anomalies in open-set semi-supervised images according to claim 1, characterized in that, The weak enhancement includes random horizontal flipping and random pruning, and the strong enhancement adds a random enhancement strategy on the basis of the weak enhancement.

3. The method for detecting out-of-distribution anomalies in open-set semi-supervised images according to claim 1, characterized in that, The semi-supervised classification loss obtained by classifying and predicting high-dimensional features includes: Calculate the supervised loss between the true label of the labeled sample and its classification prediction value; obtain the pseudo label after prediction of the weakly enhanced unlabeled sample, and calculate the consistency loss between the pseudo label and the classification prediction value of the corresponding strongly enhanced unlabeled sample; fuse the supervised loss and the consistency loss to obtain the semi-supervised classification loss.

4. The method for detecting out-of-distribution anomalies in open-set semi-supervised images according to claim 3, characterized in that, The semi-supervised classification loss is obtained by fusing the supervised loss and the consistency loss, including: The consistency loss between the pseudo-label with a prediction confidence of not less than the threshold and the classification prediction value of its strongly enhanced unlabeled sample is calculated, and the consistency loss between the pseudo-label with a prediction confidence of less than the threshold and the classification prediction value of its strongly enhanced unlabeled sample is recorded as zero. The average value of the supervision loss is fused with the average value of the consistency loss to obtain the semi-supervised classification loss.

5. The method for detecting out-of-distribution anomalies in open-set semi-supervised images according to claim 1, characterized in that, The anomaly score regression loss obtained from high-dimensional feature anomaly detection includes: An anomaly score soft target is calculated based on the transfer scores of weakly enhanced unlabeled samples and multiple cluster centers, and the classification confidence of weakly enhanced unlabeled samples. The anomaly score regression loss is then calculated based on the anomaly score soft target, the anomaly detection scores of weakly enhanced labeled samples, and the anomaly detection scores of strongly enhanced unlabeled samples.

6. The method for detecting out-of-distribution anomalies in open-set semi-supervised images according to claim 5, characterized in that, The classification confidence of the weakly enhanced unlabeled sample is obtained based on the prediction confidence and classification entropy.

7. The method for detecting out-of-distribution anomalies in open-set semi-supervised images according to claim 1, characterized in that, For the selected abnormal samples, their scaling factor is forcibly set to the maximum rejection constant.

8. The method for detecting out-of-distribution anomalies in open-set semi-supervised images according to claim 1, characterized in that, The multiple cluster centers are obtained by iteratively updating the sample feature vectors of the labeled samples and the unlabeled samples that are forward-filtered by the forward filtering mask, using momentum update coefficients.

9. The method for detecting out-of-distribution anomalies in open-set semi-supervised images according to claim 1, characterized in that, The process of using a trained anomaly detection model to detect the sample to be tested includes: Input the sample to be tested into the trained anomaly detection model to obtain the in-distribution class probability and anomaly score; The outlier score of the sample to be tested is compared with the outlier score threshold. When the score is higher than the outlier score threshold, it is identified as an in-distribution sample; when the score is not higher than the outlier score threshold, it is identified as an out-of-distribution outlier sample. For the selected samples within the distribution, the category with the highest probability within the distribution is considered as its predicted category.

10. A semi-supervised open-set image distribution anomaly detection system, characterized in that, The open-set semi-supervised image distribution out-of-distribution anomaly detection system is trained using the open-set semi-supervised image distribution out-of-distribution anomaly detection method according to any one of claims 1-9, and then the system is used to detect the test samples after training; the system includes a feature extractor, a classification prediction head, an anomaly score head, and a soft clustering module that are all connected to the feature extractor, and the anomaly score head is connected to the soft clustering module; The feature extractor is used to extract high-dimensional features of the samples, the classification prediction head is used to output the probability within the K-class distribution based on the high-dimensional features, the anomaly score head is used to output the confidence score that the sample belongs to the normal sample within the distribution based on the high-dimensional features, and the soft clustering module is used to statistically distribute and output the semantic relationship between samples based on the high-dimensional features and the anomaly scores output by the anomaly score head. The semi-supervised classification loss is obtained through the classification prediction head, the outlier score regression loss is obtained through the outlier score head, the multi-level negative refinement loss is obtained through the classification prediction head and the outlier score head, and the clustering bidirectional refinement loss is obtained through the soft clustering module.

Images