An image and feature dual-guided out-of-distribution detection method
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2026-05-22
- Publication Date
- 2026-08-11
AI Technical Summary
[0005]本发明的目的是提供一种图像与特征双引导的分布外检测方法，通过直接干预采样过程，迫使模型生成多样化、高质量且语义可控的OOD样本，从而有效解决现有技术中语义不稳定、多样性匮乏及计算成本高的问题，显著提升分布外检测的鲁棒性与泛化性能
Smart Images
Figure CN122551047A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the fields of artificial intelligence and computer vision technology, and in particular to an out-of-distribution detection method guided by both image and feature. Background Technology
[0002] In the practical deployment of machine learning systems, out-of-distribution (OOD) detection is a crucial step in ensuring system safety and reliability. Especially in safety-critical applications such as autonomous driving and medical imaging diagnosis, models not only need to accurately identify data of known categories (i.e., in-distribution, ID), but also must be able to sensitively identify unknown categories or anomalous inputs that are significantly different from the training data distribution. Consequently, deep neural networks often produce overconfident predictions when faced with unknown inputs, posing serious safety risks.
[0003] Currently, traditional OE (Out of Exposure) methods enhance model robustness by introducing auxiliary anomaly datasets, but they heavily rely on large-scale manually collected data, resulting in high costs and difficulty in covering all potential distribution variations, thus limiting generalization ability. In recent years, synthesizing OOD samples using text-to-image diffusion models has become a new trend, but existing methods (such as Dream-OOD) mostly employ an "embedding space perturbation" strategy, which maps in-distribution samples to the latent space and then applies perturbations to generate anomalous samples. This strategy has significant drawbacks: First, small perturbations in the latent space are nonlinearly amplified during the diffusion process, leading to uncontrollable semantics in the generated images, either deviating too far from the target or becoming indistinguishable from normal samples; second, simply changing the semantic embedding is insufficient to simulate complex distribution variations such as covariate shifts or domain shifts, resulting in insufficient diversity of generated samples; finally, the alignment and mapping of the latent space adds additional computational overhead and procedural complexity.
[0004] Therefore, there is an urgent need for a novel detection method that can directly guide the diffusion sampling trajectory without additional training. This method should overcome the limitations of a single embedding perturbation and utilize the gradient information of an existing classifier to implement "image and feature dual guidance" in the pixel space (low-density region) and the feature space (sparse region). Summary of the Invention
[0005] The purpose of this invention is to provide an out-of-distribution detection method guided by both image and feature. By directly intervening in the sampling process, the method forces the model to generate diverse, high-quality, and semantically controllable OOD samples, thereby effectively solving the problems of semantic instability, lack of diversity, and high computational cost in the prior art, and significantly improving the robustness and generalization performance of out-of-distribution detection.
[0006] To achieve the above-mentioned objectives, the technical solution adopted by this invention is as follows:
[0007] An out-of-distribution detection method guided by both image and feature, characterized by comprising: acquiring a pre-trained text-to-image diffusion model and an existing in-distribution classifier;
[0008] An image-level guiding function is constructed, which pushes the sampling trajectory into a low-density region of the pixel space based on the free energy gradient of the in-distribution classifier.
[0009] Construct a feature-level guiding function that pushes the sampling trajectory toward the feature-sparse region based on the gradient of the k nearest neighbor distance in the latent space of the in-distribution classifier;
[0010] By utilizing the image-level and feature-level guiding functions, the denoising sampling process of the diffusion model can be directly intervened without additional training to generate diverse out-of-distribution samples;
[0011] The generated out-of-distribution samples are used to train the in-distribution classifier for anomaly exposure, thereby optimizing the out-of-distribution detection performance.
[0012] As an improvement, the logits of the input image are calculated using the in-distribution classifier, and the free energy function is defined. ,in For the first The class's logits;
[0013] Calculate the gradient of the free energy function with respect to the input image. The gradient direction points to the low likelihood region of the data distribution, serving as an image-level guiding signal.
[0014] As an improvement, the specific steps for constructing the feature-level guiding function include:
[0015] Extract the feature representations of the intermediate layer of the in-distribution classifier and construct a feature embedding library for in-distribution samples;
[0016] For the current sampled image, calculate its feature representation in the normalized feature space and its correlation with the feature embedding library. Euclidean distance between the nearest neighbor feature vectors ;
[0017] Calculate the distance gradient of the input image The gradient direction points to the sparse region in the feature space, serving as a feature-level guiding signal.
[0018] As an improvement, the denoising sampling process of the interference diffusion model specifically includes:
[0019] At each denoising time step of the diffusion model Using a training-free guidance framework, the gradients of the image-level guidance function and the feature-level guidance function are applied to the noise samples respectively. and predicted clean samples ;
[0020] By adjusting the variance-guided strength parameter and mean guiding strength parameter This controls the degree to which the sampling trajectory deviates from the in-distribution manifold, generating out-of-distribution samples with different degrees of anomaly.
[0021] As an improvement, the generation of diverse out-of-distribution samples employs a balanced sampling strategy:
[0022] Set multiple sets of different variance-guided strength parameters and mean guiding strength parameter The combination;
[0023] For each set of parameters, a diffusion sampling process is performed to generate an out-of-distribution sample set covering a continuous spectrum from slight distribution shifts to significant semantic anomalies.
[0024] As an improvement, a mixed training set is constructed, which includes samples from the original distribution and samples from the generated out-of-distribution distribution;
[0025] Define a total loss function that includes classification loss and out-of-distribution regularization loss, where the out-of-distribution regularization loss aims to maximize the free energy of the generated sample or minimize the confidence that it is identified as within the distribution;
[0026] The total loss function is used to fine-tune the in-distribution classifier so that it maintains high accuracy on in-distribution data while producing a low-confidence response on out-of-distribution data.
[0027] As an improvement, a unified out-of-distribution scoring detection mechanism is also included for out-of-distribution judgment during the testing phase:
[0028] The image-level likelihood score of the image to be tested is calculated based on the free energy function.
[0029] Calculate the feature-level distance score of the image to be tested, based on the k nearest neighbor distance;
[0030] The image-level likelihood score and feature-level distance score are adaptively fused to obtain a unified out-of-distribution detection score.
[0031] As an improvement, the adaptive fusion specifically includes: calculating the KL divergence between the test dataset and the in-distribution validation set in the feature space distribution; and dynamically calculating interpolation coefficients based on the KL divergence. When the feature space distribution shift is significant, increase the weight of the feature-level distance score; conversely, increase the weight of the image-level likelihood score. This is achieved through the formula... Calculate the final score, where The normalized feature distance, The score is the normalized free energy score.
[0032] As an improvement, the text description of the category to be detected is used as a conditional input diffusion model, so that the initial sampling point is located near the distribution manifold of that category;
[0033] During the denoising process, the image and feature dual guidance are applied, forcing the sampling trajectory to move from the class center to the low-density and sparse region outside the decision boundary.
[0034] An electronic device includes a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, when the processor executes the program, it implements the image and feature dual-guided out-of-distribution detection method as described in any one of claims 1 to 9.
[0035] The beneficial effects of this invention are as follows: By employing a dual-guided strategy of image and feature analysis, and utilizing free energy and k-nearest neighbor gradients to directly intervene in diffusion sampling, semantically controllable and diverse out-of-distribution samples are generated. This method requires no additional training, significantly reducing computational costs; it overcomes the semantic instability limitations caused by traditional embedding perturbations. Combined with an adaptive fusion scoring mechanism, it effectively addresses covariate, domain, and semantic shifts, greatly improving the robustness and generalization ability of out-of-distribution detection, and resolving the potential for model overconfidence in safety-critical scenarios. Attached Figure Description
[0036] Figure 1 This is an overall flowchart of an out-of-distribution detection method guided by both image and feature analysis according to the present invention.
[0037] Figure 2 This is a schematic diagram illustrating the guiding mechanism of an out-of-distribution detection method guided by both image and feature in this invention. Detailed Implementation
[0038] To make the content of this invention easier to understand, the technical solutions of the embodiments of this invention will be clearly and completely described below with reference to the accompanying drawings. Identical components are represented by the same reference numerals. It should be noted that the terms "front," "rear," "left," "right," "up," and "down" used in the following description refer to directions in the accompanying drawings, while the terms "inner" and "outer" refer to directions toward or away from the geometric center of a specific component, respectively.
[0039] like Figures 1 to 2As shown, an out-of-distribution detection method using both image and feature guidance is characterized by the following steps: acquiring a pre-trained text-to-image diffusion model and an existing in-distribution classifier; constructing an image-level guidance function to push the sampling trajectory towards low-density regions in the pixel space based on the free energy gradient of the in-distribution classifier; constructing a feature-level guidance function to push the sampling trajectory towards feature-sparse regions based on the k-nearest neighbor distance gradient in the latent space of the in-distribution classifier; directly intervening in the denoising sampling process of the diffusion model without additional training using the image-level and feature-level guidance functions to generate diverse out-of-distribution samples; and using the generated out-of-distribution samples to train the in-distribution classifier for abnormal exposure, thereby optimizing out-of-distribution detection performance. The in-distribution classifier is used to calculate the logits of the input image, and the free energy function is defined. ,in For the first The `logits` method of the class calculates the gradient of the free energy function with respect to the input image. The gradient direction points to the low likelihood region of the data distribution, serving as an image-level guiding signal.
[0040] The specific steps for constructing the feature-level guiding function include: extracting the feature representations of the intermediate layers of the in-distribution classifier, constructing a feature embedding library for in-distribution samples; for the current sampled image, calculating the relationship between its feature representation in the normalized feature space and the feature embedding library. Euclidean distance between the nearest neighbor feature vectors Calculate distance gradient of the input image The gradient direction points to the sparse region in the feature space, serving as a feature-level guiding signal. The denoising sampling process of the interference diffusion model specifically includes: at each denoising time step of the diffusion model... Using a training-free guidance framework, the gradients of the image-level guidance function and the feature-level guidance function are applied to the noise samples respectively. and predicted clean samples By adjusting the variance-guided strength parameter and mean guiding strength parameter This controls the degree to which the sampling trajectory deviates from the in-distribution manifold, generating out-of-distribution samples with different degrees of anomaly.
[0041] In addition, a balanced sampling strategy is used to generate diverse out-of-distribution samples: multiple sets of different variance-guided strength parameters are set. and mean guiding strength parameter The parameters are combined; for each combination, a diffusion sampling process is performed to generate an out-of-distribution sample set covering a continuous spectrum from slight distribution shifts to significant semantic anomalies. A mixed training set containing the original in-distribution samples and the generated out-of-distribution samples is constructed; a total loss function is defined, which includes classification loss and out-of-distribution regularization loss, where the out-of-distribution regularization loss aims to maximize the free energy of the generated samples or minimize their confidence in being identified as in-distribution; the in-distribution classifier is fine-tuned using the total loss function to maintain high accuracy on in-distribution data while producing low-confidence responses to out-of-distribution data.
[0042] like Figures 1 to 2 As shown, it also includes a unified out-of-distribution scoring detection mechanism for out-of-distribution judgment during the testing phase: calculating the image-level likelihood score of the image to be tested, based on the free energy function; calculating the feature-level distance score of the image to be tested, based on the k-nearest neighbor distance; and adaptively fusing the image-level likelihood score and the feature-level distance score to obtain a unified out-of-distribution detection score.
[0043] Secondly, adaptive fusion specifically includes: calculating the KL divergence between the test dataset and the in-distribution validation set in the feature space distribution; and dynamically calculating interpolation coefficients based on the KL divergence. When the feature space distribution shift is significant, increase the weight of the feature-level distance score; conversely, increase the weight of the image-level likelihood score. This is achieved through the formula... Calculate the final score, where The normalized feature distance, The result is the normalized free energy score. The text description of the category to be detected is used as the conditional input diffusion model to make the initial sampling point located near the distribution manifold of that category. During the denoising process, image and feature dual guidance are applied to force the sampling trajectory to move from the category center to the low-density and sparse region outside the decision boundary.
[0044] like Figures 1 to 2 As shown, an electronic device includes a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that the processor executes the program to implement the image and feature dual-guided out-of-distribution detection method as claimed in any one of claims 1 to 9.
[0045] In the implementation process, a pre-trained text-to-image diffusion model and an existing in-distribution classifier are first acquired. Then, two types of guidance functions are constructed: an image-level guidance function based on the classifier's free energy gradient, used to push the sampling trajectory towards low-density regions in the pixel space; and a feature-level guidance function based on the k-nearest neighbor distance gradient in the classifier's latent space, used to push the trajectory towards feature-sparse regions. Subsequently, without additional training, these two guidance functions are used to directly intervene in the denoising sampling process of the diffusion model, generating diverse and semantically controllable out-of-distribution samples by adjusting the guidance strength parameters. Finally, the generated samples are used to train the original classifier for abnormal exposure to optimize its detection performance. During the testing phase, adaptive fusion of image-level likelihood scores and feature-level distance scores is used to achieve unified scoring and accurate detection of out-of-distribution data.
[0046] The above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.
Claims
1. An image and feature dual-guided out-of-distribution detection method, characterized in that, include: Obtain pre-trained text-to-image diffusion models and ready-made in-distribution classifiers; An image-level guiding function is constructed, which pushes the sampling trajectory into a low-density region of the pixel space based on the free energy gradient of the in-distribution classifier; Construct a feature-level guiding function that pushes the sampling trajectory toward the feature-sparse region based on the gradient of the k nearest neighbor distance in the latent space of the in-distribution classifier; By utilizing the image-level and feature-level guiding functions, the denoising sampling process of the diffusion model can be directly intervened without additional training to generate diverse out-of-distribution samples; The generated out-of-distribution samples are used to train the in-distribution classifier for anomaly exposure, thereby optimizing the out-of-distribution detection performance.
2. The image and feature dual guided out-of-distribution detection method of claim 1, wherein, The logits of the input image are calculated using the in-distribution classifier, and the free energy function is defined. ,in For the first The class's logits; Calculate the gradient of the free energy function with respect to the input image. The gradient direction points to the low likelihood region of the data distribution, serving as an image-level guiding signal.
3. The image and feature dual-guided out-of-distribution detection method of claim 2, wherein, The specific steps for constructing the feature-level guiding function include: Extract the feature representations of the intermediate layer of the in-distribution classifier and construct a feature embedding library for in-distribution samples; For the current sampled image, compute the Euclidean distance of its feature representation in the normalized feature space to the first nearest neighbor feature vector in the feature embedding library ; computing the distance with respect to the gradient of the input image the gradient direction points to a sparse region in the feature space as a feature-level guidance signal.
4. The image and feature dual-guided out-of-distribution detection method of claim 3, wherein, The denoising sampling process of the interference diffusion model specifically includes: At each denoising time step of the diffusion model , gradients of the image-level guiding function and gradients of the feature-level guiding function are respectively applied to the noisy sample and the predicted clean sample using the training-free guiding framework; By adjusting the variance-guided strength parameter and mean guiding strength parameter This controls the degree to which the sampling trajectory deviates from the in-distribution manifold, generating out-of-distribution samples with different degrees of anomaly.
5. The out-of-distribution detection method guided by both image and feature analysis according to claim 4, characterized in that, The generation of diverse out-of-distribution samples employs a balanced sampling strategy: Setting different combinations of variance and mean steering strength parameters for different groups of pixels For each set of parameters, a diffusion sampling process is performed to generate an out-of-distribution sample set covering a continuous spectrum from slight distribution shifts to significant semantic anomalies.
6. The image and feature dual guided out-of-distribution detection method of claim 5, wherein, Construct a mixed training set containing samples from the original in-distribution set and samples from the generated out-of-distribution set; Define a total loss function that includes classification loss and out-of-distribution regularization loss, where the out-of-distribution regularization loss aims to maximize the free energy of the generated sample or minimize the confidence that it is identified as within the distribution; The total loss function is used to fine-tune the in-distribution classifier so that it maintains high accuracy on in-distribution data while producing a low-confidence response on out-of-distribution data.
7. The image and feature dual guided out-of-distribution detection method of claim 6, wherein, It also includes a unified out-of-distribution scoring detection mechanism for out-of-distribution judgment during the testing phase: The image-level likelihood score of the image to be tested is calculated based on the free energy function. Calculate the feature-level distance score of the image to be tested, based on the k nearest neighbor distance; The image-level likelihood score and feature-level distance score are adaptively fused to obtain a unified out-of-distribution detection score.
8. The image and feature dual guided out-of-distribution detection method of claim 7, wherein, The adaptive fusion specifically includes: calculating the KL divergence between the test dataset and the in-distribution validation set in the feature space distribution; and dynamically calculating interpolation coefficients based on the KL divergence. When the feature space distribution shift is significant, increase the weight of the feature-level distance score; conversely, increase the weight of the image-level likelihood score. This is achieved through the formula... Calculate the final score, where The normalized feature distance, The score is the normalized free energy score.
9. The image and feature dual guided out-of-distribution detection method of claim 1, wherein, The text description of the category to be detected is used as a conditional input diffusion model to make the initial sampling point located near the distribution manifold of that category; During the denoising process, the image and feature dual guidance are applied, forcing the sampling trajectory to move from the class center to the low-density and sparse region outside the decision boundary.
10. An electronic device comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the image and feature dual-guided out-of-distribution detection method as described in any one of claims 1 to 9.