A few-shot based image out-of-distribution detection method and system

Abstract

The application belongs to the technical field of computer vision and deep learning, and specifically discloses a kind of image distribution out detection method and system based on few samples.The application first solves the inconsistency problem of distribution out feature selection caused by the confidence fluctuation of distribution in samples in dynamic prompt tuning, and through the confidence peak triggered feature archiving mechanism, a high reliability distribution out feature warehouse is maintained for each distribution in sample by constructing the confidence peak triggered feature archiving mechanism, and only the archived features are updated when the sample prediction confidence breaks through the historical peak value, ensuring that stable and reliable supervision signals are provided for model training, and the consistency of gradient update direction is ensured, so that the model can accumulate robust distribution out discriminant knowledge, improve the stability and accuracy of few sample distribution out detection, and reduce the false positive rate of detection.

Description

Technical Field

[0001] This application belongs to the field of computer vision and deep learning technology, and more specifically, relates to a method and system for out-of-distribution image detection based on a small number of samples. Background Technology

[0002] Out-of-distribution (OOD) detection is a crucial component for the reliable deployment of open-world deep learning systems. Its goal is to accurately identify samples that do not belong to the in-distribution (ID) category during model training, preventing the model from making erroneous predictions based on overconfidence regarding unknown samples. In safety-sensitive applications such as autonomous driving and medical diagnosis, the performance of out-of-distribution detection directly impacts system safety. Failure to accurately detect out-of-distribution samples such as unknown obstacles and rare diseases could lead to catastrophic consequences.

[0003] With the development of Visual-Language Models (VLMs), cue-based optimization methods have become the mainstream approach for out-of-distribution detection with few samples. These methods improve detection performance by mining out-of-distribution features from in-distribution samples with few samples and using them as regularization signals to constrain cue learning. However, existing cue-based out-of-distribution detection methods have a fundamental flaw: the dynamic cue optimization process causes significant fluctuations in the prediction confidence of in-distribution samples between training epochs, resulting in inconsistent selection of out-of-distribution features during training. That is, reliable out-of-distribution features extracted in high-confidence epochs may be discarded in low-confidence epochs. This inconsistency disrupts the directional stability of gradient updates, preventing the model from accumulating robust out-of-distribution discriminative knowledge, and ultimately severely limiting out-of-distribution detection performance in few-sample scenarios.

[0004] Early out-of-distribution detection methods only used in-distribution data to train models and decided sample types by predicting confidence levels. However, due to the lack of explicit modeling of out-of-distribution data, detection performance was limited. Some methods improved detection performance by using auxiliary datasets or synthesizing outliers, but these required a large amount of additional data and could not be adapted to scenarios with few samples. Existing self-calibration hinting optimization methods can adjust the learning weights of classification and regularization by adjusting confidence levels, which can alleviate the problem of unreliable out-of-distribution features for low-confidence samples. However, they still do not solve the core problem of inconsistent selection of out-of-distribution features during training and cannot fundamentally guarantee the stability of model training.

[0005] Therefore, how to solve the inconsistency in the selection of out-of-distribution features in dynamic prompting optimization, provide stable and highly reliable out-of-distribution supervision signals for model training, and improve the performance of out-of-distribution detection in scenarios with few samples has become an urgent technical problem to be solved. Summary of the Invention

[0006] To address the shortcomings of existing technologies, the purpose of this application is to provide a method and system for out-of-distribution image detection based on a small number of samples, aiming to solve the problem of inconsistent selection of out-of-distribution features.

[0007] To achieve the above objectives, in a first aspect, this application provides a few-shot out-of-distribution image detection method, comprising: acquiring a sample image to be detected; extracting visual features from the sample image to be detected, calculating the similarity between the extracted visual features and the in-distribution category text features, thereby obtaining the predicted probability distribution of each in-distribution category; comparing the maximum predicted probability with a set threshold, and determining whether the sample image to be detected belongs to the in-distribution or out-of-distribution category based on the comparison result; wherein, the in-distribution category text features are obtained by iteratively optimizing and training a few-shot out-of-distribution detection network using a loss function; the in-distribution category text includes a category name and learnable cue word parameters, and the in-distribution category text features are obtained by processing the in-distribution category text through a text encoder; the few-shot out-of-distribution detection network is trained based on in-distribution sample images and corresponding in-distribution category text labels, and its loss function is determined by the similarity between the in-distribution visual features and the corresponding in-distribution category text features, as well as the similarity between the out-of-distribution features corresponding to the historical confidence peak and the corresponding in-distribution category text features; during the training process, the historical confidence peak is updated only when the real-time predicted confidence of an in-distribution sample exceeds its historical confidence peak, and the currently extracted out-of-distribution features are saved.

[0008] Preferably, the few-shot out-of-distribution detection model includes: a confidence calculation module, used to predict the category of in-distribution sample images, calculate the real-time prediction confidence of in-distribution samples, and track and record the historical confidence peak of each in-distribution sample image; an out-of-distribution feature extraction module, used to identify out-of-distribution regions in in-distribution sample images, extract local features of the region, and obtain several out-of-distribution features; a confidence peak-triggered feature archiving module, used to establish a dedicated out-of-distribution feature repository for each in-distribution sample image, update the historical confidence peak only when the real-time prediction confidence of an in-distribution sample exceeds its historical confidence peak, and save all currently extracted out-of-distribution features; and a model optimization module, used to construct a loss function and iteratively train the few-shot out-of-distribution detection network using the classification results of in-distribution samples and the out-of-distribution features corresponding to the historical confidence peak as supervision signals, so as to fine-tune the in-distribution category text features.

[0009] Preferably, the confidence calculation module is built on a vision-language pre-trained model. It extracts global visual features of the sample images within the distribution through an image encoder, extracts category text embedding features within the distribution through a text encoder, calculates the cosine similarity between the visual features and the text embedding features, and obtains the prediction probability of the sample images within the distribution for each target category after Softmax normalization. The prediction probability corresponding to the true label of the sample images within the distribution is used as the real-time prediction confidence.

[0010] Preferably, the out-of-distribution feature extraction module calculates the alignment score between the local image features of the in-distribution samples and the category text embedding features, sorts the predicted probabilities of the local features, determines the local regions whose sorting results exceed the preset rank parameter K as out-of-distribution regions, and extracts the local features of the region as out-of-distribution features.

[0011] Preferably, the confidence peak-triggered feature archiving module has a built-in confidence tracker and an out-of-distribution feature warehouse. The confidence tracker is initialized to 0, and the out-of-distribution feature warehouse is initialized to an empty set. In each round of training, if the real-time prediction confidence of a sample is greater than the historical peak recorded by the confidence tracker, the confidence tracker is updated to the current real-time confidence, and the current initial out-of-distribution features are overwritten and stored in the out-of-distribution feature warehouse. If the historical peak is not exceeded, the feature warehouse remains unchanged, and the archived high-reliability out-of-distribution features continue to be output.

[0012] Preferably, the step of comparing the maximum predicted probability with a set threshold and determining whether the sample image to be detected belongs to the distribution or not is as follows: A simplified R-MCM score is used for calculation. This score is obtained by adding the MCM score obtained from global visual features and the L-MCM score obtained from local features. The MCM score is the maximum predicted probability of the global features for each category, and the L-MCM score is the average of the first k maximum predicted probabilities of the local features. When the detection score is greater than or equal to the preset threshold, the sample image to be detected is determined to be within the distribution; otherwise, it is outside the distribution.

[0013] Preferably, the similarity between visual features within the distribution and text features of the corresponding category within the distribution is represented by cross-entropy loss, and the similarity between out-of-distribution features corresponding to historical confidence peaks and text features of the corresponding category within the distribution is represented by out-of-distribution regularization loss. The real-time prediction confidence of samples within the distribution is used as the weight of cross-entropy loss, and the two parts of loss are weighted and the learnable prompt word parameters are updated by gradient descent of the loss function.

[0014] Preferably, the in-distribution training dataset with few samples is a small sample dataset with 1, 4 or 16 samples per class randomly selected from the target dataset, and the out-of-distribution test dataset has no semantic overlap with the in-distribution dataset.

[0015] To achieve the above objectives, in a second aspect, this application provides an out-of-distribution image detection system based on a small number of samples, including a memory and one or more processors; the memory is coupled to the one or more processors, the memory is used to store computer program code, the computer program code including computer instructions; the one or more processors invoke the computer instructions to cause the system to execute the out-of-distribution image detection method as described in the first aspect.

[0016] It is understandable that the beneficial effects of the second aspect mentioned above can be found in the relevant descriptions in the first aspect mentioned above, and will not be repeated here.

[0017] Overall, the technical solutions conceived in this application have the following beneficial effects compared with the prior art: This application proposes a few-sample out-of-distribution image detection method, which for the first time solves the problem of inconsistent out-of-distribution feature selection caused by fluctuations in the confidence of in-distribution samples during dynamic cue tuning. By constructing a feature archiving mechanism triggered by confidence peaks, a highly reliable out-of-distribution feature repository is maintained for each in-distribution sample. The archived features are updated only when the predicted confidence of a sample exceeds the historical peak, ensuring a stable and reliable supervision signal for model training, guaranteeing the consistency of gradient update direction, enabling the model to accumulate robust out-of-distribution discrimination knowledge, improving the stability and accuracy of few-sample out-of-distribution detection, and reducing the false positive rate. Attached Figure Description

[0018] Figure 1 This is a schematic flowchart of an out-of-distribution image detection method based on a small number of samples provided in an embodiment of this application.

[0019] Figure 2 This is a schematic diagram of the feature archiving mechanism provided in the embodiments of this application.

[0020] Figure 3 This is a flowchart of the inference process for out-of-distribution image detection provided in an embodiment of this application.

[0021] Figure 4 This is a comparison chart of results provided by the embodiments of this application, where SCT is the name of the existing best method and AHOF is the name of the method of this application. Detailed Implementation

[0022] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0023] In this application, the term "and / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent three cases: A existing alone, A and B existing simultaneously, and B existing alone. In this application, the symbol " / " indicates that the related objects are in an "or" relationship, for example, A / B means A or B.

[0024] In this application, the terms "first" and "second," etc., are used to distinguish different objects, not to describe a specific order of objects. For example, "first response message" and "second response message," etc., are used to distinguish different response messages, not to describe a specific order of response messages.

[0025] In the embodiments of this application, the terms "exemplary" or "for example" are used to indicate that something is an example, illustration, or description. Any embodiment or design that is described as "exemplary" or "for example" in the embodiments of this application should not be construed as being more preferred or advantageous than other embodiments or design. Specifically, the use of the terms "exemplary" or "for example" is intended to present the relevant concepts in a specific manner.

[0026] In the description of the embodiments of this application, unless otherwise stated, "multiple" means two or more, for example, multiple processing units means two or more processing units, multiple elements means two or more elements, etc.

[0027] The embodiments of this application are described below with reference to the accompanying drawings.

[0028] like Figure 1 As shown, this application provides an out-of-distribution image detection method based on a small number of samples, including the following steps: S1. Construct a few-sample out-of-distribution detection network, wherein the few-sample out-of-distribution detection network includes a confidence calculation module, an out-of-distribution feature extraction module, a confidence peak-triggered feature archiving module, and a model optimization module.

[0029] The confidence calculation module is used to predict in-distribution samples, calculate the real-time prediction confidence of the samples, and track and record the historical confidence peak of each sample. The out-of-distribution feature extraction module is used to identify out-of-distribution feature regions in in-distribution samples, extract local features of the regions, and form an initial out-of-distribution feature set. The confidence peak-triggered feature archiving module is used to establish a dedicated out-of-distribution feature warehouse for each in-distribution sample. Only when the real-time prediction confidence of a sample exceeds its historical confidence peak will the currently extracted initial out-of-distribution features be updated and archived to the feature warehouse, and the archived high-reliability out-of-distribution features will be output. The model optimization module is used to construct a loss function and iteratively optimize and train the detection network using the classification results of in-distribution samples and the archived high-reliability out-of-distribution features as supervision signals.

[0030] Furthermore, the confidence calculation module is built on a vision-language pre-trained model (such as CLIP), and its core calculation process is as follows: extracting global visual features of samples within the distribution through an image encoder. Extracting category text embedding features through a text encoder ,in, For samples within the distribution, For the text description of class m, calculate the cosine similarity between visual features and text features. After normalizing the temperature parameter τ using Softmax, the predicted probability of the sample for the m-th class is obtained:

[0031] Using the true labels of the samples Corresponding prediction probability This serves as the real-time prediction confidence level for the sample.

[0032] Furthermore, the out-of-distribution feature extraction module achieves out-of-distribution region identification by mining local features of in-distribution samples. Its core process is: extracting local image features of the samples. Calculate the predicted probability of each local feature for the true label. The predicted probabilities of all local features are sorted, and local regions with a rank greater than a preset parameter K are identified as out-of-distribution feature regions, from which local features are extracted. As an external feature of the initial distribution.

[0033] Furthermore, the feature archiving module triggered by the confidence peak has a built-in confidence tracker. and distribution of external feature warehouses For each in-distribution sample ,initialization , In each round of training, if the confidence level of the sample is predicted in real time... Then update and the current initial distribution features Archive to ,Right now ;like If not, the feature repository will not be updated, and output will continue. This module archives high-reliability out-of-distribution features. It ensures that only reliable out-of-distribution features extracted when the sample confidence exceeds the historical peak are used for model training, avoiding interference from low-quality features.

[0034] In one illustrated embodiment, the extraction process for out-of-distribution (OOD) features is as follows: Figure 2As shown in (a), the CLIP model's predicted probability for this sample (dog) is also obtained (0.53). This value serves as a quantitative indicator of the model's uncertainty (or confidence level) regarding the sample.

[0035] Figure 2 Figure (b) illustrates the core components of the proposed method, including the out-of-distribution feature library and the feature archiving strategy based on confidence peaks (the area within the dashed box): (i) when the sample confidence does not exceed the historical peak (the area below the green dashed line), no operation is performed; (ii) when the sample confidence is higher than the historical peak (the area above the green dashed line), the historical peak is updated to the current confidence (corresponding to the red dashed line), and the corresponding out-of-distribution features are archived into the out-of-distribution feature library. Finally, the accumulated out-of-distribution features in the library will be output as the result.

[0036] Furthermore, the total loss function constructed by the model optimization module is a weighted sum of cross-entropy loss and out-of-distribution regularization loss, specifically in the form of:

[0037] in, Cross-entropy loss is used to constrain the model's in-distribution classification accuracy. The out-of-distribution regularization loss is calculated using the negative entropy of the predicted probability vector, which is used to learn with out-of-distribution feature constraints with high reliability. and For adaptive modulation factors based on prediction confidence, improve at low confidence levels. To enhance classification learning and improve performance at high confidence levels To enhance out-of-distribution regularization; λ is the loss balance parameter, preferably set to 0.4; The training dataset is distributed. The model minimizes the total loss function using stochastic gradient descent, enabling iterative updates of network parameters such as learnable cue parameters.

[0038] S2. Using the dataset within the few-sample distribution as the training set, input it into the few-sample out-of-distribution detection network, and perform an iterative training process of confidence calculation, out-of-distribution feature extraction, feature archiving, and model optimization to obtain a trained few-sample out-of-distribution detection model.

[0039] Specifically, a small sample in-distribution dataset with 1, 4, or 16 samples from each class is preprocessed (e.g., randomly cropped to 256×256, pixel value normalized) before being input into the detection model. In each training round, real-time confidence calculation and initial out-of-distribution feature extraction are performed sequentially for each sample. A feature archiving module, triggered by the confidence peak, determines whether to update the feature repository. Finally, the model optimization module uses the archived high-reliability out-of-distribution features as a regularization signal, combined with the classification loss, to update the network parameters. This iterative training is repeated until the model converges, resulting in a trained detection model. The out-of-distribution test dataset has no semantic overlap with the in-distribution dataset.

[0040] S3. Input the test sample into the trained out-of-distribution detection model with few samples, and output the detection result of whether the sample is in-distribution or out-of-distribution by comparing the detection score calculated by the model with the preset threshold.

[0041] Furthermore, the detection score mentioned in step S3 is calculated using a simplified R-MCM score, which integrates discriminative information from both global and local features. The calculation formula is as follows:

[0042] in, The MCM score is the maximum predicted probability of global visual features for all categories. The L-MCM score is the average of the k largest predicted probabilities of local features. Compared to traditional MCM and GL-MCM scores, this score can more comprehensively uncover the in-distribution / out-of-distribution features of the sample, thus improving detection accuracy.

[0043] Specifically, the test samples are preprocessed in the same way as the training set and then input into the model. The model outputs a detection score (simplified R-MCM score) through the confidence calculation module. When the detection score is greater than or equal to the preset threshold μ, the sample is determined to be within the distribution; when the detection score is less than μ, the sample is determined to be outside the distribution.

[0044] like Figure 3 As shown, the input to the text encoder is trained using the method mentioned in this paper. For each sample, text features and image features are obtained through the text encoder and image encoder, respectively. Then, the R-MCM score of each sample is calculated using a formula. Samples with R-MCM scores less than a set threshold are out-of-distribution samples.

[0045] According to another aspect of this application, an out-of-distribution image detection system based on few samples is provided. The system includes a data input unit, a model training unit, and a detection inference unit. The units work together to achieve efficient and stable out-of-distribution detection with few samples.

[0046] The data input unit is used to preprocess the training dataset and test dataset within the distribution, including operations such as image normalization and scaling, and input the processed standardized data into the model training unit or detection inference unit according to a preset format to ensure the consistency of data input.

[0047] The model training unit is the core training module of the system, integrating a confidence calculation module, an out-of-distribution feature extraction module, a confidence peak-triggered feature archiving module, and a model optimization module. Internally, the connection relationship between each module is established according to the data processing flow. This unit receives training data from the few-sample in-distribution data input unit, executes the iterative training process of steps S1-S2, updates the network parameters through multiple training iterations, and finally outputs a converged, trained few-sample out-of-distribution detection model, and stores the model parameters to a specified path.

[0048] The detection inference unit is used to load the trained detection model to achieve rapid detection of test samples. The unit receives the preprocessed test samples from the data input unit, calls the model's confidence calculation module to calculate the R-MCM detection score of the sample, compares the detection score with a preset threshold μ, and outputs the detection result of each test sample as "in-distribution" or "out-of-distribution" in real time. It can also support the visualization and storage of detection results.

[0049] Furthermore, this system can be deployed in safety-sensitive computer vision applications such as autonomous driving, plant and animal identification, intelligent monitoring, and industrial quality inspection. It adapts to the out-of-distribution detection needs of small sample sizes in open worlds, providing technical support for the reliable deployment of deep learning systems. For example, in the plant and animal identification scenario, the sample images include real-world images of various plants and animals. The category name includes the English name of the plant or animal in the corresponding image, such as American bullfrog, bald eagle, and great white shark. The learnable cue words are the learnable context embedded before and after the category name. Specifically, it can be understood as: (what kind of) American bullfrog (where it is or what it is doing, etc.). The words inside the parentheses are the learnable cue words, which enrich the semantics as context, thus making the text features and corresponding image features more closely match during subsequent reasoning.

[0050] Example This embodiment uses ImageNet-1k as the in-distribution (ID) dataset and iNaturalist, SUN, Places, and Texture as the out-of-distribution (OOD) datasets, employing a 1-shot few-sample setting (randomly selecting 1 sample from each class as the training set) to verify the detection performance of the method in this application.

[0051] Dataset preprocessing: Images from the ImageNet-1k training set, test set, and each OOD dataset are uniformly and randomly cropped to 256×256 pixels, and pixel values ​​are normalized to [-1,1]. One sample is randomly selected from each class in ImageNet-1k to construct a 1-shot few-sample training set, and all samples in the test set are used as the ID test set.

[0052] Detection Network Construction: A detection network was built based on the CLIP-B / 16 vision-language pre-trained model, including a confidence calculation module, an out-of-distribution feature extraction module, a confidence peak-triggered feature archiving module, and a model optimization module. The rank parameter K=200 for the out-of-distribution feature extraction module, the loss balance parameter λ=0.4 for the model optimization module, and an adaptive modulation factor. and Linear calculation based on confidence level.

[0053] Model training: Input the 1-shot training set into the detection network, set the training epochs T=25, and the learning rate η=5e-5; in each training epoch, perform the following sequentially: (1) The confidence calculation module outputs the real-time prediction confidence of each sample; (2) The out-of-distribution feature extraction module extracts the initial out-of-distribution features; (3) The feature archiving module triggered by the confidence peak determines whether to update the feature warehouse and outputs high-reliability out-of-distribution features; (4) The model optimization module calculates the total loss function and updates the network's cue parameters and encoder fine-tuning parameters.

[0054] Train until the model converges to obtain a well-trained out-of-distribution detection model with few samples.

[0055] Model testing: Input the samples from the ID test set and each OOD test set into the trained model, calculate the simplified R-MCM detection score for each sample, compare it with the threshold μ=0.5, and output the detection result.

[0056] Performance evaluation: FPR95 (false positive rate at 95% true positive rate) and AUROC (area under ROC curve) were used as evaluation indicators. The smaller the FPR95 and the larger the AUROC, the better the detection performance.

[0057] The test results of this embodiment show that, under the 1-shot setting, the method of this application has an average FPR95 of 28.60% and an average AUROC of 92.87% on the iNaturalist, SUN, Places, and Texture datasets. Compared with the existing best SCT method, the average FPR95 is reduced by 3.68% and the AUROC is increased by 0.66%, which significantly improves the performance of out-of-distribution detection with few samples.

[0058] like Figure 4 As shown, each group of images, from left to right, consists of: the original image, the extraction result using the SCT method, and the extraction result using the AHOF method. The colored blocks represent the identified out-of-distribution regions. The extraction results clearly demonstrate the advantages of the proposed method in segmenting both out-of-distribution and in-distribution regions of the same image sample.

[0059] The out-of-distribution feature extraction method in this application is adaptive and highly reliable. By evaluating and selecting the feature quality across training rounds, it filters out low-quality out-of-distribution features from low-confidence rounds, avoiding interference from noisy features on model optimization. This significantly improves the performance of out-of-distribution detection in scenarios with few samples. In the 1-shot setting of the ImageNet-1k benchmark dataset, it reduces the average FPR95 (false positive rate at 95% true positive rate) by 3.68% compared to the existing best method, establishing a new performance benchmark.

[0060] The detection network proposed in this application has good compatibility with existing visual-language models and cueing optimization methods. It can be combined with mainstream methods such as LoCoOp and SCT. After combination, it can still maintain and improve out-of-distribution detection performance. At the same time, this method does not rely on a large amount of out-of-distribution data or auxiliary datasets. It can achieve efficient out-of-distribution detection using only a small number of in-distribution data, making it suitable for low-sample scenarios in practical applications.

[0061] It should be understood that the above-described device is used to execute the methods in the above embodiments. The implementation principle and technical effect of the corresponding program modules in the device are similar to those described in the above methods. The working process of the device can be referred to the corresponding process in the above methods, and will not be repeated here.

[0062] Based on the methods in the above embodiments, this application provides an electronic device that may include a processor, a communications interface, a memory, and a communication bus, wherein the processor, communications interface, and memory communicate with each other via the communication bus. The processor may invoke logical instructions stored in the memory to execute the methods in the above embodiments.

[0063] Furthermore, the logical instructions in the aforementioned memory can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application.

[0064] Based on the methods in the above embodiments, this application provides a computer-readable storage medium storing a computer program that, when run on a processor, causes the processor to execute the methods in the above embodiments.

[0065] Based on the methods in the above embodiments, this application provides a computer program product that, when run on a processor, causes the processor to execute the methods in the above embodiments.

[0066] It is understood that the processor in the embodiments of this application can be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, transistor logic devices, hardware components, or any combination thereof. A general-purpose processor can be a microprocessor or any conventional processor.

[0067] The method steps in this application embodiment can be implemented in hardware or by a processor executing software instructions. The software instructions can consist of corresponding software modules, which can be stored in random access memory (RAM), flash memory, read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), registers, hard disks, portable hard disks, CD-ROMs, or any other form of storage medium known in the art. An exemplary storage medium is coupled to the processor, enabling the processor to read information from and write information to the storage medium. Of course, the storage medium can also be a component of the processor. The processor and the storage medium can reside in an ASIC.

[0068] In the above embodiments, implementation can be achieved entirely or partially through software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented entirely or partially as a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted through the computer-readable storage medium. The computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid-state disk (SSD)).

[0069] It is understood that the various numerical designations used in the embodiments of this application are merely for the convenience of description and are not intended to limit the scope of the embodiments of this application.

[0070] Those skilled in the art will readily understand that the above description is merely a preferred embodiment of this application and is not intended to limit this application. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this application should be included within the scope of protection of this application.

Claims

1. A method for detecting out-of-distribution images based on a small number of samples, characterized in that, include: The process involves: acquiring the sample image to be detected; extracting visual features from the sample image; calculating the similarity between the extracted visual features and the text features of the in-distribution categories to obtain the predicted probability distribution of each in-distribution category; comparing the maximum predicted probability with a set threshold, and determining whether the sample image belongs to the in-distribution or out-of-distribution category based on the comparison result; and so on. The in-distribution category text features are obtained by iteratively optimizing and training a few-shot out-of-distribution detection network using a loss function; the in-distribution category text includes category names and learnable cue word parameters, and the in-distribution category text features are obtained by processing the in-distribution category text through a text encoder; the few-shot out-of-distribution detection network is trained based on in-distribution sample images and corresponding in-distribution category text labels, and its loss function is determined by the similarity between in-distribution visual features and corresponding in-distribution category text features, as well as the similarity between out-of-distribution features corresponding to historical confidence peaks and corresponding in-distribution category text features; During training, the historical confidence peak is updated only when the real-time prediction confidence of an in-distribution sample exceeds its historical confidence peak, and the currently extracted out-of-distribution features are saved.

2. The image distribution out-of-distribution detection method as described in claim 1, characterized in that, The out-of-distribution detection model with few samples includes: The confidence calculation module is used to predict the category of in-distribution sample images, calculate the real-time prediction confidence of in-distribution samples, and track and record the historical confidence peak of each in-distribution sample image. The out-of-distribution feature extraction module is used to identify out-of-distribution regions in in-distribution sample images, extract local features of these regions, and obtain several out-of-distribution features. The confidence peak-triggered feature archiving module is used to establish a dedicated out-of-distribution feature warehouse for each in-distribution sample image. It updates the historical confidence peak only when the real-time prediction confidence of an in-distribution sample exceeds its historical confidence peak, and saves all currently extracted out-of-distribution features. The model optimization module is used to construct a loss function and iteratively train a few-sample out-of-distribution detection network by using the classification results of in-distribution samples and the out-of-distribution features corresponding to historical confidence peaks as supervision signals, so as to fine-tune the in-distribution category text features.

3. The image distribution out-of-distribution detection method as described in claim 2, characterized in that, The confidence calculation module is built on a vision-language pre-trained model. It extracts global visual features of the sample images within the distribution through an image encoder, extracts the category text embedding features within the distribution through a text encoder, calculates the cosine similarity between the visual features and the text embedding features, and obtains the prediction probability of the sample images within the distribution for each target category after Softmax normalization. The prediction probability corresponding to the true label of the sample images within the distribution is used as the real-time prediction confidence.

4. The image out-of-distribution detection method as described in claim 2, characterized in that, The out-of-distribution feature extraction module calculates the alignment score between the local image features of the in-distribution samples and the category text embedding features, sorts the predicted probabilities of the local features, and determines the local regions whose sorting results exceed the preset rank parameter K as out-of-distribution regions, and extracts the local features of the regions as out-of-distribution features.

5. The image distribution out-of-distribution detection method as described in claim 2, characterized in that, The confidence peak-triggered feature archiving module has a built-in confidence tracker and an out-of-distribution feature warehouse. The confidence tracker is initialized to 0, and the out-of-distribution feature warehouse is initialized to an empty set. In each round of training, if the real-time prediction confidence of a sample is greater than the historical peak recorded by the confidence tracker, the confidence tracker is updated to the current real-time confidence, and the current initial out-of-distribution features are overwritten and stored in the out-of-distribution feature warehouse. If the historical peak value is not exceeded, the feature repository remains unchanged, and the archived high-reliability out-of-distribution features continue to be output.

6. The image out-of-distribution detection method as described in claim 1, characterized in that, The process involves comparing the maximum predicted probability with a set threshold, and determining whether the image to be detected belongs to the distribution or is outside the distribution based on the comparison result, as follows: A simplified R-MCM score is used for calculation. This score is obtained by adding the MCM score obtained from global visual features and the L-MCM score obtained from local features. The MCM score is the maximum predicted probability of global features for each category, and the L-MCM score is the average of the top k maximum predicted probabilities of local features. When the detection score is greater than or equal to the preset threshold, the image to be detected is determined to be within the distribution; otherwise, it is outside the distribution.

7. The image distribution out-of-distribution detection method as described in claim 1, characterized in that, Cross-entropy loss is used to represent the similarity between visual features within the distribution and text features of the corresponding category within the distribution. Out-of-distribution regularization loss is used to represent the similarity between out-of-distribution features corresponding to historical confidence peaks and text features of the corresponding category within the distribution. The real-time prediction confidence of samples within the distribution is used as the weight of cross-entropy loss. The two parts of the loss are weighted and the gradient descent of the loss function is used to update the parameters of the learnable prompt words.

8. The image out-of-distribution detection method according to any one of claims 1-7, characterized in that, The in-distribution training dataset with few samples is a small sample dataset of 1, 4 or 16 samples per class randomly selected from the target dataset, and the out-of-distribution test dataset has no semantic overlap with the in-distribution dataset.

9. A system for detecting out-of-distribution image distributions based on a small number of samples, characterized in that, Includes memory and one or more processors; The memory is coupled to the one or more processors, and the memory is used to store computer program code, the computer program code including computer instructions; The one or more processors invoke the computer instructions to cause the system to perform the image out-of-distribution detection method as described in any one of claims 1 to 8.

Images