A few-shot contraband image recognition method fusing language and frequency prior

Abstract

This invention relates to the field of artificial intelligence technology, specifically to a method for few-shot contraband image recognition that integrates language and frequency priors. This method includes the following steps: text prompt generation: inputting the contraband category C and scene context S into a large language model to generate structured prompt text, and extracting the prompt embedding vector through a text embedding encoder. This invention combines the structured text prompt information generated by GPT-4 with task-related frequency component analysis, jointly guiding the visual-language model (CLIP) to learn more discriminative feature representations at multiple levels, thereby improving the model's classification performance and generalization ability under data-scarce conditions.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence technology, and more specifically, to a method for recognizing contraband images using a few samples that integrates language and frequency priors. Background Technology

[0002] Few-shot classification aims to learn a classification model with strong generalization ability from a very small number of labeled samples, and has wide applications in data-scarce fields such as contraband detection and medical image analysis. Existing methods mainly rely on image spatial features or visual-language models, but have not fully explored the synergistic potential of image frequency structure features and multimodal knowledge. On the other hand, frequency domain features have been proven to be effective in improving classification robustness, especially in dealing with background noise with high similarity or fine-grained class distinctions, where they have a significant advantage.

[0003] In recent years, few-sample image recognition technology has demonstrated its wide application value in practical scenarios such as security monitoring and intelligent inspection. However, the recognition of contraband images faces two major challenges: first, due to the difficulty of data acquisition, most scenarios only provide a very small number of labeled image samples; second, the variety of contraband images, the variability in scenarios, and the inconsistent distribution of semantic features make it difficult for traditional models to generalize recognition under few-sample conditions.

[0004] Current mainstream methods mostly focus on feature extraction and matching optimization in visual space, lacking effective utilization of redundant information in the image frequency domain and structured integration of prior linguistic knowledge. Therefore, existing technologies have the following shortcomings:

[0005] 1. Lack of language-guided image-text coupling mechanism: Most few-shot recognition methods rely solely on image spatial features and do not fully utilize the category prior and contextual information carried by language descriptions, resulting in a lack of semantic guidance in the recognition results.

[0006] 2. Ignoring discrimination information in the frequency domain: Existing models are mostly trained and inferred in the spatial domain, without considering redundant components and interference patterns in the image frequency domain, thus affecting the model's ability to focus on and discriminate specific regions.

[0007] 3. Limited image-text alignment granularity: Existing image-text fusion models mostly adopt a global alignment strategy, which makes it difficult to achieve fine-grained regional semantic matching, affecting the model's accurate localization and recognition of target regions.

[0008] 4. Static loss design, lacking dynamic adaptation mechanism: Traditional multi-task loss functions cannot automatically adjust weights according to the semantic complexity and visual variation of input samples, affecting the model's adaptability among different categories.

[0009] Therefore, how to integrate the frequency features of images with linguistic semantic guidance to achieve more robust and transferable contraband identification in complex backgrounds has become a key problem that urgently needs to be solved in the field of few-shot identification. Summary of the Invention

[0010] The purpose of this invention is to provide a few-sample contraband image recognition method that integrates language and frequency priors, so as to solve the problems mentioned in the background art.

[0011] To achieve the above objectives, the present invention aims to provide a method for identifying contraband images using a few samples, which integrates language and frequency priors, comprising the following steps:

[0012] S1. Text prompt generation: Enter the category of prohibited items. and scene context Generate structured prompt text from large language models. And extract the cue embedding vector through a text embedding encoder. ;

[0013] S2, Image Embedding Extraction: Extracting the input image... Input visual encoder to extract global embedding vector and regional embedded sets ;

[0014] S3. Frequency masking processing: For the input image... The spectrum is obtained by performing a Fourier transform. Based on text embedding Generate dynamic frequency mask Image reconstructed after filtering frequency domain components ,in, These are the horizontal and vertical frequency indices in the frequency domain; This represents the complex spectrum value at that point; This represents the text prompt embedding vector generated by the language model, used to guide frequency selection; This represents the frequency mask generation network;

[0015] S4. Multi-granularity image-text alignment: Calculate the global embedding vector of the image. With text embedding cosine similarity And optimize global alignment loss. Simultaneously calculate the attention weights of region-level embeddings and text tokens. And optimize region alignment loss ;

[0016] S5. Multi-level contrastive learning: Jointly optimizing the graph-text triplet loss. Support set contrast loss and frequency consistency loss ;

[0017] S6. Dynamic Loss Weight Adjustment: Based on Text Hint Entropy Visual differences between categories Dynamically adjust the weights of each sub-loss Calculate the total loss .

[0018] As a further improvement to this technical solution, the language model in step S1 is GPT-4, and the text embedding encoder adopts CLIP text encoder.

[0019] As a further improvement to this technical solution, the frequency mask generation network in step S3... It contains three 3×3 convolutional layers and a sigmoid activation layer, and outputs a mask value. .

[0020] As a further improvement to this technical solution, the method for calculating the region alignment loss in step S4 is as follows:

[0021]

[0022] in, For image patch index, a total of indivual, Indexed by text token, total indivual, For the first Embedding of an image patch For the first Embedding of each text token, attention weight .

[0023] As a further improvement to this technical solution, the frequency consistency loss in step S5 is defined as:

[0024]

[0025] in, It is a fully connected projection head for 1-3 layers.

[0026] As a further improvement to this technical solution, the triplet loss in step S5 is defined as:

[0027]

[0028] in, Cosine distance This represents the boundary value, ranging from 0.2 to 1.0, and controls the distance between positive and negative samples. It is a Hinge function that ensures the loss is non-negative.

[0029] As a further improvement to this technical solution, the dynamic weight calculation method in step S6 is as follows:

[0030]

[0031] in, For text entropy, Here, the intra-class visual variance is denoted as , where .

[0032] As a further improvement to this technical solution, the image encoder in step S2 adopts a ViT or ResNet architecture, and the region division strategy is uniform grid segmentation.

[0033] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0034] 1. In this invention, a structured language guidance mechanism is set up, and GPT-4 is used to generate text prompts that are strongly related to the category of prohibited items. Semantic priors are embedded into the image feature extraction process, which significantly enhances the model's ability to understand the semantics of scarce samples. At the same time, by setting up a frequency masking module for text guidance, frequency domain masks are dynamically generated based on language prompts, which effectively suppresses high-frequency noise and low-frequency redundant backgrounds in the image that are irrelevant to the task. Furthermore, by setting up a multi-granularity image-text alignment architecture, combined with global feature alignment and regional semantic attention mapping, fine matching at the pixel-phrase level is achieved.

[0035] 2. In this invention, by setting a dynamic loss weight adjustment mechanism, the loss function weight is adaptively adjusted according to the text prompt entropy and intra-class visual differences, which significantly improves the stability of the model under complex sample distribution. At the same time, by setting frequency consistency constraints, the feature consistency between the original image and the frequency domain reconstructed image in the projection space is forced, which enhances the model's tolerance to spectral perturbations.

[0036] 3. In this invention, by designing a support set contrast loss, a compact intra-class feature space is constructed using support set samples from few-shot tasks. Combined with triplet loss, inter-class separability is enhanced. At the same time, by fusing language-frequency-visual multimodal information end-to-end, a unified optimization framework is constructed, which significantly reduces the dependence on labeled data while ensuring real-time performance. Attached Figure Description

[0037] Figure 1 This is a schematic diagram of the overall system architecture of the present invention.

[0038] Figure 2 This is a detailed diagram illustrating the frequency masking mechanism of the present invention.

[0039] Figure 3This is a flowchart of the dynamic loss adjustment and training process of the present invention. Detailed Implementation

[0040] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0041] In one specific embodiment, such as Figures 1-3 As shown, this invention provides a few-sample contraband image recognition method that integrates language and frequency priors, comprising the following steps:

[0042] Step 1: Construct a text prompt module: Based on the category of the prohibited item to be identified and its context, call a large language model to generate structured prompt text and extract its embedding vector;

[0043] The second step is to construct an image embedding module: input the image to be recognized into the visual-language model and extract the global semantic embedding and region-level embedding vectors of the image;

[0044] The third step is to construct a frequency masking module: perform a Fourier transform on the image and generate a frequency mask based on the text prompt embedding vector to mask redundant or interfering frequency components in the frequency domain and reconstruct the frequency domain masked image.

[0045] Step 4: Execute the multi-granularity image-text alignment module: calculate the semantic attention weight between image region embedding and text token, and perform global and regional image-text alignment;

[0046] Step 5: Construct a multi-level contrastive learning module: jointly optimize the model using image-text triplet loss, text support set loss, and frequency consistency loss;

[0047] Step 6: Dynamically adjust the weights of each sub-loss in the loss function based on the text prompt entropy and visual differences of category samples to improve the robustness and generalization ability of the model.

[0048] The core steps of this invention are as follows:

[0049] Step 1: Generate prior text prompts. Input the category of prohibited items. Scene context Generate structured prompt text using GPT-4: Text embedding vector: .

[0050] The second step is to generate image embedding vectors.

[0051] Image embedding vector It is extracted through an image encoder in a vision-language model. Specifically, the input image is... The input is fed into the model's visual encoder branch (such as the ViT or ResNet module in CLIP) to obtain its global semantic vector representation:

[0052]

[0053] in, Represents an image coding network. These are vectors in the embedding space. If the image is divided into multiple patches, then a set of region-level embeddings can be further obtained:

[0054]

[0055] Step 3: Frequency prior masking generation.

[0056] For images (Indicates the input image size is) Fourier transform of (including RGB three channels): Text-guided dynamic frequency mask: Frequency domain masking image: .

[0057] in, These are the horizontal and vertical frequency indices in the frequency domain; This represents the complex spectrum value at that point; This represents the text prompt embedding vector generated by the language model, used to guide frequency selection; This represents a frequency mask generation network (a lightweight convolutional network, consisting of 1) an input layer, 2) three 3×3 convolutional layers, and 3) a sigmoid activation layer), with the following parameters: The input consists of image frequency components and text prompt embeddings, and the output is the frequency domain weights. Indicates the frequency position Above, a masking factor is generated based on image frequency values ​​and language cues: , represents the preservation intensity, 1 means preservation, 0 means complete masking; It is the Sigmoid activation function, which restricts the output to between 0 and 1. Indicates frequency mask It is applied to the original spectrum values ​​to achieve frequency domain information masking. This represents the masked frequency domain image. The masked image is restored using inverse Fourier transform. .

[0058] In the frequency prior masking generation module described above, the symbols in the formula are defined as follows: Input image The frequency domain representation is obtained by Fourier transforming an RGB color image. Hint embeddings generated using large language models Through frequency mask generator Predict the retained weights for each frequency point, and output the frequency mask after passing the Sigmoid activation function. This mask is used to filter out frequency components useful for the task, producing the spectrum of the frequency-masked image. Finally, the masked image is restored to the spatial domain through inverse Fourier transform. .

[0059] Step 4: Align text and images at multiple granularities.

[0060] Image and text feature cosine similarity:

[0061]

[0062] Global alignment loss:

[0063]

[0064] Region structure alignment loss:

[0065]

[0066] in:

[0067] Image patch index, total indivual;

[0068] Text token index, total indivual;

[0069] Image patch With text token Semantic attention scores between them;

[0070] : No. Embedding of an image patch;

[0071] : No. Embedded text tokens.

[0072] Specific calculations:

[0073]

[0074] This is a typical method for calculating attention weights using a dot product. After normalization, it can represent the attention each image patch gives to different cue tokens. Dot-product or cosine can be chosen as the similarity kernel.

[0075] Step 5: Multi-level contrastive loss learning.

[0076] Triple loss:

[0077]

[0078] This loss function is used to compare the query image I with similar images. Closer to outlier images The distance is greater, and the spacing is at least [missing information]. .

[0079] in, Indicates the embedding of the query image. This represents the image embedding for frequency domain reconstruction. Represents the embedding vector of a positive sample image. Represents the embedding vector of the negative sample image; Represent the distance function (using cosine distance). This represents the boundary value (usually 0.2-1.0, controlling the distance between positive and negative samples). It is a Hinge function that ensures the loss is non-negative.

[0080] Contrast loss of text support set:

[0081]

[0082] This indicates support for the embedded positive samples in the set, and... Similar support set samples, This indicates the embedding of the j-th sample in the support set (including all positive and negative samples in the support set). The temperature coefficient, typically 0.05-0.5, controls the steepness of the distribution; K is the total number of samples in the support set, set for few-shot tasks (e.g., K=5 for 5-way 1-shot). This loss function is used to improve generalization ability with few samples by utilizing the relationships between the support set samples.

[0083] Frequency consistency loss:

[0084]

[0085] in, It is a projection head, a small neural network (usually 1-3 fully connected layers) that maps image embedding vectors to a feature space more suitable for contrastive learning; It is the L2 norm (Euclidean distance). This loss function is used to constrain the semantic invariance of the frequency domain transform, thereby enhancing robustness to interference.

[0086] Step 6: Dynamically adjust the loss weight.

[0087] definition:

[0088] : The weight of the k-th sub-loss.

[0089] - Balance coefficient (set based on task priors (η=0.6 indicates greater reliance on text priors)).

[0090] Text prompt entropy.

[0091] - Visual differences in category c.

[0092] For text prompt entropy, Visual differences between category samples.

[0093] Text prompt entropy Calculation method

[0094] Definition Explanation:

[0095] Text prompt entropy measures the semantic complexity or information content of prompts generated by a language model and is used to dynamically adjust the weight allocation in the loss function.

[0096] Step-by-step instructions:

[0097] For the prompt text Use a language model (such as GPT) to calculate the probability distribution for each token;

[0098] Obtain the token probability distribution Then, the standard Shannon entropy definition is used:

[0099]

[0100] Optional alternatives:

[0101] If it is difficult to obtain token-level probabilities, the cosine distance variance between token word vectors can be used as an approximate indicator of entropy to measure token diversity.

[0102] Extract the prompt text The word vector of each token ;

[0103] Calculate the cosine similarity matrix of all token pairs. ;

[0104] Approximate value of entropy:

[0105]

[0106] (Low variance → high similarity → low semantic complexity)

[0107] Visual differences between category samples The calculation method.

[0108] Definition Explanation:

[0109] This metric measures the visual distribution differences among samples within a category of the support set. It is used to control the weights of the loss function, reflecting whether the category is visually consistent (high learnability) or complex (requiring more guidance).

[0110] Calculation formula:

[0111] For category Supports centralized inclusion Zhang image, its embedding is The definition is as follows:

[0112]

[0113] It can also be replaced by the cumulative contribution rate of the principal component distribution of PCA;

[0114] Alternatively, Mahalanobis distance can be used instead of Euclidean distance to enhance class robustness.

[0115] Final total loss: .

[0116] 1. Training process.

[0117] Data preparation: Few-shot task format: N-way K-shot (N classes, K samples per class).

[0118] Data batch structure:

[0119] Support set: (support_images, support_classes, support_contexts);

[0120] Query set: (query_images, query_classes, query_contexts);

[0121] Negative sample set: (neg_images, neg_classes, neg_contexts).

[0122] Note: The support set provides a small number of reference samples for each category, used to build category prototypes, serve as a basis for classification, provide visual context for text prompt generation, and support generalization ability in few-shot learning. The query set represents the set of target samples to be identified by the model, used to simulate images to be classified in real-world scenes, calculate model prediction results (predicted categories), and participate in loss calculation; the negative sample set provides interference samples that are different from the query category, used as negative examples in contrastive learning, forcing the model to distinguish similar foreign objects and enhancing the model's robustness to background interference.

[0123] Data concatenation: The three sets of data are concatenated along the batch dimension and then input into the model.

[0124] 2. Loss function design.

[0125] Four types of loss functions for collaborative optimization are implemented:

[0126] Loss type Calculation method Optimization Objective Global alignment loss Querying image features vs. similarity matrix + cross-entropy for text embedding support sets Cross-modal feature alignment Triple loss Anchor point (query) - positive example (same type support) - negative example (different type) distance calculation + boundary constraints Intra-class compactness / Inter-class separability Support set contrast loss Querying image features vs. supporting similarity matrix + cross-entropy of image features Visual feature consistency Frequency consistency loss Projection features (MSE) of the original image vs. the masked image Spectral perturbation robustness

[0127] 3. Dynamic weight adjustment mechanism.

[0128] Automatically adjust the loss weights based on the current state of the model:

[0129] #Text complexity metric

[0130] text_probs=F.softmax(text_emb)

[0131] H_T=-Σ(text_probs*log(text_probs)) # Text embedding entropy

[0132] #Measurement of visual differences

[0133] class_means=[mean values ​​of features within the same class]

[0134] Var_c = Average distance from features to class centers

[0135] #Dynamic Weight Calculation

[0136] weights={

[0137] 'align': η*H_T+(1-η)*Var_c, # Enhance alignment when text is complex or visually distinct

[0138] 'triplet':1.0-Var_c, #Reducing constraints when intra-class differences are large

[0139] 'contrast': 0.5 + 0.5 * H_T, # Enhance contrast when dealing with complex text

[0140] 'freq_consistency':1.0 # Fixed weight

[0141] }

[0142] 4. Optimization process.

[0143] Calculate the weighted total loss: total_loss = Σ(weights[k] * losses[k])

[0144] Backpropagation gradient calculation

[0145] Gradient clipping (maximum norm 1.0)

[0146] AdamW optimizer parameter updates

[0147] Training Management

[0148] Iterative control:

[0149] Multiple training rounds (num_epochs)

[0150] Regularly log out (loss value / validation metrics)

[0151] Model saving:

[0152] Best validation accuracy model (best_model.pth)

[0153] Finally, train the complete model (final_model.pth).

[0154] Verification process:

[0155] The few-shot classification accuracy is evaluated after each epoch.

[0156] Use an independent validation set (val_loader).

[0157] The modules of this invention are as follows:

[0158] Module 1: Text Prompt Generation Module.

[0159] Function Description:

[0160] This module uses large language models (such as GPT-4) to generate structured prompts related to the category of contraband and the context, and encodes them into vector representations that can be used for graphic alignment and frequency guidance.

[0161] Algorithm steps:

[0162] Enter category labels and context description ;

[0163] Input to language model Generate text prompts:

[0164]

[0165] Use language embedding encoder Obtain the hint embedding vector:

[0166]

[0167] Module 2: Image Embedding and Extraction Module.

[0168] Function Description:

[0169] The image encoder in the vision-language model is used to extract global and local semantic features of the image, including whole-image semantic representation and patch-level embedding.

[0170] Algorithm steps:

[0171] Image I Input Image Encoder ;

[0172] Obtain the global embedding vector:

[0173]

[0174] If the image is divided into One area patch:

[0175]

[0176] Module 3: Frequency Prior Masking Module.

[0177] Function Description:

[0178] The frequency domain information of the image is extracted by Fourier transform, and the frequency mask is constructed by text prompts (the innovation point) to mask inefficient or redundant parts of the spectrum and enhance the structurally significant regions.

[0179] Algorithm formula:

[0180] Image Fourier Transform:

[0181]

[0182] in, These are the horizontal and vertical frequency indices in the frequency domain; This represents the complex spectrum value at that point. Text-guided dynamic frequency masking (innovation):

[0183] Constructing a frequency mask generation network Combining frequency points with text prompts:

[0184]

[0185] Frequency filtering and image reconstruction:

[0186]

[0187] in, These are the horizontal and vertical frequency indices in the frequency domain; This represents the complex spectrum value at that point; This represents the text prompt embedding vector generated by the language model, used to guide frequency selection; This represents the frequency mask generation network, with parameters as follows: The input consists of image frequency components and text prompt embeddings, and the output is the frequency domain weights. Indicates the frequency position Above, a masking factor is generated based on image frequency values ​​and language cues: , represents the preservation intensity, 1 indicates preservation, 0 indicates complete masking; It is the Sigmoid activation function, which restricts the output to between 0 and 1. Indicates frequency mask It is applied to the original spectrum values ​​to achieve frequency domain information masking. This represents the masked frequency domain image. The masked image is restored using inverse Fourier transform. .

[0188] Module 4: Multi-granularity image and text alignment module.

[0189] Function Description:

[0190] Deep semantic alignment between images and text is achieved through global alignment and region structure alignment, thereby improving the ability to recognize fine-grained targets.

[0191] Algorithm formula:

[0192] Image and text feature cosine similarity:

[0193] Global alignment loss:

[0194]

[0195] Region structure alignment: - Dot product attention score:

[0196]

[0197] - Local alignment loss:

[0198] (Innovation)

[0199] in:

[0200] Image patch index, total indivual;

[0201] Text token index, total indivual;

[0202] Image patch With text token Semantic attention scores between them;

[0203] : No. Embedding of an image patch;

[0204] : No. Embedded text tokens.

[0205] Specific calculations:

[0206]

[0207] This is a typical method for calculating attention weights using a dot product. After normalization, it can represent the attention each image patch gives to different cue tokens. Dot-product or cosine can be chosen as the similarity kernel.

[0208] Module 5: Multi-level Comparative Learning Module.

[0209] Function Description:

[0210] We designed a multi-dimensional contrast loss to optimize the feature space from three aspects: image-text pairing, language consistency, and frequency perturbation invariance.

[0211] Algorithm Formula

[0212] Image and text triplet loss (triplet) :

[0213]

[0214] This loss function is used to compare the query image I with similar images. Closer to outlier images The distance is greater, and the spacing is at least [missing information]. .

[0215] in, Indicates the embedding of the query image. This represents the image embedding for frequency domain reconstruction. Represents the embedding vector of a positive sample image. Represents the embedding vector of the negative sample image; Represent the distance function (using cosine distance). This represents the boundary value (usually 0.2-1.0, controlling the distance between positive and negative samples). It is a Hinge function that ensures the loss is non-negative.

[0216] 2. Supports set-based language contrast loss (intra-class compactness):

[0217]

[0218] This indicates support for the embedded positive samples in the set, and... Similar support set samples, This indicates the embedding of the j-th sample in the support set (including all positive and negative samples in the support set). The temperature coefficient, typically 0.05-0.5, controls the steepness of the distribution; K is the total number of samples in the support set, set for few-shot tasks (e.g., K=5 for 5-way 1-shot). This loss function is used to improve generalization ability with few samples by leveraging the relationships between support set samples.

[0219] 3. Frequency Consistency Loss (Innovation):

[0220]

[0221] in, It is a projection head, a small neural network (usually 1-3 fully connected layers) that maps image embedding vectors to a feature space more suitable for contrastive learning; It is the L2 norm (Euclidean distance). This loss function is used to constrain the semantic invariance of the frequency domain transform, thereby enhancing robustness to interference.

[0222] Module Six: Dynamic Adjustment Module for Loss Weights.

[0223] Function Description:

[0224] The weights of each item in the loss function are dynamically adjusted based on the complexity of the prompt text and the degree of intra-class differences in the images, thereby achieving more stable training.

[0225] Algorithm formula:

[0226] 1. Text Hint Entropy: Text hint entropy measures the semantic complexity or information content of hints generated by the language model, and is used to dynamically adjust the weight allocation in the loss function. Steps explained:

[0227] For the prompt text Use a language model (such as GPT) to calculate the probability distribution of each token;

[0228] Probability distribution of obtaining tokens Then, the standard Shannon entropy definition is used:

[0229]

[0230] Optional alternative: If it is difficult to obtain token-level probabilities, the cosine distance variance between token word vectors can be used as an approximate indicator of entropy to measure token diversity.

[0231] Visual differences between category samples:

[0232] Visual differences between category samples The calculation method is defined as follows: This metric measures the visual distribution differences among samples within a certain category in the support set. It is used to control the weights of the loss function, reflecting whether the category is visually consistent (high learnability) or complex (requiring more guidance).

[0233] If the probability distribution is unavailable If so, then the cosine distance variance approximation is used.

[0234] For category Supports centralized inclusion Zhang image, its embedding is The definition is as follows:

[0235]

[0236] 3. Final dynamic loss combination:

[0237]

[0238] in ( () is a learnable function or lookup table strategy that is dynamically adjusted according to complexity.

[0239] definition:

[0240] For text prompt entropy, Visual differences between category samples.

[0241] The foregoing has shown and described the basic principles, main features, and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited to the above embodiments. The embodiments and descriptions in the specification are merely preferred examples and are not intended to limit the invention. Various changes and modifications can be made to the invention without departing from its spirit and scope, and all such changes and modifications fall within the scope of the present invention as claimed. The scope of protection of the present invention is defined by the appended claims and their equivalents.

Claims

1. A method for identifying contraband images using a few samples, integrating language and frequency priors, characterized in that, Includes the following steps: S1. Text prompt generation: Enter the category of prohibited items. and scene context Generate structured prompt text from large language models. And extract the cue embedding vector through a text embedding encoder. ; S2, Image Embedding Extraction: Extracting the input image... Input visual encoder to extract global embedding vector and regional embedded sets ; S3. Frequency masking processing: For the input image... The spectrum is obtained by performing a Fourier transform. Based on text embedding Generate dynamic frequency mask Image reconstructed after filtering frequency domain components ,in, These are the horizontal and vertical frequency indices in the frequency domain; This represents the complex spectrum value at that point; This represents the text prompt embedding vector generated by the language model, used to guide frequency selection; This represents the frequency mask generation network; S4. Multi-granularity image-text alignment: Calculate the global embedding vector of the image. With text embedding cosine similarity And optimize global alignment loss. , Simultaneously calculate the attention weights of region-level embeddings and text tokens. And optimize region alignment loss Support set contrast loss and frequency consistency loss ;in, ； in, For image patch index, a total of indivual, Indexed by text token, total indivual, For the first Embedding of an image patch For the first Embedding of each text token, attention weight ; ， This indicates support for the embedded positive samples in the set, and... Similar support set samples, This indicates that the embedding of the j-th sample in the set is supported. Represents the temperature coefficient; K is the total number of samples in the support set; Frequency consistency loss: ； in, It's ProjectionHead, a small neural network; It is an L2 norm; S5. Multi-level contrastive learning: Jointly optimizing the graph-text triplet loss. , ,in, Indicates the embedding of the query image. This represents the image embedding for frequency domain reconstruction. Represents the embedding vector of a positive sample image. Represents the embedding vector of the negative sample image; Represents the distance function. Indicates boundary values, It is a Hinge function, which ensures that the loss is non-negative; S6. Dynamic Loss Weight Adjustment: Based on Text Hint Entropy Visual differences between categories Dynamically adjust the weights of each sub-loss Calculate the total loss ; ； The weight of the k-th sub-loss. Balance coefficient Text prompt entropy, Visual differences in category c Calculation method: For the prompt text Calculate the probability distribution of each token using a language model; Obtain the token probability distribution Then, the standard Shannon entropy definition is used: ； Calculation method: For categories Supports centralized inclusion Zhang image, its embedding is The definition is as follows: 。 2. The method for few-sample contraband image recognition based on the fusion of language and frequency priors as described in claim 1, characterized in that, The language model in step S1 is GPT-4, and the text embedding encoder uses CLIP text encoder.

3. The method for few-sample contraband image recognition based on the fusion of language and frequency priors according to claim 1, characterized in that, The frequency mask generation network in step S3 It contains three 3×3 convolutional layers and a sigmoid activation layer, and outputs a mask value. .

4. The method for few-sample contraband image recognition based on the fusion of language and frequency priors according to claim 1, characterized in that, The method for calculating the region alignment loss in step S4 is as follows: ， in, For image patch index, a total of indivual, Indexed by text token, total indivual, For the first Embedding of an image patch For the first Embedding of each text token, attention weight .

5. The method for few-sample contraband image recognition based on the fusion of language and frequency priors according to claim 1, characterized in that, The frequency consistency loss in step S5 is defined as follows: ， in, It is a fully connected projection head for 1-3 layers.

6. The method for few-sample contraband image recognition based on the fusion of language and frequency priors according to claim 1, characterized in that, The triplet loss in step S5 is defined as follows: ， in, Cosine distance This represents the boundary value, ranging from 0.2 to 1.0, and controls the distance between positive and negative samples. It is a Hinge function that ensures the loss is non-negative.

7. The method for few-sample contraband image recognition based on the fusion of language and frequency priors according to claim 1, characterized in that, The dynamic weight calculation method in step S6 is as follows: ， in, For text entropy, Here, the intra-class visual variance is denoted as , where .

8. The method for few-sample contraband image recognition based on the fusion of language and frequency priors according to claim 1, characterized in that, The image encoder in step S2 adopts a ViT or ResNet architecture, and the region partitioning strategy is uniform grid segmentation.

Images