Small sample X-ray contraband detection method based on text matching and feature decoupling

Abstract

The invention relates to the technical field of X-ray contraband detection, in particular to a small sample X-ray contraband detection method based on text matching and feature decoupling. In the training process, explicit features and implicit features of the contraband are separated through an overlapping feature interaction module, dynamic interaction of double-flow features is established through a cross attention mechanism, cross-modal similarity calculation is carried out on query vectors of the explicit features and key vectors of the implicit features, potential discrimination clues contained in implicit space are mined, and the identification accuracy of the contraband is improved. Overlapping feature decoupling and detail information supplementation are realized; extra text priori knowledge is introduced through a visual text matching module, the risk that classifier boundaries are difficult to distinguish and model fine adjustment is prone to over-fitting is relieved, inter-class discrimination and intra-class compactness of contraband features are gradually enhanced through multi-layer feature transformation, alignment of visual features and text information is achieved, and the recognition accuracy of the contraband features is improved. And the discriminative features of the types of the prohibited goods are enhanced. According to the invention, the precision of X-ray contraband detection and the detection performance of the model are improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of X-ray contraband detection, and particularly relates to a small sample X-ray contraband detection method based on text matching and feature decoupling. BACKGROUND

[0002] As an essential security and protection device in transportation hubs such as airports, railway stations, subway stations and bus stations, X-ray security inspection machines are an important line of defense for passenger safety. X-ray security inspection machines emit X-rays to penetrate the luggage carried by passengers, reconstruct corresponding X-ray images according to the material of the penetrated objects, and security personnel observe the image content by naked eye to determine whether there are contraband such as knives, guns, ammunition, explosives, toxic substances and radioactive substances. The above-mentioned manual detection of contraband is easily affected by human factors such as the working experience, professional level and mental state of the security personnel, which increases the rate of missed detection or false detection of contraband, thereby causing potential safety risks, so it is urgent to realize intelligent detection of contraband.

[0003] Unlike RGB images formed by light reflection on the surface of an object, X-ray images are formed by X-ray penetration of luggage. Due to the different degrees of X-ray energy attenuation caused by objects with different structures and densities, combined with the randomness of object placement and the mutual overlap of various objects, X-ray images of contraband targets have complex appearance characteristics such as missing color / texture information, large attitude changes and region overlap, making it difficult to extract high-quality contraband representations, so contraband detection is challenging.

[0004] To address the above problems, current X-ray image contraband detection methods usually focus on designing feature enhancement strategies, but ignore the importance of high-quality backbone feature extraction for detection, and there are few related literature reports on feature enhancement from the perspective of feature decoupling. For example Figure 1 As shown in (a), the contraband knife (labeled Knife in the database) is visible in the X-ray image, but the contraband is completely obscured and invisible in the RGB image. As shown in (b), the region that is obscured in the RGB image but visible in the X-ray image (the blue box region in the figure) has a strong correlation with the visible region in the RGB image (the Reference region in the figure). Figure 1

[0005] ​Due to the low frequency of contraband, it is difficult to collect a large number of contraband image of the corresponding category, and the lack of data makes the training process of the X-ray image contraband detection model vulnerable to overfitting. The long-tail distribution problem of contraband data leads to: 1) The features of new categories are easily represented by the features of basic categories, making it difficult for the classifier to distinguish the boundaries; 2) The data of new categories is too little to fully represent its distribution, and model fine-tuning is prone to overfitting. At present, the method of introducing cross-modal semantic information realizes the intra-class feature aggregation and inter-class distribution decoupling through visual prototypes, as shown in Figure 1 (d). However, the modal difference between visual features and text features leads to deviation in feature aggregation of text prototypes. In addition, the existing multi-modal method uses CLIP text semantics to guide sparse encoder reconstruction of images to enhance sample diversity, as shown in Figure 1 (c). But building an additional encoder will consume additional training resources. On this basis, as shown in Figure 1 (e), the generation method based on Variational Autoencoder (VAE) uses the text embedding of CLIP as a conditional variable to generate new category features. But the text description of X-ray images is completely different from that of RGB images, and the CLIP model trained according to ImageNet cannot accurately generate the features of contraband in X-ray images. SUMMARY

[0006] In view of the above problems of the prior art, the technical problem to be solved by the present application is to provide a small sample X-ray contraband detection method based on text matching and feature decoupling, which realizes overlapping feature decoupling and supplementary of detailed information through an overlapping feature interaction module, and introduces additional text priori knowledge through a visual text matching module and realizes the alignment of visual features and text information, thereby improving the precision of X-ray contraband detection and the detection performance of the model.

[0007] In order to solve the above technical problems, the technical scheme adopted by the present application is as follows:

[0008] The small sample X-ray contraband detection method based on text matching and feature decoupling comprises:

[0009] S1: obtaining an X-ray image to be detected;

[0010] S2: inputting the X-ray image to be detected into a trained contraband detection model to output a prediction result of the contraband category and the bounding box;

[0011] The processing steps for training the contraband detection model comprise:

[0012] S201: obtaining X-ray images and the true results of contraband categories and bounding boxes as training data as model input;

[0013] S202: performing feature extraction on the X-ray image through the backbone network to obtain backbone feature maps of multiple scales;

[0014] S203: enhancing the backbone feature maps of multiple scales through an overlapping feature interaction module to obtain enhanced feature maps of multiple scales; wherein the overlapping feature interaction module uses a cross-attention mechanism to establish dynamic interaction of double-flow features, performs cross-modal similarity calculation on a query vector of explicit features and a key vector of implicit features, mines potential discriminative clues contained in the implicit space, and realizes complementary enhancement of the explicit features;

[0015] S204: performing region of interest feature extraction on the enhanced feature maps of multiple scales through an RoI head to obtain a region of interest feature map;

[0016] S205: calculating a prohibited item category prediction label and a bounding box prediction position as prediction results based on the region of interest feature map through a classification head and a regression head;

[0017] S206: realizing alignment of visual features and text descriptions based on the region of interest feature map and introduced text priori knowledge through a visual-text matching module to obtain an aligned prediction result;

[0018] S207: calculating a loss function based on the aligned prediction result and the prediction results and real results of the prohibited item category and the bounding box, and reversely optimizing parameters of the prohibited item detection model;

[0019] S208: repeating steps S201 to S207 to iteratively train the prohibited item detection model until the model converges or a training termination condition is reached;

[0020] S3: taking the prediction results of the prohibited item category and the bounding box output by the prohibited item detection model as prohibited item detection results of the X-ray image to be detected.

[0021] Preferably, in step S2, a pre-trained ResNet50 is used as the backbone network to perform feature extraction on the X-ray image to obtain backbone feature maps of multiple scales.

[0022] Preferably, in step S203, the processing steps of the overlapping feature interaction module include:

[0023] S2031: obtaining an input backbone feature map F i ;

[0024] S2032: modeling global context information of each channel of the backbone feature map F i using a global average pooling layer, and adaptively generating a channel attention weight map M c through two cascaded nonlinear transformation layers;

[0025] The formula is:

[0026]

[0027] In the formula, GAP represents a global average pooling operation; W1 and W2 represent weight parameters of two linear transformation layers respectively; sigma represents a Sigmoid activation function, and B and C represent a batch number and a channel dimension;

[0028] S2033: The backbone feature map F i is processed by a depth separable convolution structure, and then is element-wise multiplied with the channel attention weight map M c to obtain an enhanced feature map F e ;

[0029] The formula is:

[0030]

[0031] In the formula, ReLU is an activation function; BN is batch normalization; DWC is a depth separable convolution; represents element-wise multiplication, H and W represent the height and width of the feature F e ;

[0032] S2034: The backbone feature map F i is respectively subjected to an average pooling operation and a maximum pooling operation, and the average pooling and maximum pooling results are spliced and mapped by a Sigmoid activation function to generate a significance weight map M s of the prohibited item;

[0033] The formula is:

[0034]

[0035] In the formula, Concat represents a tensor splicing operation; AvgPool represents an average pooling operation; MaxPool represents a maximum pooling operation;

[0036] S2035: The significance weight map M s is element-wise multiplied with the enhanced feature map F e to obtain an explicit feature map F o ; and after the significance weight map M s is subjected to an inversion operation, the enhanced feature map F e is element-wise multiplied with the enhanced feature map F l ;

[0037] S2036: An explicit feature map F o and an implicit feature map F lhigh correlation between the features of the explicit feature map F e and the implicit feature map F e and the explicit feature map F o to obtain the enhanced feature map F c after information complementation.

[0038] S2037: Extracting the high-frequency component of the backbone feature map F i through a high-pass filter to obtain the high-frequency feature map F h .

[0039] S2038: Element-wise adding the enhanced feature map F c after information complementation and the high-frequency feature map F h to obtain the final enhanced feature map F

[0040] The formula is as follows:

[0041]

[0042] Preferably, the processing step of step S2036 comprises:

[0043] S20361: Extracting the query vector Q o , the key vector K o and the value vector V o from the explicit feature map F o through a 1x1 convolution layer.

[0044] S20362: Extracting the query vector Q l and the key vector K l from the implicit feature map F l through a 1x1 convolution layer.

[0045] S20363: According to the context awareness of the explicit feature to the implicit feature and the guidance of the implicit feature to the explicit feature, constructing the interaction weights A o and A l .

[0046] The formula is as follows:

[0047]

[0048] In the formula, d o = 256 is the dimension of the key vector K l ; d l = 256 is the dimension of the key vector K o .

[0049] S20364: Element-wise adding the interaction weights A o and A lAn element-wise multiplication is performed to obtain the final interaction weight map A e ;

[0050] The formula is as follows:

[0051] A e = A o ⊙ A l ;

[0052] S20365: After multiplying the interaction weight map A e and the value vector V o , an element-wise addition is performed with the explicit feature map F o to obtain the enhanced feature map F c after information complementation.;

[0053] The formula is as follows:

[0054]

[0055] In the formula: represents an element-wise addition.

[0056] Preferably, in step S2037, the high-pass filter is constructed by the difference between the unit convolution kernel E and the low-pass filter;

[0057] The formula of the high-frequency feature map F h is as follows:

[0058]

[0059] In the formula: represents a convolution operation; Conv represents a convolution layer with a convolution kernel size of 3x3.

[0060] Preferably, in step S206, the processing steps of the visual text matching module include:

[0061] S2061: The CLIP text encoder and the CLIP visual encoder with frozen weights are used to calculate the corresponding text feature vector and the visual feature vector for each type of X-ray image in advance, and store them in a buffer;

[0062] S2062: The region of interest feature map F roi is obtained.;

[0063] S2063: The text feature vector and the visual feature vector are obtained from the buffer, and the text feature vector and the visual feature vector are taken as the value vector V T and the key vector KF Meanwhile, the region-of-interest feature map F roi As a query, the feature matching is realized by the key-value vector;

[0064] S2064: The region-of-interest feature map F roi In the input Adapter, the adaptive feature F a is obtained; wherein the Adapter comprises three cascaded trainable linear layers, and a Relu activation function is arranged between adjacent two linear layers;

[0065] The formula is expressed as:

[0066] F a = clamp(W5·ReLu(W4·ReLu(W3·F roi ), -5, 5);

[0067] In the formula: clamp(·) represents a function of limiting the numerical range of a tensor; W3, W4, W5 represent the training weights of the three linear layers; ReLu(·) is a ReLu activation function;

[0068] S2065: The key vector K F and the value vector V T are multiplied by the adaptive feature F a to obtain the matching result P c ;

[0069] The formula is expressed as:

[0070] P c = exp(-α(1-F a ·(K F ) T ))⊙V T ;

[0071] In the formula: exp() represents an exponential operation; α=0.5 is a hyperparameter; () T represents a transposition operation; and ⊙ represents an element-wise multiplication;

[0072] S2066: The region-of-interest feature map F roi is multiplied by the frozen weights W CLIP of the CLIP text encoder, and then is element-wise multiplied with the matching result P c to obtain the alignment prediction result P t ;

[0073] The formula is expressed as:

[0074]

[0075] In the formula: represents element-wise addition; β = 0.5 is a hyperparameter.

[0076] Preferably, in step S2061, the following steps are specifically included:

[0077] S20611: Obtain the X-ray image of the small sample training set and the text description of the category of the contraband belonging to it;

[0078] S20612: Input the text description of the category of the contraband belonging to the X-ray image of the small sample training set into the CLIP text encoder with frozen weights to extract text features and obtain a text feature vector and the text vector as the value vector V T of the buffer is stored.

[0079] The formula is as follows:

[0080]

[0081] In the formula: represents the i-th text feature vector of the c-th category; TextEncoder(·) represents the CLIP text encoder with frozen weights; I c represents the text description of the c-th category.

[0082] S20613: Input the X-ray image of the small sample training set into the CLIP visual encoder with frozen weights to extract visual features and obtain a visual feature vector and the visual feature vector as the key vector K F of the buffer is stored.

[0083] The formula is as follows:

[0084]

[0085] In the formula: represents the visual feature vector of the i-th X-ray image of the c-th category; Concat(·) represents the concatenation operation along the channel dimension; ||·|| represents the L2 norm; VisualEncoder(·) represents the CLIP visual encoder with frozen weights. represents the background category input image in the small sample training set. represents the i-th X-ray image of the n-th contraband category in the small sample training set.

[0086] Preferably, in step S2, the training of the contraband detection model includes a base class training phase and a new class fine-tuning phase.

[0087] The loss function of the base class training phase is as follows:

[0088] Ltotal = L rpn + L cls + L reg ;

[0089] The visual text matching module is introduced in the new class fine-tuning stage, and the loss function of the new class fine-tuning stage is as follows:

[0090] L total = L rpn + L cls + L reg + L focal ;

[0091] In the formula, L rpn represents the RPN head loss; L cls represents the cross-entropy loss of the classification head; L reg represents the regression loss of the regression head; and L Focal represents the Focal loss calculated based on the alignment prediction result.

[0092] Preferably, the calculation formula of the Focal loss L Focal is as follows:

[0093] L Focal = -ω(1 - P t ) γ log(P t );

[0094] In the formula, ω and γ are hyperparameters; log(·) is a logarithmic operation; and P t represents the alignment prediction result.

[0095] Preferably, the RPN head loss L rpn is calculated as follows: the enhanced feature map is input into the RPN head to obtain the RPN prediction result of the prohibited item category and the bounding box; the cross-entropy loss of the RPN head is calculated based on the prohibited item category prediction label in the RPN prediction result and the prohibited item category true label in the real result; the bounding box regression loss of the RPN is calculated based on the bounding box prediction position in the RPN prediction result and the bounding box true position in the real result; and the cross-entropy loss and the bounding box regression loss of the RPN head are added to obtain the RPN head loss L rpn .

[0096] The cross-entropy loss L cls of the classification head is calculated based on the prohibited item category prediction label in the prediction result and the prohibited item category true label in the real result.

[0097] The regression loss L reg of the regression head is calculated based on the bounding box prediction position in the prediction result and the bounding box true position in the real result.

[0098] Compared with the prior art, the small sample X-ray contraband detection method based on text matching and feature decoupling in the application has the following beneficial effects:

[0099] In order to solve the challenge of overlapping contraband under the condition of limited supervision information, the applicant deeply explores the characteristics of the X-ray image. Unlike the conventional occlusion, the occluded part in the X-ray image is also visible. The applicant demonstrates that there is a strong correlation between the occluded area (occluded but visible) and the non-occluded area (visible area). Based on this characteristic, the application designs an overlapping feature interaction module (OFIM). The overlapping feature interaction module separates the explicit features (non-occluded area) and the implicit features (occluded area) of the contraband through layer-by-layer feature extraction and Sigmoid function, and realizes the cross-modal similarity calculation of the query vector of the explicit feature and the key vector of the implicit feature through the dynamic interaction of the double-flow features by the cross-attention mechanism, mines the potential discriminative clues contained in the implicit space, realizes the complementary enhancement of the explicit features through the feature fusion layer, finally extracts the potential information of the occluded area, realizes the decoupling of the overlapping features and the supplement of the detailed information, and thus improves the precision of X-ray contraband detection.

[0100] In order to solve the problem of domain difference between X-ray images and natural images, the application designs a visual text matching module (VTMM). Combined with Figure 1 (f) shows that the visual text matching module introduces additional text prior knowledge through CLIP (Contrastive Language-Image Pretraining), establishes a knowledge matrix of visual features and corresponding text descriptions in the small sample training set, helps the model to learn the consistency of vision and text, and alleviates the risk of difficult to distinguish the classifier boundary and easy to overfit the model fine-tuning; and uses the Cache stored by the visual text knowledge matrix to build a trainable Adapter, gradually enhances the inter-class discriminativeness and intra-class compactness of the contraband features through multi-layer feature transformation, realizes the alignment of visual features and text information, enhances the discriminative features of the contraband categories, and thus improves the precision of X-ray contraband detection and the detection performance of the model. BRIEF DESCRIPTION OF DRAWINGS

[0101] In order to make the purpose, technical scheme and advantages of the application more clear, the application will be further described in detail below in combination with the drawings, in which:

[0102] Figure 1(a) GT map of the contraband Knife; (b) Manhattan distance between the reference region (valid region not occluded) and the normalized randomly selected region; (c) Textual semantic guided image reconstruction using CLIP; (d) Expanding the decision boundary by visual prototypes; (e) Generating new class features based on the conditional VAE model of semantic information; (f) Constructing the visual-textual adapter to improve the class classification ability of the present method.

[0103] Figure 2 Network structure diagram of the contraband detection model.

[0104] Figure 3 Network structure diagram of the overlapping feature interaction module (OFIM).

[0105] Figure 4 Network structure diagram of the visual-textual matching module (VTMM). DETAILED DESCRIPTION

[0106] In order to make the objects, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, rather than all the embodiments. The components of the embodiments of the present application described and shown in the drawings herein can be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present application provided in the drawings is not intended to limit the scope of the claimed present application, but only represents selected embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative labor fall within the scope of protection of the present application.

[0107] It should be noted that similar reference numerals and letters refer to like items in the accompanying drawings, and once an item is defined in one drawing, it need not be further defined and explained in subsequent drawings. In the description of the application, it needs to be explained that the terms "center", "upper", "lower", "left", "right", "vertical", "horizontal", "inner", "outer", and the like indicate the orientation or positional relationship based on the orientation or positional relationship shown in the drawings, or the orientation or positional relationship when the product of the application is usually placed, and are only for the convenience of describing the application and simplifying the description, and do not indicate or imply that the indicated device or element must have a particular orientation, be constructed and operated in a particular orientation, and therefore cannot be understood as a limitation on the application. In addition, the terms "first", "second", "third", and the like are only used to distinguish the description and cannot be understood as indicating or implying relative importance. In addition, the terms "horizontal", "vertical", and the like do not mean that the parts must be absolutely horizontal or hanging, but can be slightly inclined. For example, "horizontal" only means that its direction is more horizontal relative to "vertical", and does not mean that the structure must be completely horizontal, but can be slightly inclined. In the description of the application, it also needs to be explained that unless otherwise explicitly specified and limited, the terms "arrangement", "installation", "connection", "connection" should be understood broadly, for example, it can be fixedly connected, or it can be detachably connected, or integrally connected; it can be mechanically connected, or it can be electrically connected; it can be directly connected, or it can be indirectly connected through an intermediate medium; it can be the communication between two elements inside. For those skilled in the art, the specific meaning of the above terms in the application can be understood according to the specific circumstances.

[0108] The specific embodiments are further described in detail below:

[0109] Embodiments:

[0110] A small sample X-ray contraband detection method based on text matching and feature decoupling is disclosed in this embodiment.

[0111] The small sample X-ray contraband detection method based on text matching and feature decoupling comprises:

[0112] S1: obtaining an X-ray image to be detected;

[0113] S2: inputting the X-ray image to be detected into a trained contraband detection model to output a prediction result of the contraband category and the bounding box;

[0114] As shown in Figure 2 The processing steps of training the contraband detection model include:

[0115] S201: obtaining an X-ray image as training data and a true result of the contraband category and the bounding box as model input;

[0116] S202: performing feature extraction on the X-ray image through a backbone network to obtain backbone feature maps of multiple scales;

[0117] S203: enhancing the backbone feature maps of multiple scales through an overlapping feature interaction module (OFIM) to obtain enhanced feature maps of multiple scales; wherein the overlapping feature interaction module uses a cross-attention mechanism to establish dynamic interaction of double-flow features, performs cross-modal similarity calculation on query vectors of explicit features and key vectors of implicit features, mines potential discriminative clues contained in the implicit space, and realizes complementary enhancement of the explicit features;

[0118] S204: performing region of interest feature extraction on the enhanced feature maps of multiple scales through a RoI head to obtain region of interest feature maps;

[0119] S205: calculating, based on the region of interest feature maps, a prohibited item category prediction label and a bounding box (center point) prediction position as prediction results through a classification head and a regression head;

[0120] S206: realizing alignment of visual features and text descriptions based on the region of interest feature maps and introduced text priori knowledge through a vision-text matching module (VTMM) to obtain aligned prediction results;

[0121] S207: calculating a loss function based on the aligned prediction results, the prediction results of the prohibited item category and the bounding box, and real results, and reversely optimizing parameters of the prohibited item detection model;

[0122] S208: repeating steps S201 to S207 to iteratively train the prohibited item detection model until the model converges or a training termination condition is reached;

[0123] In this embodiment, the training termination condition can be a preset maximum training round or a maximum iteration number.

[0124] When the prohibited item detection model is trained, the prohibited item category prediction label and the bounding box prediction position output by the classification head and the regression head are taken as outputs of the prohibited item detection model;

[0125] S3: taking the prediction results of the prohibited item category and the bounding box output by the prohibited item detection model as prohibited item detection results of the X-ray image to be detected.

[0126] Currently, the X-ray contraband detection method based on the Transformer still has the limitation of being difficult to effectively separate the overlapping targets when facing the task of contraband detection under complex occlusion scenes. This limitation mainly comes from the following two factors: First, in the data labeling aspect, in order to reduce the labeling cost, the rectangular bounding box labeling method is generally used, which can provide the basic position information of the target, but cannot accurately represent the occlusion relationship between the targets. Secondly, in the algorithm design aspect, the existing methods have two key limitations: 1) The method based on deformable attention mechanism (DAT) guides the model to focus on the overall shape of the target through feature bias, but in the occlusion scene, the bias information often comes from the unoccluded area, which is difficult to effectively solve the feature confusion problem; 2) The method based on self-supervised learning tries to enhance the model's perception of the overall information of the target through shape constraints, but in the scene with serious background interference, the quality of the generated pseudo-labels is often difficult to guarantee. It is worth noting that the commonality of existing methods is that they try to introduce additional target information to constrain model learning, which essentially ensures the integrity of feature representation and enhances the model's attention to occluded features. However, in the contraband detection task, since the objects in the X-ray image usually lack obvious texture information, obtaining accurate strong supervision labeling often requires a large amount of time and labor cost.

[0127] In order to solve the challenge of overlapping contraband under limited supervision information, the applicant deeply explores the characteristics of X-ray images. Unlike conventional occlusion, the occluded part of the X-ray image is also visible. The applicant demonstrates that there is a strong correlation between the occluded area (occluded but visible) and the unoccluded area (visible area). Based on this characteristic, the invention designs an overlapping feature interaction module (OFIM). The overlapping feature interaction module separates the explicit features (unoccluded area) and implicit features (occluded area) of the contraband through layer-by-layer feature extraction and Sigmoid function, and establishes dynamic interaction between the two features through cross-attention mechanism to realize cross-modal similarity calculation between the query vector of explicit features and the key vector of implicit features, mine the potential discriminative clues in the implicit space, realize the complementary enhancement of explicit features through feature fusion layer, finally extract the potential information of the occluded area, realize the decoupling of overlapping features and the supplement of detailed information, and thus improve the accuracy of X-ray contraband detection.

[0128] Currently, in order to deal with the long-tail distribution problem of the X-ray contraband dataset sample, researchers introduce a small sample target detection method based on metrics and generation to construct a detection model with better generalization ability under the condition of limited samples. However, in the actual application of the contraband detection model, due to the lack of texture information and the interference of overlapping objects, the visual features are unreliable, and the contraband feature prototype fitted by the visual features based on the prototype method has deviation. In addition, the generation model used in the small sample detection method based on generation is pre-trained on ImageNet, and the features learned by the generation model are different from the features of the X-ray image. Therefore, directly migrating the pre-trained generation model of the natural image will cause the feature distribution to deviate, and the synthetic samples generated by the generation model not only cannot effectively enhance the contraband feature representation, but also will introduce interference noise, resulting in degradation of the performance of the classifier.

[0129] In order to solve the problem of domain difference between X-ray images and natural images, the present application designs a visual text matching module (VTMM). In combination with Figure 1 (f) As shown, the visual text matching module introduces additional text prior knowledge through CLIP (Contrastive Language-Image Pretraining), establishes a knowledge matrix of visual features and corresponding text descriptions of the small sample training set, helps the model to learn the consistency of visual and text, and alleviates the risk of difficulty in distinguishing the boundaries of the classifier and easy overfitting of the model. And use the Cache stored by the visual text knowledge matrix to build a trainable Adapter, gradually enhance the inter-class discriminativeness and intra-class compactness of the contraband feature through multi-layer feature transformation, realize the alignment of visual features and text information, enhance the discriminative features of the contraband category, thereby improve the precision of X-ray contraband detection and the detection performance of the model.

[0130] The present application sends the aligned feature vector (region of interest feature map) into the detection head and the regression head, accurately locates and classifies the contraband in the X-ray image, and outputs the contraband detection image; the model calculates the boundary box regression loss and the classification loss of the class label respectively, trains and optimizes the parameters of the contraband detection model. Extensive experiments on the FSX dataset show that the present application significantly improves the performance of contraband detection.

[0131] In order to better introduce the technical scheme of the present application, the present embodiment is explained through the following several parts.

[0132] I. Backbone network

[0133] In the present embodiment, a pre-trained ResNet50 is used as the backbone network. For each input X-ray contraband image where H and W represent the height and width of the image, respectively. OFIM uses a pre-trained ResNet50 as a backbone network to extract multi-scale features where B, C, H and W represent the batch number, channel dimension, height and width of the i-th stage feature F i .

[0134] II. Overlapping Feature Interaction Module (OFIM)

[0135] In combination Figure 3 with FIG. 1, the processing steps of the overlapping feature interaction module include:

[0136] S2031: Obtain the input backbone feature map F i .

[0137] S2032: In order to accurately extract the salient features of the prohibited items, the present application uses a global average pooling (GAP) layer to model the global context information of each channel of the backbone feature map F i , and generates a channel attention weight map M c adaptively through two cascaded linear ReLU (LL) layers, thereby enhancing the response of key channels and suppressing irrelevant features.

[0138] The formula is expressed as:

[0139]

[0140] In the formula: GAP represents the global average pooling operation; W1 and W2 represent the weight parameters of the two linear transformation layers, respectively; σ represents the Sigmoid activation function, and B and C represent the batch number and the channel dimension;

[0141] S2033: After processing the backbone feature map F i through a depthwise separable convolution structure (DWC), it is multiplied element by element with the channel attention weight map M c to obtain an enhanced feature map F e ; the present application introduces a depthwise separable convolution structure, which expands the receptive field range and enhances the perception of local features while keeping the spatial resolution of the feature map unchanged.

[0142] The formula is expressed as:

[0143]

[0144] In the formula: ReLU is the activation function; BN is batch normalization; DWC is depthwise separable convolution; ⊙ represents element-wise multiplication, and H and W represent the feature Fe height and width;

[0145] S2034: respectively perform the backbone feature map F i After performing the average pooling and maximum pooling operations, the results of average pooling and maximum pooling are concatenated and mapped with the Sigmoid activation function to generate the saliency weight map M of the contraband. s ; To accurately extract the explicit features F of contraband o and implicit feature F l ,This paper combines average pooling and maximum pooling operations to extract key feature information.

[0146] The formula is:

[0147]

[0148] Where: Concat represents the tensor concatenation operation; AvgPool represents the average pooling operation; MaxPool represents the maximum pooling operation;

[0149] S2035: The significance weight map M s With the enhanced feature map F e Perform element-by-element multiplication to obtain the explicit feature map F o ; Execute the saliency weight map M s After the inversion operation, and the enhanced feature map F e Perform element-by-element multiplication to obtain the implicit feature map F l ;

[0150] S2036: Using the cross attention mechanism, based on the explicit feature map F o and implicit feature map F l Calculate the interaction weight graph A between the high correlation features e , and the interaction weight graph A e and explicit feature map F o Fusion is performed to obtain the enhanced feature map F after information complementation c ;

[0151] S2037: Extract backbone feature map F through (additional) high-pass filter (HPF) i The high-frequency component of the high-frequency feature map F h ;

[0152] S2038: Enhanced feature map F after information complementation c and high-frequency feature map F h Add element by element to get the final enhanced feature map

[0153] The formula is:

[0154]

[0155] 1. Use cross-attention mechanism to achieve information complementarity

[0156] After obtaining the explicit feature F o and implicit feature F l Afterwards, OFIM relies on the high correlation between its features and uses the cross-attention mechanism to achieve information complementarity. The specific processing steps include:

[0157] S20361: Extract from explicit feature map F through 1×1 convolution layer o The query vector Q o , key vector K o Sum value vector V o ;

[0158] S20362: Extract the implicit feature map F through a 1×1 convolutional layer l The query vector Q l and key vector K l ;

[0159] S20363: Construct the interaction weight A based on the contextual perception of implicit features by explicit features and the guidance of explicit features by implicit features o and A l ;

[0160] The formula is:

[0161]

[0162] Where: d o =256 is the key vector K l Dimension; d l =256 is the key vector K o Dimensions;

[0163] S20364: Interaction weight A o and A l Perform element-by-element multiplication to obtain the final interaction weight graph A e ;

[0164] The formula is:

[0165] A e =A o ⊙A l ;

[0166] S20365: Interaction weight graph A e With the value vector V o After multiplication, with the explicit feature map F oAn element-wise addition is performed to obtain the enhanced feature map F after information complementation c ;

[0167] The formula is:

[0168]

[0169] In the formula: represents an element-wise addition.

[0170] 2. High-pass filter

[0171] The contraband detection task often needs features with fine spatial boundary details of the contraband. However, the layer-by-layer convolution operation is prone to cause loss of detail information in the down-sampling process, limiting the model's ability to extract boundary detail information. In the frequency domain, high-frequency components are related to boundary details and object contours. Therefore, OFIM introduces an additional high-pass filter (HPF) to extract the high-frequency components F i of the input feature F h . The high-pass filter is constructed by the difference between the identity kernel E (IK) and the low-pass filter (LPF).

[0172] The formula for the high-frequency feature map F h is:

[0173]

[0174] In the formula: represents a convolution operation; Conv represents a convolution layer with a convolution kernel size of 3x3.

[0175] III. RoU head

[0176] On the basis of the enhanced backbone feature F of OFIM, the enhanced feature F is transmitted to the RoU head to extract the region-of-interest feature F , where B represents the number of F roi , and H and W represent the height and width of F roi . Subsequently, the fully connected layer converts the flattened F roi to a d-dimensional feature space, and outputs the feature vector F

[0177] IV. Visual-text matching module (VTMM)

[0178] The VTMM constructs an Adapter containing three trainable linear layers, and adds a Relu activation function after the first two layers to introduce non-linear changes. Then the region-of-interest feature vector F roi, the more adaptive feature F is outputted by the Adapter a It should be noted that, in order to prevent the mapping value from being too large to affect the subsequent calculation, the application limits the range of the output adaptive feature F a The range is limited using a clipping operation.

[0179] As shown in Figure 4 The processing steps of the visual text matching module include:

[0180] S2061: The corresponding text feature vector and visual feature vector for each category of X-ray image are calculated in advance by the CLIP text encoder and the CLIP visual encoder with frozen weights, and are stored in the cache (key-value cache);

[0181] S2062: Obtain the region of interest feature map F roi ;

[0182] S2063: Obtain the text feature vector and visual feature vector from the cache, and take the text feature vector and visual feature vector as the value vector V T and the key vector K F , respectively, while taking the region of interest feature map F roi as the query, and realize feature matching through the key-value vector;

[0183] S2064: Input the region of interest feature map F roi into the Adapter to obtain the adaptive feature F a ; wherein the Adapter includes three cascaded trainable linear layers, and a Relu activation function (introducing non-linear change) is arranged between adjacent two linear layers;

[0184] The formula is:

[0185] F a = clamp(W5·ReLu(W4·ReLu(W3·F roi )), -5, 5);

[0186] In the formula: clamp(·) represents a function for limiting the numerical range of a tensor; W3, W4, W5 represent the training weights of the three linear layers; ReLu() is the ReLu activation function;

[0187] S2065: Take the key vector K F and the value vector V T and the adaptive feature F aMatrix vector multiplication is performed to obtain the matching result P c ;

[0188] The formula is:

[0189] P c = exp(-a(1-F a ·(K F ) T ))⊙V T ;

[0190] In the formula: exp() represents the exponential operation; a = 0.5 is a hyperparameter; F T represents the transpose operation; and ⊙ represents element-wise multiplication.

[0191] S2066: After multiplying the region of interest feature map F roi and the frozen weights W CLIP of the CLIP text encoder, element-wise multiplication is performed with the matching result P c to obtain the alignment prediction result P t .

[0192] The formula is:

[0193]

[0194] In the formula: represents element-wise addition; and β = 0.5 is a hyperparameter.

[0195] Five, key-value Cache (buffer)

[0196] In this embodiment, the key-value Cache is constructed by the following steps:

[0197] S20611: Obtain an X-ray image of a small sample training set and a text description of the belonging prohibited item category;

[0198] S20612: Unlike the existing simple text prompt method based on a general visual language model such as CLIP (such as a single category label ‘straight knife’ or ‘nail clippers’), the present application enhances the semantic representation of the X-ray image by constructing a semantically rich text description I c (such as ‘an X-ray image revealing a straight knife inside luggage’) to enhance the semantic representation of the X-ray image. The detailed description method can provide more comprehensive semantic context information and establish a more accurate correspondence between visual features and text information.

[0199] Specifically, the text description of the X-ray image belonging to the contraband category in the small sample training set is input into the CLIP text encoder with frozen weights for text feature extraction to obtain a text feature vector And the text vector is stored as the value vector V of the buffer T ;

[0200] The formula is:

[0201]

[0202] In the formula: represents the i-th text feature vector of the c-th category; TextEncoder(·) represents the CLIP text encoder with frozen weights; I c represents the text description of the c-th category;

[0203] S20613: VTMM extracts visual features of the small sample training set D novel through the frozen CLIP visual encoder It is worth noting that in the process of extracting visual features, the extracted background category visual features are individually subjected to L2 norm normalization processing. This is to avoid the interference of the visual features of the background category on the discriminative nature of the contraband category in cross-modal similarity calculation, so the present application imposes a constraint condition on the visual representation of the background category.

[0204] Specifically, the X-ray image of the small sample training set is input into the CLIP visual encoder with frozen weights for visual feature extraction to obtain a visual feature vector And the is stored as the key vector K of the buffer F ;

[0205] The formula is:

[0206]

[0207] In the formula: represents the visual feature vector of the i-th X-ray image of the c-th category; Concat(·) represents the concatenation operation along the channel dimension; ||·|| represents the L2 norm; VisualEncoder(·) represents the CLIP visual encoder with frozen weights; represents the input image of the background category in the small sample training set; represents the i-th X-ray image of the n-th contraband category in the small sample training set (n [1, 20]).

[0208] Six, loss function

[0209] In this embodiment, the training of the contraband detection model includes a base class training stage and a new class fine-tuning stage; only the classification layer and the Adapter are optimized in the fine-tuning process.

[0210] The loss function of the base class training stage is as follows:

[0211] L total =L rpn +L cls +L reg ;

[0212] The visual text matching module is introduced in the new class fine-tuning stage, and the loss function of the new class fine-tuning stage is as follows:

[0213] L total =L rpn +L cls +L reg +L focal ;

[0214] In the formula, L rpn represents the RPN head loss; L cls represents the cross-entropy loss of the classification head; L reg represents the regression loss of the regression head; and L Focal represents the Focal loss calculated based on the alignment prediction result.

[0215] In order to suppress the excessive dependence on the gradient contribution of easily classified samples (such as background) in the training process, the Focal loss L Focal is introduced to replace the standard cross-entropy loss. The calculation formula of the Focal loss L Focal is as follows:

[0216] L Focal =-ω(1-P t ) γ log(P t );

[0217] In the formula, ω and γ are hyperparameters; log(·) is a logarithmic operation; and P t represents the alignment prediction result.

[0218] The RPN head loss L rpn is calculated as follows: the enhanced feature map is input into the RPN head to obtain the RPN prediction result of the contraband category and the bounding box; the cross-entropy loss of the RPN head is calculated based on the contraband category prediction label in the RPN prediction result and the contraband category true label in the real result; the bounding box regression loss of the RPN is calculated based on the bounding box prediction position in the RPN prediction result and the bounding box true position in the real result; and the cross-entropy loss and the bounding box regression loss of the RPN head are added to obtain the RPN head loss L rpn .

[0219] The cross-entropy loss L of the classification head is calculated based on the contraband category prediction label in the prediction result and the contraband category real label in the real result cls ;

[0220] The regression loss L of the regression head is calculated based on the bounding box prediction position in the prediction result and the bounding box real position in the real result reg .

[0221] In the base class training stage, the present application separates the explicit features and the implicit features by using the transmission of the X-ray image, extracts the contraband information in the implicit features, and fuses the explicit features to realize feature decoupling. In the new class fine-tuning stage, the present application designs the VTMM to fuse multi-modal information and improve the discrimination ability and feature clustering effect of the small sample classes in the contraband detection. It is worth noting that the backbone network parameters are frozen in the fine-tuning stage, and only the classification layer is fine-tuned.

[0222] Seven, experimental explanation

[0223] In order to better illustrate the advantages of the technical scheme of the present application, the following experiments are disclosed in the present embodiment.

[0224] 1, experimental data set and evaluation index

[0225] The data set plays a crucial role in deep learning training, and the existing X-ray contraband data set is few to meet the needs of small sample tasks. Among them, the X-RAY FSOD data set integrates OPIXray, HiXray and EDS data sets, a total of 20 contraband categories.

[0226] In this experiment, the X-RAY FSOD data set is selected as the benchmark, and the FSX data set is designed on this basis to better adapt to the research needs of the model framework of the present application.

[0227] Following the setting of PascalVOC, the FSX data set selects 'laptop', 'lighter', 'portable power bank 1', 'electric shoe heater','straight knife', 'folding knife','scissors','multi-functional knife', 'umbrella', 'glass bottle', 'battery','metal cup', 'nail clipper', 'high-pressure tank', and 'alcohol spray' as basic categories, and the rest 'portable power bank 2', 'craft knife','mobile phone','metal tank', and 'beverage bottle' as new categories. In the FSX data set, the number of instances of different categories is between 500 and 5000. In this experiment, about 37000 X-ray images are uniformly selected from the basic categories as the basic class training set, and 1 / 2 / 3 / 5 / 10 instances are selected from each category of the basic category and the new category as the new class fine-tuning training set. It is worth noting that.

[0228] To evaluate the performance of the model accurately, about 18000 images were selected from the remaining basic classes and new classes as the test set. nAP is a commonly used indicator for small sample target detection tasks, and the average precision of new classes. The present application uses nAP50 as an evaluation indicator, which represents the intersection over union (IoU) threshold greater than 0.5.

[0229] 2. Experimental implementation details

[0230] The present experiment uses Faster-RCNN as the basic framework, and pre-trained ResNet50 on ImageNet as the backbone network. The present experiment selects a two-stage fine-tuning training method, which is divided into a basic class training stage and a new class fine-tuning stage. The basic class training stage is trained on the basic class, and the new class fine-tuning stage is fine-tuned on the new class and a small amount of data of the basic class. For two-stage training, the present experiment selects SGD as the optimizer, the momentum is 0.9, the weight decay is 0.0001, and the training batch size is 2. In the basic class training stage, the present experiment sets the learning rate to 0.0025 for 72,000 iterations. In the new class fine-tuning stage, the learning rate is set to 0.0025 for 1 / 2 / 3 / 5 / 10-shot for 4,000 iterations. During fine-tuning, the backbone network parameters of the model are frozen, and the final linear layer of the classification head is trained to fine-tune the model.

[0231] 3. Experimental comparison

[0232] The method based on small sample X-ray contraband detection is relatively scarce (only a small amount of work by WEN et al.). The present experiment integrates the latest small sample learning method and the most advanced X-ray contraband detection algorithm to build a baseline covering 6 combinations. Specifically, OSS, I2O and FDTNet are selected as representative methods for X-ray contraband detection, and SIFT and TIP small sample learning methods are used to form a comprehensive small sample X-ray contraband detection method combination matrix.

[0233] Table 1 nAP50 (%) performance of different methods on the FSXPID dataset. The best result is marked in black bold

[0234]

[0235] Table 1 presents a comparison of the performance of various methods for small-shot X-ray contraband detection on the FSX dataset. Experimental data demonstrates that the proposed method significantly outperforms existing methods in the 1-, 2-, 3-, 5-, and 10-shot settings, achieving performance improvements of 1.46% to 2.99% over the existing small-shot X-ray contraband detection method, WEN. Of particular note, the proposed method maintains excellent robustness in the challenging 1-shot setting, achieving a 1.87% improvement over the second-best small-shot detection method, FDTNet-SIFT. Method comparison analysis reveals that introducing the TIP method to optimize class discrimination during the classification phase actually leads to a decrease in detection performance. This is due to the domain difference between semantic and visual features, which affects the model learning process. As the number of small-shot class samples increases, the proposed method significantly outperforms other methods, demonstrating its robustness. In summary, the proposed method demonstrates superior performance and generalization in small-shot X-ray image detection tasks, significantly improving contraband detection performance under complex conditions.

[0236] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention rather than to limit the technical solutions. Those skilled in the art should understand that modifications or equivalent replacements of the technical solutions of the present invention that do not depart from the purpose and scope of the technical solutions of the present invention should be included in the scope of the claims of the present invention.

Claims

1. A small sample X-ray contraband detection method based on text matching and feature decoupling, characterized by: include: S1: Acquire the X-ray image to be detected; S2: Input the X-ray image to be detected into the trained contraband detection model and output the prediction results of the contraband category and bounding box; The processing steps for training a contraband detection model include: S201: Obtain X-ray images as training data and real results of prohibited item categories and bounding boxes as model input; S202: extracting features from the X-ray image through the backbone network to obtain backbone feature maps at multiple scales; S203: The backbone feature maps of each scale are enhanced through the overlapping feature interaction module to obtain enhanced feature maps of multiple scales. The overlapping feature interaction module uses the cross-attention mechanism to establish dynamic interaction between two-stream features, calculates cross-modal similarity between the query vector of the explicit feature and the key vector of the implicit feature, and mines the potential discriminative clues contained in the implicit space to achieve complementary enhancement of the explicit features. S204: extracting region of interest features from the enhanced feature maps at multiple scales using an RoI head to obtain a region of interest feature map; S205: Calculating the contraband category prediction label and bounding box prediction position as the prediction result based on the feature map of the region of interest through the classification head and the regression head; S206: Aligning the visual features with the text description based on the feature map of the region of interest and the introduced text prior knowledge through the visual text matching module to obtain an alignment prediction result; S207: Calculating a loss function based on the aligned prediction results, the prediction results of the prohibited item category and the bounding box, and the actual results, and reversely optimizing the parameters of the prohibited item detection model; S208: Repeat steps S201 to S207 to iteratively train the contraband detection model until the model converges or the training termination condition is met; S3: The contraband category and bounding box prediction results output by the contraband detection model are used as the contraband detection results of the X-ray image to be detected.

2. The small sample X-ray contraband detection method based on text matching and feature decoupling according to claim 1, characterized in that: In step S2, the pre-trained ResNet50 is used as the backbone network to extract features from the X-ray image and obtain backbone feature maps of multiple scales.

3. The small sample X-ray contraband detection method based on text matching and feature decoupling according to claim 1, characterized in that: In step S203, the processing steps of the overlapping feature interaction module include: S2031: Get the input backbone feature map F i ; S2032: Modeling backbone feature map F using global average pooling layer i The global context information of each channel is used to adaptively generate the channel attention weight map M through two cascaded nonlinear transformation layers c ; The formula is: Where: GAP represents the global average pooling operation; W1 and W2 represent the weight parameters of the two linear transformation layers respectively; σ represents the Sigmoid activation function, B and C represent the number of batches and channel dimensions; S2033: Backbone feature map F through depth-separable convolution structure i After processing, it is combined with the channel attention weight map M c Perform element-by-element multiplication to obtain the enhanced feature map F e ; The formula is: Where: ReLU is the activation function; BN is batch normalization; DWC is depth-wise separable convolution; ⊙ represents element-wise multiplication, H and W represent feature F e height and width; S2034: respectively perform the backbone feature map F i After performing the average pooling and maximum pooling operations, the results of average pooling and maximum pooling are concatenated and mapped with the Sigmoid activation function to generate the saliency weight map M of the contraband. s ; The formula is: Where: Concat represents the tensor concatenation operation; AvgPool represents the average pooling operation; MaxPool represents the maximum pooling operation; S2035: The significance weight map M s With the enhanced feature map F e Perform element-by-element multiplication to obtain the explicit feature map F o ; Execute the saliency weight map M s After the inversion operation, and the enhanced feature map F e Perform element-by-element multiplication to obtain the implicit feature map F l ; S2036: Using the cross attention mechanism, based on the explicit feature map F o and implicit feature map F l Calculate the interaction weight graph A between the high correlation features e , and the interaction weight graph A e and explicit feature map F o Fusion is performed to obtain the enhanced feature map F after information complementation c ; S2037: Extract backbone feature map F through high-pass filter i The high-frequency component of the high-frequency feature map F h ; S2038: Enhanced feature map F after information complementation c and high-frequency feature map F h Add element by element to get the final enhanced feature map The formula is:

4. The small sample X-ray contraband detection method based on text matching and feature decoupling according to claim 3, characterized in that: The processing steps of step S2036 include: S20361: Extract from explicit feature map F through 1×1 convolution layer o The query vector Q o , key vector K o Sum value vector V o ; S20362: Extract the implicit feature map F through a 1×1 convolutional layer l The query vector Q l and key vector K l ; S20363: Construct the interaction weight A based on the contextual perception of implicit features by explicit features and the guidance of explicit features by implicit features o and A l ; The formula is: Where: d o =256 is the key vector K l Dimension; d l =256 is the key vector K o Dimensions; S20364: Interaction weight A o and A l Perform element-by-element multiplication to obtain the final interaction weight graph A e ; The formula is: A e =A o ⊙A l ; S20365: Interaction weight graph A e With the value vector V o After multiplication, with the explicit feature map F o Add element by element to obtain the enhanced feature map F after information complementation c ; The formula is: Where: Represents element-wise addition.

5. The small sample X-ray contraband detection method based on text matching and feature decoupling according to claim 3, characterized in that: In step S2037, a high-pass filter is constructed by the difference between the unit convolution kernel E and the low-pass filter; High-frequency feature map F h The formula is expressed as: Where: Represents a convolution operation; Conv represents a convolution layer with a convolution kernel size of 3×3.

6. The small sample X-ray contraband detection method based on text matching and feature decoupling according to claim 1, characterized in that: In step S206, the processing steps of the visual text matching module include: S2061: Pre-calculate the corresponding text feature vector for each category of X-ray image through the CLIP text encoder and CLIP visual encoder with frozen weights and visual feature vector and stored in the cache; S2062: Obtaining feature map F of the region of interest roi ; S2063: Get text feature vector from cache and visual feature vector And the text feature vectors and visual feature vector As the value vector V T and key vector K F , and at the same time the feature map of the region of interest F roi As a query, feature matching is achieved through key-value vectors; S2064: The feature map of the region of interest F roi Input into Adapter and get the adaptation feature F a ; Among them, the Adapter includes three cascaded trainable linear layers, and a Relu activation function is set between two adjacent linear layers; The formula is: F a =clamp(W5·ReLu(W4·ReLu(W3·F roi )),-5,5); Where: clamp(·) represents a function that limits the numerical range of the tensor; W3, W4, and W5 represent the training weights of the three linear layers; ReLu(·) is the ReLu activation function; S2065: key vector K F Sum value vector V T and the adaptation feature F a Perform matrix-vector multiplication to obtain the matching result P c ; The formula is: P c =exp(-α(1-F a ·(KF) T ))⊙V T ; Where: exp(·) represents the exponential operation; α = 0.5 is a hyperparameter; (·) T represents the transpose operation; ⊙ represents element-by-element multiplication; S2066: The feature map of the region of interest F roi With the frozen weights W of the CLIP text encoder CLIP After multiplication, the matching result P c Perform element-by-element multiplication to obtain the aligned prediction result P t ; The formula is: Where: represents element-wise addition; β = 0.5 is a hyperparameter.

7. The small sample X-ray contraband detection method based on text matching and feature decoupling according to claim 6, characterized in that: Step S2061 specifically includes the following steps: S20611: Obtain X-ray images of a small sample training set and text descriptions of the prohibited item categories to which they belong; S20612: Input the text description of the contraband category of the X-ray image in the small sample training set into the CLIP text encoder with frozen weights to extract text features and obtain the text feature vector And the text vector V as a value vector for the buffer T storage; The formula is: Where: represents the i-th text feature vector of the c-th category; TextEncoder(·) represents the CLIP text encoder with frozen weights; I c Represents the text description of the c-th category; S20613: Input the X-ray image of the small sample training set into the CLIP visual encoder with frozen weights to extract visual features and obtain the visual feature vector and will The key vector K of the cache F storage; The formula is: Where: represents the visual feature vector of the i-th X-ray image of the c-th category; Concat(·) represents the concatenation operation along the channel dimension; ||·|| represents the L2 norm; VisualEncoder(·) represents the CLIP visual encoder with frozen weights; Represents the background category input image in the small sample training set; represents the i-th X-ray image of the n-th contraband category in the small sample training set.

8. The small sample X-ray contraband detection method based on text matching and feature decoupling as claimed in claim 1, characterized in that: In step S2, the training of the contraband detection model includes a base class training phase and a new class fine-tuning phase; The loss function of the base class training phase is as follows: L total =L rpn +L cls +L req ; The visual text matching module is introduced in the new class fine-tuning stage. The loss function of the new class fine-tuning stage is as follows: L total =L rpn +L cls +L req +L focal ; Where: L rpn represents the RPN head loss; L cls represents the cross entropy loss of the classification head; L reg represents the regression loss of the regression head; L Focal Represents the Focal loss calculated based on the aligned prediction results.

9. The small sample X-ray contraband detection method based on text matching and feature decoupling as claimed in claim 8, characterized in that: Focal loss L Focal The calculation formula is expressed as: L Focal =-ω(1-P t ) γ log(P t ); Where: ω and γ are hyperparameters; log(·) is a logarithmic operation; P t Represents the alignment prediction result.

10. The small sample X-ray contraband detection method based on text matching and feature decoupling as claimed in claim 8, characterized in that: Calculate RPN head loss L rpn : Input the enhanced feature map into the RPN head to obtain the RPN prediction results of the contraband category and bounding box; calculate the cross entropy loss of the RPN head based on the contraband category prediction label in the RPN prediction result and the contraband category true label in the true result; calculate the bounding box regression loss of the RPN based on the bounding box prediction position in the RPN prediction result and the bounding box true position in the true result; add the cross entropy loss of the RPN head and the bounding box regression loss to get the RPN head loss L rpn ; Calculate the cross entropy loss L of the classification head based on the predicted label of the contraband category in the predicted result and the true label of the contraband category in the true result cls ; The regression loss L of the regression head is calculated based on the predicted position of the bounding box in the predicted result and the true position of the bounding box in the ground truth result. reg .

Images