X-ray contraband detection method based on contour perception and optimal distribution migration

Abstract

The invention discloses an X-ray contraband detection method based on contour perception and optimal distribution migration. The method comprises the following steps: S1, acquiring an X-ray image to be detected; s2, inputting an X-ray image to be detected into the trained contraband detection model, and outputting a corresponding contraband detection image; in the contraband detection model, the contour information of the contraband is estimated through a contour sensing module to enhance a multi-scale feature map, and an extra new category feature is generated through fitting of an optimal distribution migration module to enhance a roi feature map; and S3, outputting the contraband detection image as a contraband detection result of the to-be-detected X-ray image. According to the method, key contour details describing foreground region space geometry are estimated through a contour sensing module and fused into a feature extraction process; and meanwhile, contour information of foreground prohibited articles and mobility among different types of feature distributions are fully utilized through an optimal distribution migration module, so that overfitting of the model is effectively prevented, and the performance of X-ray prohibited article detection is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of X-ray contraband detection, and in particular to an X-ray contraband detection method based on contour perception and optimal distribution migration. Background Art

[0002] The X-ray Contraband Detection (XPID) model aims to automatically analyze X-ray images of passenger luggage to detect potential contraband, such as knives, guns, ammunition, etc., which plays a vital role in maintaining public safety. However, due to the strict supervision of relevant departments, the frequency of such contraband is low, making it difficult to collect enough X-ray images for model training. This scarcity of data increases the risk of overfitting of the XPID model. To address this problem, the researchers turned their focus to the small sample X-ray Contraband Detection (FSXPID) task, which aims to train the detector with a limited number of X-ray images to classify and locate new categories of contraband.

[0003] The applicant first summarizes the two challenges currently faced in the FSXPID task. The first challenge is how to enhance the recognizability of the features of contraband extracted from X-ray images. Unlike traditional RGB images, contraband in X-ray images have unique features. Due to the difference in the absorption of X-ray energy by contraband made of different materials, X-ray images usually lack rich appearance details due to color degradation, and only the edges and contours are relatively clear. The lack of significant appearance details of contraband will significantly reduce the recognition of its features. Previous XPID methods have proven that the features of contraband can be effectively improved by utilizing the spatial geometric structure of the edge. These methods incorporate the edge information extracted from the entire image by hand-designed Canny or Sobel operators into the features of contraband. However, this approach inevitably introduces edge details from irrelevant background areas, resulting in additional background noise in the features. Naturally, the applicant considers utilizing the contour information of foreground contraband and incorporating it into the multi-scale feature extraction process. The second challenge is to mitigate the risk of overfitting of the FSXPID model under data scarcity conditions. In addition to the challenges posed by the unique characteristics inherent in contraband X-ray images, the limited training data further exacerbates the difficulty of contraband detection. Existing FSXPID methods simplify the classification process of small-shot features by relying on visual prototypes to establish clear decision boundaries between categories. However, visual prototypes extracted from limited image data show significant intra-class differences. In recent years, new progress has been made in small-shot learning, and more intuitive feature synthesis-based strategies have been explored to generate additional features for new categories through estimated feature distributions. These methods focus on calibrating the distribution statistical properties of new categories, but ignore the transferability between feature distributions of different categories.

[0004] In summary, how to utilize the contour information of foreground prohibited items to enhance the recognizability of prohibited items features extracted from X-ray images and how to generate additional small sample features through the transferability between feature distributions of different categories to prevent the risk of model overfitting are issues that technicians in this field urgently need to solve. Summary of the invention

[0005] In view of the deficiencies of the above-mentioned prior art, the technical problem to be solved by the present invention is: how to provide an X-ray contraband detection method based on contour perception and optimal distribution migration, wherein the key contour details describing the spatial geometry of the foreground area are estimated by a contour perception module and integrated into the feature extraction process, thereby realizing the use of the contour information of the foreground contraband items to enhance the recognizability of the contraband item features extracted from the X-ray image; at the same time, the contour information of the foreground contraband items and the transferability between feature distributions of different categories are fully utilized by the optimal distribution migration module, thereby realizing the generation of additional small sample features through the transferability between feature distributions of different categories, effectively preventing model overfitting, and thus improving the performance of X-ray contraband detection.

[0006] Improve the recognition of contraband features and the performance of the model.

[0007] In order to solve the above technical problems, the present invention adopts the following technical solutions:

[0008] X-ray contraband detection method based on contour perception and optimal distribution migration, including:

[0009] S1: Acquire the X-ray image to be detected;

[0010] S2: Input the X-ray image to be detected into the trained contraband detection model and output the corresponding contraband detection map;

[0011] The processing steps for training a contraband detection model are as follows:

[0012] S201: Acquire an X-ray image used as a training sample and a corresponding contraband contour map and contraband annotation map;

[0013] S202: Input the X-ray image into the backbone network for feature extraction to obtain a multi-scale feature map;

[0014] S203: inputting the multi-scale feature map into the contour perception module to estimate the contour information of the contraband, and obtaining a contour probability map; and enhancing the multi-scale feature map by the contour probability map, and obtaining a multi-scale enhanced feature map;

[0015] S204: Calculating a contour loss function based on the contour probability map and the contraband contour map to optimize parameters of the contour perception module;

[0016] S205: extracting the region of interest from the multi-scale enhanced feature map to obtain a roi feature map;

[0017] S206: inputting the roi feature map into the optimal distribution migration module to fit the feature distribution of the new category, generating additional new category features; and enhancing the roi feature map through the new category features to obtain an enhanced roi feature map;

[0018] S207: inputting the enhanced roi feature map into the detection head for detection to obtain a prohibited item detection map; wherein the prohibited item detection map includes a target frame of the prohibited item and a corresponding prohibited item type;

[0019] S208: Calculating a loss function based on the contraband detection graph and the contraband annotation graph to optimize parameters of the contraband detection model;

[0020] S209: repeating steps S201 to S208 to iteratively train the contraband detection model until the model converges or reaches a preset number of iterations;

[0021] S3: Outputting the contraband detection image as the contraband detection result of the X-ray image to be detected.

[0022] Preferably, in step S202, feature extraction is performed through a backbone network based on the two-stage detector Faster RCNN to obtain a multi-scale feature map.

[0023] Preferably, in step S203, the processing steps of the contour perception module generating the contour probability map are as follows:

[0024] S2031: Obtain multi-scale feature map F i＝1，2，3，4 ;

[0025] S2032: Multi-scale feature map F i＝1，2，3，4 Project to the same dimension, and perform top-down concatenation and enhancement on the projected feature map to obtain the contour feature map F′ i＝1,2,3,4 ;

[0026] The formula is:

[0027]

[0028] The projection formula is expressed as:

[0029]

[0030] Where: BL(·) represents the base layer consisting of 3×3 convolution, batch normalization and ReLU activation function; up(·) represents the upsampling operation; ⊙ represents element-wise multiplication; PL(·) represents the projection layer consisting of 1×1 convolution, 3×3 convolution and ReLU activation function;

[0031] S2033: Contour feature map F′ 1 Perform 1×1 convolution to generate contour probability map C P .

[0032] Preferably, in step S203, the processing steps of the contour perception module generating the enhanced feature map are as follows:

[0033] S2034: Multi-scale feature map F i＝1，2，3，4 Input the transition layer composed of 1×1 convolution and 3×3 convolution to obtain the feature map TL (F i );

[0034] S2035: For the feature map F of the last scale 4 :The feature map TL(F 4 ) and contour probability map C P Input the basic module together for feature fusion to obtain the enhanced feature map of the final scale

[0035] The calculation formula of the basic module is expressed as:

[0036]

[0037] S2036: For the feature map F before the last scale i＝1,2,3 :The feature map TL(F i＝1,2,3 ), contour probability map C P And the enhanced feature map of the next scale Input the basic module together for feature fusion to obtain the enhanced feature map of this scale

[0038] The calculation formula of the basic module is expressed as:

[0039]

[0040] Where: [·,·] represents the concatenation operation; the adjustment parameters γ and β are composed of a series of different 3×3 convolutions; represents element-wise addition; represents element-wise multiplication;

[0041] S2037: Obtain multi-scale enhanced feature maps through the previous steps

[0042] Preferably, in step S204, weighted binary cross entropy loss and weighted intersection-over-union loss are calculated based on the contour probability map and the contraband contour map respectively; and the weighted binary cross entropy loss and the weighted intersection-over-union loss are weighted summed to obtain the contour loss function L contour ;

[0043] The formula is:

[0044] L contour =L wBCE +L wIoU ,

[0045] Where: L wBCE represents the weighted binary cross entropy loss; L wIoU represents the weighted intersection-over-union loss.

[0046] Preferably, in step S205, the multi-scale enhanced feature map is input into the RPN network and the feature of interest alignment module to extract the region of interest to obtain the roi feature map.

[0047] Preferably, in step S206, the optimal distribution migration module generates new category features in the following processing steps:

[0048] S2061: Based on roi feature map F roi Modeling New Categories The characteristic distribution of

[0049] S2062: Obtain basic category dataset D base ; From the basic category dataset D base Extract feature vectors from where f i represents the feature vector corresponding to the i-th category, N represents the total number of basic categories; use f to calculate the mean μ of each basic category base and the corresponding covariance matrix ∑ base , get the basic category The characteristic distribution of

[0050] S2063: Calculate the cosine similarity between the basic category and the new category in the feature space;

[0051] S2064: From the basic category Identify and identify new categories The base class mean with the highest cosine similarity and its corresponding covariance matrix Then model the basic category with the highest cosine similarity with the new category The characteristic distribution of

[0052] S2065: From the basic category and new categories Sampling the features of the basic category and the new category, and obtaining the basic category feature S base and the new category feature S novel ; The basic category feature S base and the new category feature S novel The i-th sample feature in is expressed as and m represents the number of sampled features;

[0053] S2066: Calculate basic category features S using the Earth Mover distance algorithm base and the new category feature S novel The optimal transmission scheme between

[0054] S2067: Adjust the new category feature S through the optimal transmission scheme P novel , get the calibrated new category feature S′ novel ;

[0055] The formula is expressed as;

[0056]

[0057] Where: Represents matrix multiplication;

[0058] S2068: The calibrated new category feature S′ novel and roi feature map F roi Splice to get the enhanced roi feature map F roi_aug ;

[0059] The formula is:

[0060] F roi_avg =[F roi ,MLP(S′ novel )],

[0061] Where: MLP represents a multi-layer perceptron consisting of a fully connected layer and an activation function.

[0062] Preferably, in step S2061, the new category is calculated by the following formula: The characteristic distribution of

[0063] Where: Represents the roi feature map F roi The i-th foreground roi feature in; M represents the roi feature map F roi The number of foreground roi features in the image.

[0064] Preferably, in step S2063, the cosine similarity between the basic category and the new category in the feature space is calculated by the following formula:

[0065]

[0066] Where: represents the cosine similarity between the mean of the jth basic category and the mean of the new category; d represents the number of basic categories.

[0067] Preferably, in step S2066, the optimal transmission solution P is calculated by the following formula:

[0068]

[0069] Where: P ij From the basic category The i-th sampled feature is transferred to the new category The allocation ratio of the jth sampling feature in ; Represents and from the basic category The i-th sampled feature is transferred to the new category The distance cost matrix associated with the j-th sampled feature in ||·|| 2 express -norm.

[0070] Compared with the prior art, the X-ray contraband detection method based on contour perception and optimal distribution migration in the present invention has the following beneficial effects:

[0071] First, in view of the inherent visual characteristics of X-ray images, the present invention designs a contour perception module for the contour information of foreground prohibited items. The module focuses on estimating the key contour details that describe the spatial geometry of the foreground area, and effectively integrates the foreground contour information of the prohibited items into the feature extraction process, thereby realizing the use of the contour information of the foreground prohibited items to enhance the recognizability of the prohibited items features extracted from the X-ray images, and can enhance the model's ability to capture the spatial geometric structure of the object and the ability to distinguish the features of the prohibited items, thereby improving the model's feature recognition in the feature extraction process, and further improving the performance and accuracy of X-ray contraband detection. Secondly, the present invention designs an optimal distribution migration module based on the transferability between feature distributions of different categories. Different from directly learning the decision boundary of a new category, this module makes full use of the contour information of the foreground contraband items and the transferability between feature distributions of different categories, transfers the base class distribution with rich data to the new category distribution estimated by limited data, and uses the optimal transmission strategy to accurately model the feature distribution of small sample categories in small sample scenarios, and realizes the generation of additional small sample features through the transferability between feature distributions of different categories. With the help of these carefully synthesized features, the FSXPID model is able to have a strong small sample learning ability, effectively preventing the model from overfitting, thereby improving the small sample detection performance and detection accuracy of the model. Finally, the present invention uses the above two modules as the core to construct a novel FSXPID model based on Faster R-CNN to effectively enhance the distinguishing ability and detection performance of contraband features. A large number of experimental results have proved that the method proposed by the present invention significantly improves the detection performance and shows excellent robustness. BRIEF DESCRIPTION OF THE DRAWINGS

[0072] In order to make the purpose, technical solution and advantages of the invention more clear, the present invention will be further described in detail below with reference to the accompanying drawings, in which:

[0073] Figure 1 4 is an overall framework diagram of the contraband detection model in an embodiment of the present invention.

[0074] Figure 2 illustrative diagram of the architecture of a contour sensing module (CAM) in an embodiment of the present invention.

[0075] Figure 3 Schematic diagram of an optimal distributed migration (ODTM) module in an embodiment of the present invention. DETAILED DESCRIPTION

[0076] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. The components of the embodiments of the present invention generally described and shown in the drawings here can be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present invention provided in the drawings is not intended to limit the scope of the claimed invention, but only represents selected embodiments of the present invention. Based on the embodiments in the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work belong to the scope of protection of the present invention.

[0077] The following is a further detailed description through specific implementation methods:

[0078] Example:

[0079] This embodiment discloses an X-ray contraband detection method based on contour perception and optimal distribution migration.

[0080] X-ray contraband detection method based on contour perception and optimal distribution migration, including:

[0081] S1: Acquire the X-ray image to be detected;

[0082] S2: Input the X-ray image to be detected into the trained contraband detection model (hereinafter also referred to as FSXPID), and output the corresponding contraband detection map;

[0083] Combination Figure 1 As shown, the processing steps for training the contraband detection model are as follows:

[0084] S201: Acquire an X-ray image used as a training sample and a corresponding contraband contour map and contraband annotation map;

[0085] S202: Input the X-ray image into the backbone network for feature extraction to obtain a multi-scale feature map;

[0086] S203: inputting the multi-scale feature map into a contour awareness module (CAM) to estimate the contour information of the contraband and obtain a contour probability map; and enhancing the multi-scale feature map by using the contour probability map to obtain a multi-scale enhanced feature map;

[0087] S204: Calculating a contour loss function based on the contour probability map and the contraband contour map to optimize parameters of the contour perception module;

[0088] S205: extracting the region of interest from the multi-scale enhanced feature map to obtain a roi feature map;

[0089] S206: inputting the roi feature map into the Optimal Distribution Transfer Module (ODTM) module to fit the feature distribution of the new category, generating additional new category features; and enhancing the roi feature map through the new category features to obtain an enhanced roi feature map;

[0090] S207: inputting the enhanced roi feature map into the detection head for detection to obtain a prohibited item detection map; wherein the prohibited item detection map includes a target frame of the prohibited item and a corresponding prohibited item type;

[0091] In this embodiment, the detection head used is an existing network structure, which mainly includes a regression module ( Figure 1 "Reg.") and the Classification Module ( Figure 1 "Cls.") shown, which has not been improved.

[0092] S208: Calculating a loss function based on the contraband detection graph and the contraband annotation graph to optimize parameters of the contraband detection model;

[0093] In this embodiment, no improvement is made to this part of the loss function, and the existing loss function is used.

[0094] S209: repeating steps S201 to S208 to iteratively train the contraband detection model until the model converges or reaches a preset number of iterations;

[0095] S3: Outputting the contraband detection image as the contraband detection result of the X-ray image to be detected.

[0096] First, in view of the inherent visual characteristics of X-ray images, the present invention designs a contour perception module for the contour information of foreground prohibited items. The module focuses on estimating the key contour details that describe the spatial geometry of the foreground area, and effectively integrates the foreground contour information of the prohibited items into the feature extraction process, thereby realizing the use of the contour information of the foreground prohibited items to enhance the recognizability of the prohibited items features extracted from the X-ray images, and can enhance the model's ability to capture the spatial geometric structure of the object and the ability to distinguish the features of the prohibited items, thereby improving the model's feature recognition in the feature extraction process, and further improving the performance and accuracy of X-ray contraband detection. Secondly, the present invention designs an optimal distribution migration module based on the transferability between feature distributions of different categories. Different from directly learning the decision boundary of a new category, this module makes full use of the contour information of the foreground contraband items and the transferability between feature distributions of different categories, transfers the base class distribution with rich data to the new category distribution estimated by limited data, and uses the optimal transmission strategy to accurately model the feature distribution of small sample categories in small sample scenarios, and realizes the generation of additional small sample features through the transferability between feature distributions of different categories. With the help of these carefully synthesized features, the FSXPID model is able to have a strong small sample learning ability, effectively preventing the model from overfitting, thereby improving the small sample detection performance and detection accuracy of the model. Finally, the present invention uses the above two modules as the core to construct a novel FSXPID model based on Faster R-CNN to effectively enhance the distinguishing ability and detection performance of contraband features. A large number of experimental results have proved that the method proposed by the present invention significantly improves the detection performance and shows excellent robustness.

[0097] In order to better introduce the technical solution of the present invention, this embodiment is explained through the following steps.

[0098] 1. Backbone network

[0099] In this embodiment, feature extraction is performed through a backbone network based on the two-stage detector Faster RCNN to obtain a multi-scale feature map. The backbone network is a ResNet50 network, and its feature extraction process can be expressed as Among them, H i , W i and C i Represent the i-th stage feature F i The height, width, and channel dimensions of the .

[0100] 2. Contour Perception (CAM) Module

[0101] The applicant has found that since X-ray images of contraband lack significant texture information, it is particularly important to capture edge contours to represent the spatial geometric shape of contraband during the detection process. The present invention proposes a contour awareness (CAM) module that estimates the contour of contraband using features from different stages of the feature extractor. Guided by the estimated contour information, CAM adaptively fuses the spatial geometric information of contraband into the features.

[0102] CAM uses features from different stages of the feature extractor to estimate key contour details that describe the spatial geometry of the foreground area and incorporates these details into the multi-scale feature extraction process. This module is designed to enhance the model's ability to capture detailed contour information in the backbone feature extraction stage, thereby improving the recognition of prohibited item features. Figure 2 As shown in the figure, CAM first adopts a structure similar to FPN to fuse the features of each stage of the backbone network step by step, and then adaptively estimates the foreground contour probability map of prohibited items based on the fused features. It is worth noting that CAM uses contour annotation to supervise the contour perception process during the training phase. Once the foreground contour probability map is successfully estimated, CAM embeds the corresponding contour information into the backbone features through conditional batch normalization, which can significantly improve the subsequent detection performance by enhancing the spatial geometric structure representation capability of the prohibited items features.

[0103] Specifically, the processing steps of the contour perception module to generate the contour probability map are as follows:

[0104] S2031: Obtain multi-scale feature map F i＝1，2，3，4 ;

[0105] Given an X-ray contraband image Where H and W represent the height and width of the image respectively. CAM uses ResNet50 as the feature extractor Extract the corresponding feature I from F. The feature extraction process of ResNet50 can be expressed as Among them, H i , W i and C i Represent the i-th stage feature F i The height, width and channel dimensions of F 0 contains too much noise, CAM only uses F i＝1，2，3，4 To estimate the contour probability map C P .

[0106] S2032: Multi-scale feature map F i＝1，2，3，4 Project to the same dimension, and perform top-down concatenation and enhancement on the projected feature map to obtain the contour feature map F′ i＝1,2,3,4 ;

[0107] The formula is:

[0108]

[0109] The projection formula is expressed as:

[0110]

[0111] Where: BL(·) represents the base layer consisting of 3×3 convolution, batch normalization and ReLU activation function; up(·) represents the upsampling operation; ⊙ represents element-wise multiplication; PL(·) represents the projection layer consisting of 1×1 convolution, 3×3 convolution and ReLU activation function;

[0112] S2033: Contour feature map F′ 1 Perform 1×1 convolution to generate contour probability map C P .

[0113] Specifically, CAM supervises the learning process of the contour probability map through contour loss: weighted binary cross entropy loss and weighted intersection-over-union loss are calculated based on the contour probability map and the contraband contour map respectively; and the weighted binary cross entropy loss and weighted intersection-over-union loss are weighted summed to obtain the contour loss function L contour ;

[0114] The formula is:

[0115] L contour =L wBCE +L wIoU ,

[0116] Where: L wBCE represents the weighted binary cross entropy loss; L wIoU It is worth noting that CAM applies Sobel operation on the segmentation labels of the image during the training phase to obtain the true labels of the contours.

[0117] In obtaining C P Finally, CAM considers embedding contour information into the multi-scale feature extraction process. In different security inspection scenarios, the scale of the same contraband in the X-ray image may vary greatly due to the different placement and orientation of the objects. CAM uses a concatenation operation to embed the features F processed by the transition layer into the multi-scale feature extraction process. i With features The fused features are then input into another basic module and combined with the features CAM then adjusts the obtained results using a method similar to conditional batch normalization. The adjustment parameters γ and β are passed through two different 3×3 convolutional layers from C P The purpose of learning is to embed contour information into the representation space.

[0118] Specifically, the processing steps for the contour perception module to generate an enhanced feature map are as follows:

[0119] S2034: Multi-scale feature map F i＝1，2，3，4 Input the transition layer composed of 1×1 convolution and 3×3 convolution to obtain the feature map TL (F i );

[0120] S2035: For the feature map F of the last scale 4 :The feature map TL(F 4 ) and contour probability map C P Input the basic module together for feature fusion to obtain the enhanced feature map of the final scale

[0121] The calculation formula of the basic module is expressed as:

[0122]

[0123] S2036: For the feature map F before the last scale i＝1,2,3 :The feature map TL(F i＝1,2,3 ), contour probability map C P And the enhanced feature map of the next scale Input the basic module together for feature fusion to obtain the enhanced feature map of this scale

[0124] The calculation formula of the basic module is expressed as:

[0125]

[0126] Where: [·,·] represents the concatenation operation; the adjustment parameters γ and β are composed of a series of different 3×3 convolutions; represents element-wise addition; represents element-wise multiplication;

[0127] S2037: Obtain multi-scale enhanced feature maps through the previous steps and The channel dimension is unified to 256.

[0128] 3. Region of Interest Extraction

[0129] In this embodiment, the multi-scale enhanced feature map is input into the RPN network and the feature alignment module of interest (such as the ROIAlign module) to extract the region of interest (ROI) to obtain the ROI feature map F roi Specifically, the enhanced features are input into the RPN network to obtain the roi feature vector. On the basis of Passed into the subsequent modules of Faster R-CNN to extract features of the region of interest Where B represents F roi The number of r represents F roi The width or height of F is then flattened through a fully connected (FC) layer. roi Convert to d-dimensional space and generate output roi feature map This embodiment uses the existing RPN network and feature of interest alignment module, and does not make any improvements to this part.

[0130] Optimal Distribution Migration (ODTM) Module

[0131] In order to alleviate the overfitting problem caused by limited training data, the applicant adopts ODTM to more accurately model the feature distribution of new categories through cross-category distribution transfer. ODTM assumes that the feature distribution between similar categories is transferable and follows a multivariate normal distribution.

[0132] First, ODTM estimates the feature distribution of basic categories with sufficient samples in the pre-training stage. During the training process, ODTM models the feature distribution of new categories online. However, due to data scarcity, the estimated distribution of new categories is often inaccurate. In order to directly calibrate these estimated distributions, ODTM calculates the cost matrix between the basic category and the new category distribution, and evaluates the migration path in the feature space through the optimal transfer theory. Next, ODTM uses Earth Mover's Distance to solve the minimum cost required to migrate the basic category distribution to the new category distribution. By optimizing the distribution migration from the basic category to the new category, ODTM is able to accurately characterize the new category distribution and sample more new category features from it. With these carefully synthesized features, the FSXPID model has powerful small sample learning capabilities.

[0133] Existing FSXPID methods mainly learn sharper decision boundaries to distinguish new categories by utilizing limited training data. In contrast, ODTM uses optimal transfer theory to accurately model the feature distribution of new categories to generate additional small sample features, such as Figure 3 As shown, ODTM assumes that the feature distribution of each category follows a multivariate normal distribution.

[0134] Specifically, the processing steps for the optimal distribution migration module to generate new category features are as follows:

[0135] S2061: Based on roi feature map F roi Modeling New Categories The characteristic distribution of

[0136] The new category is calculated by the following formula The characteristic distribution of

[0137]

[0138] Where: Represents the roi feature map F roi The i-th foreground roi feature in; M represents the roi feature map F roi Using the calculated statistics, ODTM models the new category However, due to the limited training data for new categories, the estimated It can only capture part of the characteristics of the true distribution of new categories.

[0139] S2062: Feature distributions of similar categories can be effectively transferred between different categories. ODTM uses the distribution of basic categories with rich samples to recalibrate inaccurate To achieve this goal, we obtain a base class dataset D before fine-tuning on new classes. base ; Use the pre-trained network (ResNet50 network) to extract the basic category dataset D base Extract feature vectors from where f i represents the feature vector corresponding to the i-th category, N represents the total number of basic categories; use f to calculate the mean μ of each basic category base and the corresponding covariance matrix ∑ base , get the basic category The characteristic distribution of In order to ensure base Maintaining positive definiteness, ODTM in ∑ base Add a small positive constant ε=1×10 to the diagonal elements of -4 .

[0140] S2063: Calculate the cosine similarity between the basic category and the new category in the feature space;

[0141] The cosine similarity between the basic category and the new category in the feature space is calculated using the following formula:

[0142]

[0143] Where: represents the cosine similarity between the mean of the jth basic category and the mean of the new category; d represents the number of basic categories.

[0144] S2064: From the basic category Identify and identify new categories The base class mean with the highest cosine similarity and its corresponding covariance matrix Then model the basic category with the highest cosine similarity with the new category The characteristic distribution of

[0145] S2065: From the basic category and new categories Sampling the features of the basic category and the new category, and obtaining the basic category feature S base and the new category feature S novel ; The basic category feature S base and the new category feature S novel The i-th sample feature in is expressed as and m represents the number of sampled features;

[0146] S2066: Calculate the basic category feature S by using the Earth Mover's Distance (EMD) algorithm base and the new category feature S novel The optimal transmission scheme between To minimize the total transmission cost;

[0147] The optimal transmission solution P is calculated by the following formula:

[0148]

[0149] Where: P ij From the basic category The i-th sampled feature is transferred to the new category The allocation ratio of the jth sampling feature in ; Represents and from the basic category The i-th sampled feature is transferred to the new category The distance cost matrix associated with the j-th sampled feature in ||·|| 2 express -norm. It is worth noting that ODTM uses the Sinkhornal algorithm to solve the above optimal transfer problem. Overall, the ultimate goal of ODTM is to transfer the basic category distribution to the new category distribution with minimal cost while satisfying the constraints.

[0150] S2067: Adjust the new category feature S through the optimal transmission scheme P novel , get the calibrated new category feature S′ novel ;

[0151] The formula is expressed as;

[0152]

[0153] Where: Represents matrix multiplication;

[0154] S2068: The calibrated new category feature S′ novel and roi feature map F roi Splice to get the enhanced roi feature map F roi_aug ;

[0155] The formula is:

[0156] F roi_aug =[F roi ,MLP(S′ novel )]

[0157] Where: MLP represents a multi-layer perceptron consisting of a fully connected layer (FC) and an activation function. At the same time, ODTM also needs to be based on S′ novel The category of F roi_aug Therefore, ODTM can further optimize F by leveraging the guidance of category label information. roi_aug The subsequent classification and box regression tasks will significantly benefit from the well-learned enhanced features from the new categories.

[0158] 5. Experimental Description

[0159] 1. Experimental detailed parameter settings and data sets

[0160] This experiment uses Faster R-CNN as the detection framework, combined with the ResNet50 backbone network pre-trained on ImageNet. The brief process of the present invention is as follows: First, input the X-ray image and the contour image, extract the features through the backbone ResNet50 with Faster RCNN as the main body of the two-stage detector, and obtain the predicted contraband contour information through CAM, and guide the model to focus on the foreground contraband features through contour loss, and enhance the backbone features; then, input the enhanced features into the RPN network to obtain the roi feature vector, and input the roi feature vector into ODTM to fit the accurate new category feature distribution, generate additional new category features, and send them to the subsequent detection together with the original roi feature vector; finally, input the enhanced feature vector into the detection head to generate the final detection map.

[0161] This experiment uses an RTX 4090 GPU for training and testing, and adopts the following parameter configuration: During the optimization process, stochastic gradient descent (SGD) with momentum of 0.9 and weight decay of 0.0001 is used, and the training batch size is set to 2. In the basic category training stage, the learning rate is set to 0.0025 and trained for 72,000 iterations. After that, the last fully connected (FC) layer of the classification head is replaced with a randomly initialized FC layer. In the new category fine-tuning stage, the learning rate is reduced by one order of magnitude after the 2,800th and 3,400th iterations, respectively, and the entire stage is performed for a total of 4,000 iterations. The parameters d and λ are set to 2048 and 0.2, respectively. The present invention samples 150 features for each new category from the transferred distribution.

[0162] Among them, PIDray is a recently released dataset designed specifically for the XPID task, covering a variety of real-world scenarios for detecting prohibited items. In addition, the segmentation labels in PIDray help to extract the contour labels of prohibited items. Based on the PIDray dataset, this experiment developed a customized FSXPID dataset. According to the category division setting of the small sample object detection task, the FSXPID dataset specifies "scissors", "wrench", "gun", "bullet", "handcuffs", "knife" and "lighter" in PIDray as basic categories, while "baton", "pliers", "hammer" and "mobile power bank" are specified as new categories. The FSXPID dataset contains about 12,000 images that only contain basic categories in the PIDray training set, which are used to construct D base Subsequently, the FSXPID dataset samples k = 1, 2, 3, 5, and 10 labeled instances containing only new categories from the remaining training set to construct D novel .

[0163] In order to comprehensively evaluate the model performance, the FSXPID dataset collects about 12,000 images containing basic categories and new categories from the PIDray test set as the test set of the FSXPID dataset. It is worth noting that each training image in the FSXPID dataset only contains simple type images, while the test set includes three types of images: simple, difficult, and hidden. Simple images only contain instances of a single category, difficult images contain instances of multiple categories, and hidden images contain prohibited items that are difficult to detect.

[0164] 2. Evaluation indicators

[0165] This experiment uses nAP50 to evaluate the proposed method. nAP50 represents the average precision for new categories when the intersection over union (IoU) threshold is 0.5, and is calculated as follows:

[0166]

[0167] Among them, AP50 refers to the average precision calculated when the IoU threshold is fixed at 0.50. μ is usually a statistical value based on the dataset or target category, which is used to make the performance indicators of different models more comparable. This experiment calculates the nAP50 (%) of the proposed FSXPID method under different settings (1 / 2 / 3 / 5 / 10-shot). The higher the value, the better the performance of the model.

[0168] 3. Experimental comparison

[0169] On the FSXPID dataset, the method of the present invention is compared with 9 recent state-of-the-art and reproducible methods under different settings (1 / 2 / 3 / 5 / 10-shot), and the results are shown in Table 1. The algorithm of the present invention is improved by 11.43% compared with the existing FSXPID method WEN. In addition, the method of the present invention is significantly better than the FSXPID method obtained by different combination methods. However, in the 1-shot setting, the method of the present invention is only improved by 0.98% compared with OOS-DC. The applicant speculates that this limited improvement stems from the scarcity of shared attributes of the category distribution extracted from a single image. As the number of samples increases, the small sample learning performance of the method of the present invention gradually improves. In addition, OOS-SIFT and DCP-SIFT rely on semantic embedding to generate new category features. However, the word embedding generated by RGB images does not match the semantics of X-ray images, which limits the effect of SIFT-based methods. Overall, the experimental results show that the method of the present invention is specifically designed to make full use of the unique features of X-ray images and significantly improves the performance of prohibited items detection.

[0170] Table 1 nAP50 (%) results on the FSXPID dataset: black bold indicates the best result

[0171]

[0172] 4. Ablation experiment

[0173] In the experiment, the method proposed in the present invention conducted a series of ablation experiments on the FSXPID dataset to verify the effectiveness of the proposed method. All tables in this experiment show the best results in bold black. In order to verify the nAP50 (%) performance of different components on the FSXPID dataset. The present invention conducted the ablation experiments in Table 2 to evaluate the effects of the contour perception module (CAM) and the optimal distribution transfer module (ODTM). The experimental results show that the use of CAM and ODTM alone can help improve the detection performance under 1 / 2 / 3 / 5 / 10-shot settings. It is worth noting that the use of CAM alone failed to significantly improve the performance under 1-shot and 2-shot settings, which may be attributed to the impact of the limited training data on the model performance. The use of ODTM alone improves the detection performance by 1.57% to 5.58% under different shot settings. In addition, the combination of CAM and ODTM always brings more significant performance improvements. The results in Table 2 demonstrate the effectiveness of these two modules in improving the detection performance of the model.

[0174] Table 2 nAP50 (%) performance of different components on the FSXPID dataset

[0175]

[0176] In order to evaluate the effects of different supervisory images used in CAM, the present invention conducted the ablation experiments in Table 3. In the first row of Table 3, the present invention applies the Sobel operator to extract edges from the entire image and uses it as a supervisory signal. In the second row of Table 3, the present invention only uses the edge information of the foreground area to supervise the training process. The experimental results show that the background noise in the edge of the image significantly affects the edge supervision and hinders the extraction of effective features. Using only the foreground edge as a supervisory signal significantly improves the detection ability of the model. However, part of the foreground area is still obscured by the background and is still interfered by noise. In order to further optimize the learning process, the present invention uses the clear contour information of the foreground prohibited items as a supervisory signal. Under the 1 / 2 / 3 / 5 / 10-shot setting, CAM achieves the best feature extraction performance.

[0177] Table 3 nAP50 (%) performance of different supervisory signals used in CAM on the FSXPID dataset

[0178]

[0179] In order to evaluate the effects of different distribution modeling strategies used in ODTM, the present invention conducts the ablation experiments shown in Table 4. The online modeling in Table 4 refers to directly modeling the distribution of new classes in the new class fine-tuning stage, while DC represents modeling the distribution of new classes by using the distribution statistics of the base classes. The distribution of new classes obtained by online modeling is inaccurately estimated due to data scarcity, which limits the performance of small sample learning.

[0180] Although DC improves performance by using distribution statistics for calibration, the low semantic association between contraband categories may limit further performance improvement. The present invention directly transfers the distribution of the basic category to the distribution of the new category through the optimal path. This transfer makes the distribution representation of the new category more accurate and significantly improves the small sample detection performance.

[0181] Table 4 nAP50 (%) performance of different distribution modeling strategies used in ODTM on the FSXPID dataset

[0182]

[0183] The present invention evaluates the influence of different scaling factors λ through the ablation experiment in Table 5. According to the experimental results in the table, when λ=0.2, nAP50 always shows the highest detection performance under the 1 / 2 / 3 / 5 / 10-shot settings. Increasing or decreasing the scaling factor will lead to a decrease in the detection effect. From the trend in Table 5, it can be seen that the performance shows a decreasing pattern centered on λ=0.2. Therefore, the present invention sets λ=0.2 as the optimal scaling factor.

[0184] Table 5 nAP50 (%) performance of different scaling factors λ on FSXPID dataset

[0185]

[0186] In order to evaluate the impact of different contour supervision losses, the present invention conducts the ablation experiments shown in Table 6. The cross entropy loss L widely used in segmentation tasks bce And the mean square error loss L in the regression task contour , all show limited performance improvement. In addition, the present invention applies L tversky This method is usually used in medical X-ray imaging to effectively improve detection performance. contour To enhance the perception of contour information and alleviate the problem of imbalance between foreground and background pixels. The ablation experiment results show that L contour effectiveness.

[0187] Table 6 nAP50 (%) performance of different contour supervision losses on FSXPID dataset

[0188]

[0189] The present invention evaluates the effect of different sampling numbers of new category features through ablation experiments, and the experimental results are shown in Table 7. Under the 5-shot and 10-shot settings, the detection performance reaches a peak when the sampling number is 100, while under the 1-shot, 2-shot and 3-shot settings, the detection performance is relatively low. After increasing the sampling number to 150, the performance of the 1-shot, 2-shot and 3-shot settings is significantly improved, and the 5-shot and 10-shot settings still maintain good detection effects. However, after the sampling number exceeds 150, the performance decreases significantly. These results show that a moderate increase in the number of samples helps to improve detection capabilities. Therefore, the experiment concludes that the number of samples of 150 is optimal.

[0190] Table 7 nAP50 (%) performance of different numbers of sampled features on the FSXPID dataset

[0191]

[0192] Finally, it should be noted that the above embodiments are only used to illustrate the technical solution of the present invention rather than to limit the technical solution. Those skilled in the art should understand that those modifications or equivalent substitutions of the technical solution of the present invention that do not depart from the purpose and scope of the technical solution should be included in the scope of the claims of the present invention.

Claims

1. An X-ray contraband detection method based on contour perception and optimal distribution migration, characterized in that: include: S1: Acquire the X-ray image to be detected; S2: Input the X-ray image to be detected into the trained contraband detection model and output the corresponding contraband detection map; The processing steps for training a contraband detection model are as follows: S201: Acquire an X-ray image used as a training sample and a corresponding contraband contour map and contraband annotation map; S202: Input the X-ray image into the backbone network for feature extraction to obtain a multi-scale feature map; S203: inputting the multi-scale feature map into the contour perception module to estimate the contour information of the contraband, and obtaining a contour probability map; and enhancing the multi-scale feature map by the contour probability map, and obtaining a multi-scale enhanced feature map; S204: Calculating a contour loss function based on the contour probability map and the contraband contour map to optimize parameters of the contour perception module; S205: extracting the region of interest from the multi-scale enhanced feature map to obtain a roi feature map; S206: inputting the roi feature map into the optimal distribution migration module to fit the feature distribution of the new category, and generating additional new category features; And enhance the roi feature map through the new category feature to obtain the enhanced roi feature map; S207: inputting the enhanced roi feature map into the detection head for detection to obtain a prohibited item detection map; wherein the prohibited item detection map includes a target frame of the prohibited item and a corresponding prohibited item type; S208: Calculating a loss function based on the contraband detection graph and the contraband annotation graph to optimize parameters of the contraband detection model; S209: repeating steps S201 to S208 to iteratively train the contraband detection model until the model converges or reaches a preset number of iterations; S3: Outputting the contraband detection image as the contraband detection result of the X-ray image to be detected.

2. The X-ray contraband detection method based on contour perception and optimal distribution migration as claimed in claim 1, characterized in that: In step S202, feature extraction is performed through a backbone network based on the two-stage detector FasterRCNN to obtain a multi-scale feature map.

3. The X-ray contraband detection method based on contour perception and optimal distribution migration as claimed in claim 1, characterized in that: In step S203, the contour perception module generates a contour probability map in the following steps: S2031: Obtain multi-scale feature map F i＝1，2，3，4 ; S2032: Multi-scale feature map F i＝1，2，3，4 Project to the same dimension, and perform top-down concatenation and enhancement on the projected feature map to obtain the contour feature map F′ i＝1，2，3，4 ; The formula is: The projection formula is expressed as: Where: BL(·) represents the base layer consisting of 3×3 convolution, batch normalization and ReLU activation function; up(·) represents the upsampling operation; ⊙ represents element-wise multiplication; PL(·) represents the projection layer consisting of 1×1 convolution, 3×3 convolution and ReLU activation function; S2033: Perform 1×1 convolution on the contour feature map F′1 to generate a contour probability map C P .

4. The X-ray contraband detection method based on contour perception and optimal distribution migration as claimed in claim 3, characterized in that: In step S203, the contour perception module generates an enhanced feature map in the following processing steps: S2034: Multi-scale feature map F i＝1，2，3，4 Input the transition layer composed of 1×1 convolution and 3×3 convolution to obtain the feature map TL (F i ); S2035: For the feature map F4 of the last scale: the feature map TL (F4) after transition processing and the contour probability map c P - Input the basic module together for feature fusion to obtain the enhanced feature map of the final scale The calculation formula of the basic module is expressed as: S2036: For the feature map F before the last scale i＝1，2，3 :The feature map TL(F i＝1，2，3 ), contour probability map c P And the enhanced feature map of the next scale Input the basic module together for feature fusion to obtain the enhanced feature map of this scale The calculation formula of the basic module is expressed as: Where: [·,·] represents the splicing operation; the adjustment parameter γ and β consist of a series of different 3×3 convolutions; represents element-wise addition; represents element-wise multiplication; S2037: Obtain multi-scale enhanced feature maps through the previous steps 5. The X-ray contraband detection method based on contour perception and optimal distribution migration as claimed in claim 1, characterized in that: In step S204, the weighted binary cross entropy loss and the weighted intersection-over-union loss are calculated based on the contour probability map and the contraband contour map, respectively; and the weighted binary cross entropy loss and the weighted intersection-over-union loss are weighted summed to obtain the contour loss function L contour ; The formula is: L contour =L wBCE +L wIoU , Where: L wBCE represents the weighted binary cross entropy loss; L wIoU represents the weighted intersection-over-union loss.

6. The X-ray contraband detection method based on contour perception and optimal distribution migration as claimed in claim 1, characterized in that: In step S205, the multi-scale enhanced feature map is input into the RPN network and the feature of interest alignment module to extract the region of interest to obtain the roi feature map.

7. The X-ray contraband detection method based on contour perception and optimal distribution migration as claimed in claim 1, characterized in that: In step S206, the optimal distribution migration module generates new category features in the following processing steps: S2061: Based on roi feature map F roi Modeling New Categories The characteristic distribution of S2062: Obtain basic category dataset D base ; From the basic category dataset D base Extract feature vectors from where f i represents the feature vector corresponding to the i-th category, N represents the total number of basic categories; use f to calculate the mean μ of each basic category base and the corresponding covariance matrix ∑ base , get the basic category The characteristic distribution of S2063: Calculate the cosine similarity between the basic category and the new category in the feature space; S2064: From the basic category Identify and identify new categories The base class mean with the highest cosine similarity and its corresponding covariance matrix Then model the basic category with the highest cosine similarity with the new category The characteristic distribution of S2065: From the basic category and new categories Sampling the features of the basic category and the new category, and obtaining the basic category feature S base and the new category feature S novel ; The basic category feature S base and the new category feature S novel The i-th sample feature in is expressed as and m represents the number of sampled features; S2066: Calculate basic category features S using the Earth Mover distance algorithm base and the new category feature S novel The optimal transmission scheme between S2067: Adjust the new category feature S through the optimal transmission scheme P novel , get the calibrated new category feature S′ novel ; The formula is expressed as; Where: Represents matrix multiplication; S2068: The calibrated new category feature S′ novel and roi feature map F roi Splice to get the enhanced roi feature map F roi_aug ; The formula is: F roi_aug =[F roi ,MLP(S′ novel )], Where: MLP represents a multi-layer perceptron consisting of a fully connected layer and an activation function.

8. The X-ray contraband detection method based on contour perception and optimal distribution migration as claimed in claim 7, characterized in that: In step S2061, the new category is calculated by the following formula The characteristic distribution of Where: Represents the roi feature map F roi The i-th foreground roi feature in; M represents the roi feature map F roi The number of foreground roi features in the image.

9. The X-ray contraband detection method based on contour perception and optimal distribution migration as claimed in claim 7, characterized in that: In step S2063, the cosine similarity between the basic category and the new category in the feature space is calculated by the following formula: Where: represents the cosine similarity between the mean of the jth basic category and the mean of the new category; d represents the number of basic categories.

10. The X-ray contraband detection method based on contour perception and optimal distribution migration as claimed in claim 7, characterized in that: In step S2066, the optimal transmission solution P is calculated by the following formula: Where: P ij From the basic category The i-th sampled feature is transferred to the new category The allocation ratio of the jth sampling feature in ; Represents and from the basic category The i-th sampled feature is transferred to the new category The distance cost matrix associated with the j-th sampled feature in ; ||·||2 represents -norm.

Images