A Security Inspection Method for Contraband Detection Based on Multi-Scale Attention and Data Augmentation

Abstract

This invention discloses a method for detecting contraband in security checks based on multi-scale attention and data augmentation. The method includes the following steps: constructing a contraband target detection model, including a backbone network; training the backbone network on a natural image classification dataset to obtain pre-trained weights; then training the contraband target detection model with the pre-trained weights of the backbone network on a natural image target detection dataset to obtain pre-trained weights for the contraband target detection model; initializing the contraband target detection model and retraining it on a security X-ray image dataset to obtain a trained contraband target detection model; adjusting the trained contraband target detection model, inputting the X-ray image to be detected, and obtaining the category and bounding box of the contraband. This invention can effectively detect the location of contraband targets in X-ray images with cluttered backgrounds and overlapping items, improving the efficiency and reliability of security checks.

Description

Technical Field

[0001] This invention belongs to the field of target detection, specifically a security inspection method for detecting contraband based on multi-scale attention and data augmentation. Background Technology

[0002] In large public transportation hubs with high population density, such as subway stations, train stations, and airports, X-ray security scanners are essential security equipment used to check passengers' luggage for prohibited items such as controlled knives, lighters, and firearms, ensuring public safety. However, in actual security checks, each piece of luggage requires a human security inspector to actively observe the X-ray image to determine if it contains prohibited items. But the number of prohibited items in real-world scenarios is often small, and security inspectors are prone to missing items due to fatigue or lack of concentration. Furthermore, prolonged repetitive work is detrimental to the physical and mental health of security inspectors. On the other hand, the complex background objects in the scanned images limit the inspection efficiency of security inspectors and can cause passenger congestion during peak travel periods. Therefore, automatic prohibited item detection algorithms have significant practical value.

[0003] The rapid development of deep learning algorithms has made convolutional neural network (CNN)-based computer vision algorithms mainstream tools in many scenarios such as image processing and visual understanding. Locating contraband from security X-ray images can be categorized as an object detection problem in computer vision. Currently, contraband detection algorithms all use CNN-based models, including two-stage algorithms that focus on improving accuracy and single-stage algorithms that focus on improving real-time performance. With the introduction of the Transformer architecture into the field of computer vision in recent years, DETR, as a new detection framework, treats object detection as a direct set prediction problem, eliminating the need for many manually designed components and simplifying the detection process. However, it has not been applied to solve the contraband detection problem.

[0004] In the field of contraband detection, the scarcity of publicly available high-quality datasets makes models prone to overfitting and poor robustness due to a lack of training data. To address this issue, data augmentation methods can be used to generate more fake samples, increasing training data and improving the model's generalization ability. CutOut randomly selects a region on an image and removes all pixels from that region, but this method can easily obscure key targets in the image, causing the loss of crucial features and affecting model learning. Mixup generates a new image by weighted fusion of two images, creating new mixed-class samples among different categories, smoothing the distribution of minority class samples, and improving the model's generalization performance. However, on datasets with severe aliasing, this method exacerbates background aliasing, limiting the improvement in model accuracy. CutMix replaces a region in the current image with a region from a different image, instead of the full-image blending of the Mixup method, mitigating background aliasing to some extent. However, the selected region is random, easily choosing non-target regions, leading to label mismatch of the target object and the cropping of key targets, resulting in the loss of crucial features.

[0005] The challenge of contraband detection lies in the penetrating power of X-rays and the random placement of objects in scanned luggage. This means that objects of different sizes in security X-ray images can easily overlap or obscure other items, causing interference from background noise. Existing solutions primarily utilize higher-level features with stronger semantic information to eliminate lower-level noise and introduce attention mechanisms to enhance low-level features such as edge contours and color. The CHR model attempts to insert back connections between different network layers, using higher-level features to provide higher-level supervision to lower-level features and eliminate background noise interference. However, its refinement function lacks explicit expression and focuses on classification problems. The DOAM module aims to make the model pay more attention to the contour shape and color texture features of the target, but this module only operates at the model input and does not consider multi-scale information of the target, resulting in limited effectiveness. The LIM module combines the above two ideas, using higher-level features with stronger semantic information to eliminate noise in lower-level features, while extracting boundary information in four directions to enhance the identifiable features of the target. However, its edge extraction method is simple, inaccurate, and prone to generating false information. Ding Jiangang et al. proposed a multi-scale feature coding network in their patent "A Method and System for Intelligent Detection of Foreign Objects Based on High-Resolution X-Ray Images". The network uses a hierarchical pyramid approach to fuse multi-scale features. However, it does not directly obtain multi-scale information during the feature extraction stage, which is not conducive to the refinement of local detail features. Furthermore, the channel attention and spatial attention mechanisms proposed in the backbone network of this patent cannot directly interact with multi-scale feature information, which is not conducive to filtering noise from background objects of different sizes. Summary of the Invention

[0006] To address the aforementioned issues, this invention provides a security inspection method for detecting contraband based on multi-scale attention and data augmentation. It utilizes the ObjectMix data augmentation method to accurately select target regions and fuse input images, improving the model's generalization ability to complex data. Furthermore, it designs an MSANet backbone network to effectively extract refined multi-scale features in a finer-grained manner. Simultaneously, a multi-scale attention mechanism is employed to highlight target region information and filter irrelevant background noise, thereby improving the accuracy of contraband detection.

[0007] The objective of this invention is achieved by at least one of the following technical solutions.

[0008] A method for detecting contraband in security inspections based on multi-scale attention and data augmentation is proposed. The ObjectMix data augmentation method is introduced to enrich the diversity of the data. An MSANet backbone network is designed to enhance the extraction capability of target feature information. Furthermore, an MSA-DETR model is constructed to detect contraband in X-ray security inspection images. The method includes the following steps:

[0009] S1. Construct the MSA-DETR contraband target detection model, including a data augmentation module, a backbone network, a location encoding module, a Transformer encoder-decoder, and a target detection head;

[0010] S2. Train the backbone network MSANet on the natural image classification dataset to obtain the pre-trained weights of MSANet. Then train the contraband target detection model MSA-DETR with the pre-trained weights of MSANet on the natural image target detection dataset to obtain the pre-trained weights of MSA-DETR.

[0011] S3. Adjust the target detection head of the contraband target detection model, load the pre-trained weights of MSA-DETR obtained in S2 to initialize the contraband target detection model, and then retrain the contraband target detection model with the pre-trained weights on the security X-ray image dataset to obtain the trained MSA-DETR model.

[0012] S4. Adjust the trained contraband target detection model, input the X-ray image to be detected, and obtain the category and bounding box of the contraband.

[0013] Further, in step S1, the data augmentation module uses the ObjectMix method to preprocess the input image data, as follows:

[0014] In a random region of an input image X1, all contraband target regions P in another input image X2 are extracted and fused with them multiple times according to a set ratio λ, and then used as input data. After obtaining the output of the contraband target detection model, the loss function of the contraband target detection model is calculated based on the fusion ratio λ. The calculation process is shown in the following formula:

[0015]

[0016]

[0017] Where P represents the set of all contraband target regions in image X2, M represents the set of image binary masks corresponding to the randomly fused regions in image X1, and P and M have the same number of elements; λ represents the fusion ratio obtained by random sampling, r is a random number, and the proportion of data augmentation samples is controlled by the hyperparameter R; ⊙ represents the element-wise multiplication of matrices. This indicates the summation at the fused region of X1; L1 represents the loss function of X1 before fusion. P This represents the loss function between the output of the object detection model and the bounding box information corresponding to the new object set P. This represents the loss function of the target detection model after fusion processing.

[0018] Further, in step S1, the backbone network MSANet first passes the input image through a 7×7 convolutional layer with a stride of 2 and a 3×3 max pooling layer, and then through a network layer composed of multiple basic residual blocks MSANet Blocks stacked together. When training the image classification task, it also needs to pass through a global average pooling layer, a fully connected layer and a softmax function to obtain the output of MSANet.

[0019] Each basic residual block MSANet Block consists of a 1×1 convolutional layer, a basic module MSA Module, and 1×1 convolutional layers connected in a residual manner. The basic module MSA Module consists of a multi-scale feature extraction module, a multi-scale channel attention module, and a multi-scale spatial attention module.

[0020] The input image size of the backbone network is H×W, and the output feature map sizes are H / 8×W / 8, H / 16×W / 16 and H / 32×W / 32, respectively. The smallest feature map is then subjected to a 3×3 convolution with a stride of 2, resulting in four image feature maps of different scales.

[0021] Furthermore, the multi-scale feature extraction module divides the input feature map X into S sub-feature maps x according to the channel dimension. i , where X∈R C×H×W , Then divide each sub-feature map x i Each feature map y is generated by passing it through a set of 3×3 convolutional layers and then merging the results of each convolutional layer.i ,in Then y i Including the multi-scale information of the first i-1 sub-feature maps after convolution, the calculation process is shown in the following formula:

[0022]

[0023] Among them, K i This represents the i-th sub-feature map x. i A set of 3×3 convolutional layers; to reduce the number of parameters in the multi-scale feature extraction module, the 3×3 convolution operation adopts grouped convolution.

[0024] Furthermore, the multi-scale channel attention module provides attention to each sub-feature map y output by the multi-scale feature extraction module. i First, global average pooling and max pooling are performed in the spatial dimension to obtain F. i c-avg and F i c-max To aggregate global spatial information, where F i c-avg , The interdependencies between channels are then captured using a shared multilayer perceptron, and the two are superimposed before the channel attention map is activated using a sigmoid function. in To achieve cross-channel information exchange between feature maps of different scales, attention maps of all channels are used. Perform softmax operation to obtain multi-scale channel attention map in This enables the contraband target detection model to adaptively select specific channel information from feature maps at specific scales, and then, based on... The channel weights in each feature map y i Recalibrate to obtain in The calculation process is shown in the following formula:

[0025]

[0026]

[0027]

[0028] Where MLP is a multilayer perceptron with one hidden layer, and σ represents the sigmoid function. This indicates element-wise multiplication of matrices;

[0029] Finally, the feature map The features are concatenated along the channel dimension to obtain the output feature map U of the multi-scale channel attention module. c as follows:

[0030]

[0031] Among them, U c ∈R C×H×W .

[0032] Furthermore, the multi-scale spatial attention module provides attention to each sub-feature map y output by the multi-scale feature extraction module. i First, global average pooling and max pooling are performed along the channel dimension to obtain... and To aggregate global channel information, where The two are then concatenated and passed through a deformable convolutional layer to capture the interdependencies between spatial locations, and the spatial attention map is activated using the sigmoid function. in To achieve cross-spatial information exchange between feature maps of different scales, all spatial attention maps... Perform softmax operation to obtain multi-scale spatial attention map in This enables the contraband target detection model to adaptively select specific spatial location information from feature maps at specific scales, and then, based on... The spatial location weights in each feature map y i Recalibrate to obtain in The calculation process is shown in the following formula:

[0033]

[0034]

[0035]

[0036] in, This is a deformable convolution with a kernel size of 7×7 to increase the receptive field and adapt to changes in the shape and size of the target. σ represents the sigmoid function. This indicates element-wise multiplication of matrices;

[0037] Finally, the feature map The features are concatenated along the channel dimension to obtain the output feature map U of the multi-scale spatial attention module. s as follows:

[0038]

[0039] Among them, U s ∈R C×H×W .

[0040] Furthermore, the output feature map U of the basic module MSA Module is derived from the output feature map U of the multi-scale channel attention module. c and the output feature map U of the multi-scale spatial attention module s The summation of corresponding elements is shown in the following formula:

[0041] U = U c +U s

[0042] Furthermore, in step S1, the overall detection process of the contraband target detection model MSA-DETR is as follows:

[0043] First, the input X-ray image is preprocessed through a data augmentation module, transforming part of the input image into an ObjectMix transformed image. Then, the processed image data is processed by the backbone network MSANet to extract multi-scale feature maps. Simultaneously, position encoding is performed on the multi-scale feature maps to represent the positional information in the feature maps. The multi-scale feature maps and their position encodings are then transformed into serialized data and input into a Transformer encoder-decoder structure, which outputs feature information of a fixed number of N target rectangles. The self-attention module of the encoder and the cross-attention module of the decoder use a multi-scale deformable attention module. Finally, the projection matrix in the target detection head decouples the feature information of the predicted N target rectangles into the corresponding categories and bounding box coordinates.

[0044] Furthermore, in step S4, the trained contraband target detection model is adjusted as follows:

[0045] Remove the data augmentation module from the trained contraband detection model to avoid introducing additional contraband into the image and affecting the detection of real contraband.

[0046] Compared with the prior art, the present invention has at least the following technical effects:

[0047] This invention addresses the characteristics of contraband in security X-ray images by proposing the ObjectMix data augmentation method to expand the sample data distribution and enhance the robustness of the model. It also designs an MSANet backbone network based on a multi-scale attention mechanism to strengthen the ability to extract multi-scale features of targets and identify contraband in complex backgrounds. This results in the construction of an MSA-DETR contraband target detection model, which can effectively locate contraband targets in cluttered backgrounds and overlapping items in X-ray images, improving the efficiency and reliability of security checks. Attached Figure Description

[0048] Figure 1 This is an overall flowchart of a security inspection method for detecting prohibited items based on multi-scale attention and data augmentation according to an embodiment of the present invention;

[0049] Figure 2 This is a schematic diagram of the MSANet Block in an embodiment of the present invention;

[0050] Figure 3 This is a schematic diagram of the multi-scale feature extraction module in an embodiment of the present invention;

[0051] Figure 4 This is a schematic diagram of the channel attention module in an embodiment of the present invention;

[0052] Figure 5 This is a schematic diagram of the multi-scale channel attention module in an embodiment of the present invention;

[0053] Figure 6 This is a schematic diagram of the spatial attention module in an embodiment of the present invention;

[0054] Figure 7 This is a schematic diagram of the multi-scale spatial attention module in an embodiment of the present invention. Detailed Implementation

[0055] To make the objectives, technical solutions, and technical effects of this invention clearer, the specific embodiments of this invention will be further described below with reference to the accompanying drawings. The described embodiments are only a part of the embodiments of this application, not all of them. All other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this invention.

[0056] This invention uses the PIDray dataset as the experimental dataset, which includes 47,677 images from three different security scanners. Each image includes the category of prohibited items and bounding box annotation information. The detected targets include 12 categories: guns, knives, wrenches, pliers, scissors, hammers, handcuffs, batons, pepper spray, power banks, lighters, and bullets. The test set is divided into three subsets according to the detection difficulty: Easy, Hard, and Hidden.

[0057] Example 1:

[0058] A method for detecting contraband in security inspections based on multi-scale attention and data augmentation is proposed. The method uses the ObjectMix data augmentation technique to enrich the diversity of the data, constructs an MSANet backbone network to enhance the extraction capability of target feature information, and further builds an MSA-DETR model to detect contraband in X-ray security inspection images. The method includes the following steps:

[0059] S1. Construct the MSA-DETR contraband target detection model, including a data augmentation module, a backbone network, a position encoding module, a Transformer encoder-decoder, and a target detection head, such as... Figure 1 As shown;

[0060] In this embodiment, the input image is first subjected to basic preprocessing operations such as random horizontal flipping, random scaling, random cropping, and normalization; the size of the input image is randomly scaled and adjusted so that the shortest side is 480 pixels and the longest side is 800 pixels, while the longest side is 1333 pixels.

[0061] The data augmentation module uses the ObjectMix method to further process the preprocessed input image data, as follows:

[0062] In a random region of an input image X1, all contraband target regions P in another input image X2 are extracted and fused with them multiple times according to a set ratio λ, and then used as input data. After obtaining the output of the contraband target detection model, the loss function of the contraband target detection model is calculated based on the fusion ratio λ. The calculation process is as follows:

[0063]

[0064] Where P represents the set of all contraband target regions in image X2, M represents the set of image binary masks corresponding to the fused random regions in image X1, and P and M have the same number of elements; λ represents the fusion ratio obtained by random sampling from the beta distribution Beta(β,β), r is a random number in [0,1), and the proportion of data augmentation samples is controlled by the hyperparameter R; ⊙ represents the element-wise multiplication of matrices. This indicates that the values ​​at the merged region X1 are added together.

[0065]

[0066] Where L1 represents the loss function of X1 before fusion processing, L P This represents the loss function between the output of the object detection model and the bounding box information corresponding to the new object set P. This represents the loss function of the object detection model after fusion processing; the loss function includes classification loss and bounding box regression loss.

[0067] To obtain the binary mask M of the image by sampling, the width r of the contraband target in P is used. w and high r h Multiple rectangular regions B = (r) are randomly selected in image X1. x ,r y ,r w ,rh As shown in the following formula:

[0068] r x ~Unif(0,Wr w )

[0069] r y ~Unif(0,Hr h )

[0070] Among them, (r x ,r y ) represents the coordinates of the top left corner of the rectangular region B, W and H represent the width and height of the image X1, and Unif(a,b) represents a uniform distribution.

[0071] The backbone network MSANet first passes the input image through a 7×7 convolutional layer with a stride of 2 and a 3×3 max pooling layer, and then through a network layer composed of multiple basic residual blocks MSANet Blocks stacked together. When training the image classification task, it also needs to pass through a global average pooling layer, a fully connected layer and a softmax function to obtain the output of MSANet.

[0072] Each basic residual block (MSANet Block) consists of a 1×1 convolutional layer, a basic MSA module, and another 1×1 convolutional layer connected in a residual manner, such as... Figure 2 As shown, the basic module MSA Module consists of a multi-scale feature extraction module, a multi-scale channel attention module, and a multi-scale spatial attention module;

[0073] The backbone network takes an input image of size H×W and outputs feature maps of sizes H / 8×W / 8, H / 16×W / 16, and H / 32×W / 32, respectively. After 1×1 convolution and 3×3 convolution with a stride of 2 on the smallest feature map, four image feature maps of different scales and 256 channels are finally obtained: H / 8×W / 8, H / 16×W / 16, H / 32×W / 32, and H / 64×W / 64.

[0074] The multi-scale feature extraction module divides the input feature map X into S sub-feature maps x according to the channel dimension. i , where X∈R C×H×W , Then divide each sub-feature map x i Each feature map y is generated by passing it through a set of 3×3 convolutional layers and then merging the results of each convolutional layer. i ,in Then y i Including multi-scale information from the first i-1 sub-feature maps after convolution, such as Figure 3 As shown, the calculation process is as follows:

[0075]

[0076] Among them, K i This represents the i-th sub-feature map x. i A set of 3×3 convolutional layers; to reduce the number of parameters in the multi-scale feature extraction module, the 3×3 convolution operation adopts grouped convolution.

[0077] In this embodiment, S = 4, and the number of groups in the 3×3 grouped convolution is 4.

[0078] The multi-scale channel attention module processes each sub-feature map y output by the multi-scale feature extraction module. i First, global average pooling and max pooling are performed in the spatial dimension to obtain F. i c-avg and F i c-max To aggregate global spatial information, where F i c-avg , The interdependencies between channels are then captured using a shared multilayer perceptron, and the two are superimposed before the channel attention map is activated using a sigmoid function. in like Figure 4 As shown; in order to achieve cross-channel information interaction between feature maps of different scales, attention maps of all channels are... Perform softmax operation to obtain multi-scale channel attention map in This enables the model to adaptively select specific channel information from feature maps at a specific scale, and then, based on... The channel weights in each feature map y i Recalibrate to obtain in like Figure 5 As shown, the calculation process is as follows:

[0079]

[0080]

[0081]

[0082] Here, MLP is a multilayer perceptron that includes a hidden layer with C / r nodes and applies the ReLU activation function after the hidden layer. The hyperparameter r is used to control the reduction of the number of parameters, and σ represents the sigmoid function. This indicates element-wise multiplication of matrices;

[0083] Finally, the feature map The features are concatenated along the channel dimension to obtain the output feature map U of the multi-scale channel attention module. c as follows:

[0084]

[0085] Among them, U c ∈R C×H×W .

[0086] The multi-scale spatial attention module processes each sub-feature map y output by the multi-scale feature extraction module. i First, global average pooling and max pooling are performed along the channel dimension to obtain... and To aggregate global channel information, where The two are then concatenated and passed through a deformable convolutional layer to capture the interdependencies between spatial locations, and the spatial attention map is activated using the sigmoid function. in like Figure 6 As shown; in order to achieve cross-spatial information interaction between feature maps of different scales, all spatial attention maps are... Perform softmax operation to obtain multi-scale spatial attention map in This enables the model to adaptively select specific spatial location information from feature maps at a specific scale, and then, based on... The spatial location weights in each feature map y i Recalibrate to obtain in like Figure 7 As shown, the calculation process is as follows:

[0087]

[0088]

[0089]

[0090] in, This is a deformable convolution with a kernel size of 7×7 to increase the receptive field and adapt to changes in the shape and size of the target. σ represents the sigmoid function. This indicates element-wise multiplication of matrices;

[0091] Deformable convolution operation at each sampling position p+p of the convolution kernel k In addition, learn an offset Δp k To adapt to changes in the target's shape and size, a modulation amount Δm is introduced.k To adjust the input feature amplitude at different spatial locations, the calculation process is as follows:

[0092]

[0093] Where K represents the number of sampling points of the convolution kernel, x(p+p k +Δp k ) represents the position p+p k +Δp k The input feature value, y(p) represents the output feature value at position p, w k represents the kernel weights for the k-th sampling point.

[0094] Offset Δp k and modulation amount Δm k This is achieved by passing the input feature map through a regular convolutional layer. The output feature map of this convolutional layer has the same spatial size as the input feature map, and the number of output channels is 3K, where the first 2K channels correspond to K two-dimensional offsets Δp. k The last K channels are input to the sigmoid layer to obtain Δm. k Therefore, Δm k ∈(0,1). Due to Δp k It is usually a non-integer when calculating x(p+p) k +Δp k When using bilinear interpolation, the following method is employed.

[0095] Finally, the feature map The features are concatenated along the channel dimension to obtain the output feature map U of the multi-scale spatial attention module. s as follows:

[0096]

[0097] Among them, U s ∈R C×H×W .

[0098] The output feature map U of the basic module MSA Module is derived from the output feature map U of the multi-scale channel attention module. c and the output feature map U of the multi-scale spatial attention module s The summation of corresponding elements is obtained as follows:

[0099] U = U c +U s

[0100] The overall detection process of the MSA-DETR contraband target detection model is as follows:

[0101] First, the input X-ray image is preprocessed through a data augmentation module, transforming part of the input image into an ObjectMix transformed image. Then, the processed image data is processed by the backbone network MSANet to extract multi-scale feature maps. Simultaneously, position encoding is performed on the multi-scale feature maps to represent the positional information in the feature maps. The multi-scale feature maps and their position encodings are then transformed into serialized data and input into a Transformer encoder-decoder structure, which outputs feature information of a fixed number of N target rectangles. The self-attention module of the encoder and the cross-attention module of the decoder use a multi-scale deformable attention module. Finally, the projection matrix in the target detection head decouples the feature information of the predicted N target rectangles into the corresponding categories and bounding box coordinates.

[0102] In this embodiment, the position encoding module uses sine and cosine functions to perform fixed two-dimensional encoding on each position of the feature map. The encoding length is 256, with the first 128 dimensions and the last 128 dimensions representing the position encoding in the x and y directions, respectively.

[0103] The multi-scale deformable attention module used in the Transformer encoder-decoder assigns only a small number of keys to each query on the multi-scale feature map to alleviate the problems of slow model convergence and high computational complexity. The calculation process is shown in the following formula:

[0104]

[0105] in, This represents input feature maps at L scales. The z represents the normalized coordinates of the reference point for the query element q. q For query features, K represents the number of samples for the key, M represents the number of attention heads, and A mlqk and Δp mlqk These are the attention weights and the sampling offset, respectively, derived from z. q The function obtained by performing a linear matrix transformation Normalized coordinates Scale to the feature map of layer l.

[0106] In this embodiment, L=4, K=4, M=8, the encoder and decoder both have 6 layers, and the number of target queries is 300.

[0107] The object detection head predicts the class through a fully connected layer and the bounding box coordinates through a three-layer perceptron. The softmax function is used when predicting the class label, and an additional label is used to represent the background class bounding box. The predicted bounding box coordinates are the normalized center coordinates, height, and width relative to the input image. In this embodiment, the number of bounding boxes predicted by the detection head is 300.

[0108] S2. Train the backbone network MSANet on the natural image classification dataset to obtain the pre-trained weights of MSANet. Then train the contraband target detection model MSA-DETR with the pre-trained weights of MSANet on the natural image target detection dataset to obtain the pre-trained weights of MSA-DETR.

[0109] In this embodiment, to reduce overfitting and better leverage the powerful global modeling capabilities of the Transformer structure, the backbone network is first pre-trained on the ImageNet dataset, and then the entire detection model is pre-trained on the COCO 2017 dataset. Before calculating the loss function, the Hungarian binary matching algorithm is used to obtain a one-to-one matching relationship between the predicted bounding boxes and the ground truth boxes. The Focal Loss loss function is used for the classification loss of the bounding boxes to alleviate the imbalance between positive and negative samples, and L1 Loss and GIoU Loss loss functions are introduced to regress the bounding box coordinates. The overall loss function of the detection model is the weighted sum of the three loss functions, with weights of 1, 5, and 2, respectively.

[0110] S3. Adjust the target detection head of the contraband target detection model, change the number of categories to 12, load the pre-trained weights of MSA-DETR obtained in S2 to initialize the contraband target detection model, and then retrain the contraband target detection model after loading the pre-trained weights on the security X-ray image dataset PIDray to obtain the trained MSA-DETR model.

[0111] S4. Adjust the trained contraband detection model by removing the data augmentation module to avoid introducing additional contraband into the image and affecting the detection of real contraband. Input the X-ray image to be detected to obtain the category and bounding box of the contraband.

[0112] Example 2:

[0113] In this embodiment, unlike Embodiment 1, to accelerate the convergence speed during the training of the detection model, a denoising task is added to the decoder query in the Transformer decoder module, in addition to the binary matching task in Embodiment 1. The input of this task is a noisy ground truth box, and the output is a reconstructed ground truth box after removing the noise.

[0114] Specifically, the denoising task adds random noise to the category labels and bounding box coordinates of all ground truth boxes in each input image. To improve denoising efficiency, multiple sets of random noise are added to each ground truth box to obtain multiple noisy ground truth boxes. This task adds noise to the bounding box coordinates using two methods: center point movement and bounding box scaling. λ1 and λ2 are defined as scale factors for these two types of noise, where λ1,λ2∈(0,1). Center point movement adds a random offset (Δx,Δy) to the center point of the bounding box, where... w and h are the width and height of the bounding box, ensuring that the coordinates of the moved center point remain within the original bounding box. Bounding box scaling involves randomly sampling a width and height value within [(1-λ2)w,(1+λ2)w] and [(1-λ2)h,(1+λ2)h] respectively, and then adjusting the bounding box size accordingly. For class labels, this task randomly flips the ground truth box labels to other labels at a ratio γ. This label flipping forces the model to predict the true labels based on the noisy bounding boxes, thus better capturing the relationship between labels and bounding boxes. In this embodiment, λ1 = 0.4, λ2 = 0.4, γ = 0.2, and the number of noisy ground truth box groups is 5.

[0115] Similar to the loss function of the MSA-DETR contraband detection model, the reconstruction loss function for the denoising task uses Focal Loss, L1 Loss, and GIoU Loss. Furthermore, attention masks are used to prevent information leakage between the denoised and matching parts, and between different denoised groups. During inference in the contraband detection model, the denoising task is removed, retaining only the matching task to avoid additional computational overhead.

[0116] Example 3:

[0117] In this embodiment, unlike in Embodiment 1, the bounding box regression loss function is replaced with Smooth L1 Loss and CIoU Loss to improve the accuracy of the target regression.

[0118] The formula for calculating Smooth L1 Loss is as follows:

[0119]

[0120] Where x represents the difference between the coordinates of the predicted box and the true box.

[0121] In the early stages of training, the difference between the predicted bounding box and the ground truth box is large. Smooth L1 Loss can limit the gradient magnitude, avoid gradient explosion, and is less affected by outliers, making the model more robust. In the later stages of training, the difference between the predicted bounding box and the ground truth box is small. Compared with L1 Loss, the function is smoother near the zero point and the gradient is smaller, allowing the model to converge to higher accuracy.

[0122] The formula for calculating CIoU Loss is as follows:

[0123]

[0124]

[0125]

[0126] Wherein, ρ(b,b) gt () indicates the center point b of the prediction box and the center point b of the truth box. gt The Euclidean distance between the two bounding boxes is given by c, where c represents the diagonal length of the minimum bounding rectangle of the two bounding boxes, α is an adjustment factor, and v is used to measure the aspect ratio of the predicted bounding box. Aspect Ratio of Truth Box Consistency.

[0127] Compared to GIoU Loss, CIoU Loss considers the overlap area between the predicted and ground truth boxes, the center distance, and the aspect ratio simultaneously, significantly improving convergence speed and making bounding box regression more accurate. Specifically, by adjusting the factor α, the overlap area has a higher priority in the regression.

Claims

1. A method for detecting prohibited items in security checks based on multi-scale attention and data augmentation, characterized in that, Includes the following steps: S1. Construct a contraband target detection model MSA-DETR, including a data augmentation module, a backbone network, a position encoding module, a Transformer encoder-decoder, and a target detection head; the backbone network is MSANet, and its basic module is MSAModule; the MSAModule includes a multi-scale feature extraction module, a multi-scale channel attention module, and a multi-scale spatial attention module; The multi-scale feature extraction module will input the feature map. Divide into S sub-feature maps according to the channel dimension. Then, each sub-feature map Through a group The convolutional layers are then stacked and fused together to output S feature maps. ; The multi-scale channel attention module processes each sub-feature map output by the multi-scale feature extraction module. First, global average pooling and max pooling are performed in the spatial dimension to obtain... and The system aggregates global spatial information, then captures the interdependencies between channels through a shared multilayer perceptron. The two are then superimposed and activated using a sigmoid function to activate the channel attention map. To achieve cross-channel information interaction between feature maps of different scales, attention maps of all channels are processed. Perform softmax operation to obtain multi-scale channel attention map This enables the contraband target detection model to adaptively select specific channel information from feature maps at specific scales, and then, based on... The channel weights in each feature map Recalibrate to obtain feature map ; Finally, the feature map The features are concatenated along the channel dimension to obtain the output feature map of the multi-scale channel attention module. ; The multi-scale spatial attention module processes each sub-feature map output by the multi-scale feature extraction module. First, global average pooling and max pooling are performed along the channel dimension to obtain... and By aggregating global channel information, concatenating the two, and then using a deformable convolutional layer to capture the interdependencies between spatial locations, a sigmoid function is used to activate the spatial attention map. ; S2. Train the backbone network MSANet on the natural image classification dataset to obtain the pre-trained weights of MSANet. Then train the contraband target detection model MSA-DETR with the pre-trained weights of MSANet on the natural image target detection dataset to obtain the pre-trained weights of MSA-DETR. S3. Adjust the target detection head of the contraband target detection model, load the pre-trained weights of MSA-DETR obtained in S2 to initialize the contraband target detection model, and then retrain the contraband target detection model with the pre-trained weights on the security X-ray image dataset to obtain the trained MSA-DETR model. S4. Remove the data augmentation module from the trained contraband target detection model, input the X-ray image to be detected into the trained contraband target detection model, and obtain the category and bounding box of the contraband.

2. The security inspection method for detecting prohibited items based on multi-scale attention and data augmentation according to claim 1, characterized in that, In step S1, the data augmentation module uses the ObjectMix method to preprocess the input image data, as follows: In an input image Extract another input image from a random region. All prohibited items target areas Rather than according to the set ratio Multiple fusions are used as input data And after obtaining the output of the contraband target detection model, based on the fusion ratio The magnitude of the value is used to calculate the loss function of the contraband target detection model. The calculation process is shown in the following formula: in, Representing an image A collection of all prohibited items target areas in the country. This indicates that the image corresponds to the image. A set of binary masks for the merged random regions of the image. and The number of elements in them is the same; This indicates the fusion ratio obtained from random sampling. For a random number, via hyperparameters To control the proportion of data augmentation samples; This indicates element-wise multiplication of matrices. Indicates in The locations of the merged regions are added together; express The loss function before fusion processing This represents the output of the object detection model and the new set of objects. The loss function between the corresponding bounding box information This represents the loss function of the target detection model after fusion processing.

3. The security inspection method for detecting prohibited items based on multi-scale attention and data augmentation according to claim 1, characterized in that, In step S1, the backbone network MSANet first passes the input image through a 7×7 convolutional layer with a stride of 2 and a 3×3 max pooling layer, and then through a network layer composed of multiple basic residual blocks MSANet Blocks stacked together. When training the image classification task, it also needs to pass through a global average pooling layer, a fully connected layer and a softmax function to obtain the output of MSANet. Each basic residual block MSANet Block consists of a 1×1 convolutional layer, a basic module MSA Module, and 1×1 convolutional layers connected in a residual manner. The basic module MSA Module consists of a multi-scale feature extraction module, a multi-scale channel attention module, and a multi-scale spatial attention module. The input image size of the backbone network is The output feature map sizes are respectively , and Then, perform a step size of 2 on the feature map with the smallest size. Convolution results in four image feature maps at different scales.

4. The security inspection method for detecting prohibited items based on multi-scale attention and data augmentation according to claim 3, characterized in that, The multi-scale feature extraction module will input the feature map. Divide into S sub-feature maps according to the channel dimension. ,in Then, each sub-feature map Through a group The convolutional layers are then stacked and fused together to output S feature maps. ,in ,but Including the multi-scale information of the first i-1 sub-feature maps after convolution, the calculation process is shown in the following formula: in, This represents the sub-feature map corresponding to the i-th sub-feature map. A group Convolutional layers; to reduce the number of parameters in the multi-scale feature extraction module, The convolution operation uses grouped convolution.

5. The security inspection method for detecting prohibited items based on multi-scale attention and data augmentation according to claim 4, characterized in that, The multi-scale channel attention module processes each sub-feature map output by the multi-scale feature extraction module. First, global average pooling and max pooling are performed in the spatial dimension to obtain... and To aggregate global spatial information, in which Then, the interdependencies between channels are captured by a shared multilayer perceptron, and the two are superimposed and activated by a sigmoid function to activate the channel attention map. ,in To enable cross-channel information exchange between feature maps of different scales, attention maps for all channels are processed. Perform softmax operation to obtain multi-scale channel attention map ,in This enables the contraband target detection model to adaptively select specific channel information from feature maps at specific scales, and then, based on... The channel weights in each feature map Recalibrate to obtain ,in The calculation process is shown in the following formula: MLP stands for Multilayer Perceptron, which includes one hidden layer. This represents the sigmoid function. This indicates element-wise multiplication of matrices; Finally, the feature map The features are concatenated along the channel dimension to obtain the output feature map of the multi-scale channel attention module. as follows: in, .

6. The security inspection method for detecting prohibited items based on multi-scale attention and data augmentation according to claim 4, characterized in that, The multi-scale spatial attention module processes each sub-feature map output by the multi-scale feature extraction module. First, global average pooling and max pooling are performed along the channel dimension to obtain... and To aggregate global channel information, where The two are then concatenated and passed through a deformable convolutional layer to capture the interdependencies between spatial locations, and the spatial attention map is activated using the sigmoid function. ,in .

7. The security inspection method for detecting prohibited items based on multi-scale attention and data augmentation according to claim 6, characterized in that, To achieve cross-spatial information exchange between feature maps of different scales, all spatial attention maps... Perform softmax operation to obtain multi-scale spatial attention map ,in This enables the contraband target detection model to adaptively select specific spatial location information from feature maps at specific scales, and then, based on... The spatial location weights in each feature map Recalibrate to obtain ,in The calculation process is shown in the following formula: in, The kernel size is Deformable convolutions are used to increase the receptive field and adapt to changes in the shape and size of the target. This represents the sigmoid function. This indicates element-wise multiplication of matrices; Finally, the feature map The feature maps of the multi-scale spatial attention module are concatenated along the channel dimension. as follows: in, .

8. A method for detecting prohibited items in security checks based on multi-scale attention and data augmentation according to any one of claims 3 to 7, characterized in that, The output feature map of the basic module MSA Module Output feature map of multi-scale channel attention module and the output feature map of the multi-scale spatial attention module The summation of corresponding elements is shown in the following formula: 。 9. The security inspection method for detecting prohibited items based on multi-scale attention and data augmentation according to claim 1, characterized in that, In step S1, the overall detection process of the contraband target detection model MSA-DETR is as follows: First, the input X-ray image is preprocessed through a data augmentation module, transforming part of the input image into an ObjectMix transformed image. Then, the processed image data is processed by the backbone network MSANet to extract multi-scale feature maps. Simultaneously, position encoding is performed on the multi-scale feature maps to represent the positional information in the feature maps. The multi-scale feature maps and their position encodings are then transformed into serialized data and input into a Transformer encoder-decoder structure, which outputs feature information of a fixed number of N target rectangles. The self-attention module of the encoder and the cross-attention module of the decoder use a multi-scale deformable attention module. Finally, the projection matrix in the target detection head decouples the feature information of the predicted N target rectangles into the corresponding categories and bounding box coordinates.

Images