X-ray security image contraband detection method

Abstract

The present application relates to a kind of X-ray security image detection method of contraband, belong to computer vision target detection and identification technical field.The method includes the following steps: S1: obtaining security data set, the security data set obtained is preprocessed, and the data set after pre-processing is randomly divided into training set and verification set according to proportion;S2: WHIoU Loss and attention feature fusion module are used to build WHLAFF-YOLO model;S3: the training set in S1 is input into the WHLAFF-YOLO model of S2 construction, and the model is trained, and the WHLAFF-YOLO model of training completion is obtained;S4: using the WHLAFF-YOLO model of S3 training completion, the X-ray image containing contraband collected in security process is detected for contraband.The present application guarantees detection speed, and reaches higher detection precision at the same time.

Description

Technical Field

[0001] This invention belongs to the field of computer vision target detection and recognition technology, and involves deep learning, target detection and X-ray security inspection image processing technology. Background Technology

[0002] To maintain public transportation safety, the use of X-ray security scanners to conduct security checks on passengers' luggage has become an important means.

[0003] X-rays are electromagnetic waves capable of penetrating objects. By detecting density differences within an object, they can identify contraband or dangerous items hidden inside. During security checks, X-ray scanners emit high-energy X-ray beams, which are received by detectors after passing through the object and converted into X-ray images. These images clearly show the internal structure and shape of the object, providing important reference information for security personnel.

[0004] Currently, the mainstream security screening method involves security personnel observing the shape, density, and structure of objects in X-ray images in real time to determine the presence of prohibited items. However, with the expansion of transportation systems and the increase in passenger flow, this manual inspection method can no longer meet the demands. Finding a cost-effective and efficient way to automatically detect prohibited items using security images while reducing the workload of security personnel has become a new challenge and requirement.

[0005] To address this issue, early researchers used machine learning for automatic identification of contraband. However, due to significant visual differences between X-ray images and natural light images, as well as issues such as the haphazard placement and severe overlap of items in security check scenarios, it is difficult to extract effective features of contraband, resulting in insufficient detection accuracy of existing target detection algorithms.

[0006] With the rapid development of deep learning, extracting image features using convolutional neural networks has become mainstream. Deep learning-based algorithms for automatic detection of contraband items have also developed rapidly. These algorithms can automatically identify and process objects in X-ray images, extract useful feature information, and determine the presence of contraband items. However, due to the complexity and diversity of X-ray images, as well as various interference factors in security inspection scenarios, these algorithms still face problems of low detection accuracy and low efficiency. Summary of the Invention

[0007] In view of this, the purpose of the present invention is to provide a method for detecting prohibited items in X-ray security inspection images, thereby solving the problems existing in the prior art.

[0008] To achieve the above objectives, the present invention provides the following technical solution:

[0009] A method for detecting prohibited items in X-ray security images, the method comprising the following steps:

[0010] S1: Obtain the security check dataset, preprocess the obtained security check dataset, and randomly divide the preprocessed dataset into training set and validation set according to the proportion;

[0011] S2: Replace CIoU Loss with WHIoU Loss as the new bounding box loss function, introduce the attention feature fusion module AFFM to improve the feature fusion method of the neck network in the YOLOv7 network model, and construct the WHLAFF-YOLO model, a network for detecting prohibited items in security inspection images.

[0012] S3: Input the training set in S1 into the WHLAFF-YOLO model constructed in S2, train the model, and obtain the trained WHLAFF-YOLO model.

[0013] S4: Using the WHLAFF-YOLO model trained in S3, detect prohibited items in X-ray images collected during security checks.

[0014] Furthermore, in step S1, the acquired security inspection dataset is preprocessed, specifically including: deleting images with a small number of sample categories from the acquired security inspection dataset.

[0015] Furthermore, in step S1, the preprocessed dataset is randomly divided into a training set and a validation set in a 9:1 ratio.

[0016] Furthermore, in S2, WHIoU Loss is introduced to address the issue that the aspect ratio penalty function in CIoU Loss loses its constraint effect when the width and height of the predicted bounding box and the true bounding box have the same linear proportional relationship. The calculation method is expressed as follows:

[0017]

[0018] Where IoU is the intersection-union ratio between the predicted bounding box and the ground truth bounding box, ρ 2 (b,b gt ) is the Euclidean distance between the center point of the predicted bounding box and the center point of the ground truth bounding box, c is the diagonal distance of the smallest bounding rectangle that simultaneously contains both the predicted bounding box and the ground truth bounding box, and w max and w min These are the larger and smaller values ​​of the widths of the predicted bounding box and the ground truth bounding box, respectively, h. max and h min These are the larger and smaller values ​​of the predicted bounding box and the ground truth bounding box, respectively.

[0019] Furthermore, in the S2 attention feature fusion module AFFM, when fusing shallow and deep features, it assigns a spatial attention (SA) module that focuses more on shallow details of the target to shallow features, and a channel attention (CA) module that focuses more on deep semantic information of the target to deep features. The spatial attention (SA) module performs average pooling and max pooling on the input feature map in the dimensional direction to aggregate feature information, and then generates a spatial attention map through a convolutional layer, assigning a new attention level to each pixel of the feature map. The channel attention (CA) module performs global average pooling and global max pooling on the input feature map to obtain a feature map of size 1×1×C with a global perceptual field, aggregates global context information, and captures the dependencies between channels through two fully connected layers. Finally, the attention feature fusion module AFFM performs deep fusion on the separately processed feature layers in a dimensional superposition manner, fully preserving all feature information of the object, and outputting a feature map that combines detailed information and semantic information.

[0020] Furthermore, in S2, the WHLAFF-YOLO model includes an image preprocessing module, a backbone network, a neck network, and a multi-layer detection head;

[0021] The image preprocessing module uses Mosaic and Mixup data augmentation strategies to augment the training data and then input it into the backbone network to extract its features.

[0022] The backbone network consists of CBS modules, ELAN modules, MP modules, and SPPCSPC modules;

[0023] The CBS module integrates convolutional layers, batch normalization layers, and activation functions; the ELAN module enables the model to learn more information and has stronger robustness by stacking feature layers from different gradient paths; the MP module contains two branches, namely a max pooling branch and a convolutional branch, which can retain the feature information of the target object to the maximum extent during downsampling; the SPPC module uses pooling kernels of different sizes to adapt the model to images of different resolutions and increase the receptive field.

[0024] The backbone network extracts features, and the extracted features are then fed into the neck network. A feature pyramid structure is used for bottom-up upsampling, and the feature layers from the backbone network are fused using AFFM. Next, a path aggregation network is used for top-down downsampling, and the feature layer information output from the feature pyramid is fused using AFFM. The fused feature layers are then fed into a multi-layer detection head, which contains three detection heads designed for different input sizes to detect large, medium, and small targets, respectively. The multi-layer detection head uses WHIoU Loss to measure the difference between the model's prediction and the true label, and then outputs the detection results.

[0025] The beneficial effects of this invention are as follows:

[0026] First, by using WhioU Loss as a new bounding box loss function and introducing the attention feature fusion module AFFM to improve the YOLOv7 network model, this invention can more effectively detect and locate prohibited items, thereby improving detection accuracy and efficiency.

[0027] Secondly, by using a multi-layer detection head and WHIoU Loss to measure the degree of difference between the model prediction results and the true labels, this invention can more accurately identify the types of prohibited items in the image and pinpoint their location, thereby reducing the probability of missed detections and false detections.

[0028] Third, the spatial attention module in this invention can capture information about direction perception and position perception, and better learn the details of the target so as to direct the network's attention to more detailed spatial location information.

[0029] Fourth, the channel attention module in this invention uses multiple receptive fields to weight depth features in order to capture highly discriminative channel features.

[0030] Fifth, the detection method of the present invention not only makes the predicted bounding box closer to the real bounding box, but also detects prohibited items that the original model missed, and the confidence level is significantly improved compared with the original model.

[0031] Other advantages, objectives, and features of the invention will be set forth in part in the description which follows, and in part will be apparent to those skilled in the art from the following examination, or may be learned from practice of the invention. The objectives and other advantages of the invention can be realized and obtained through the following description. Attached Figure Description

[0032] To make the objectives, technical solutions, and advantages of the present invention clearer, the preferred embodiments of the present invention will be described in detail below with reference to the accompanying drawings, wherein:

[0033] Figure 1This is a flowchart of the method in this invention;

[0034] Figure 2 This is a diagram of the overall network structure in this invention;

[0035] Figure 3 This is a network structure diagram of the attention fusion module in this invention;

[0036] Figure 4 The detection results using the original YOLOv7 network model are shown.

[0037] Figure 5 This is a diagram showing the detection performance of the WHLAFF-YOLO network model proposed in this invention. Detailed Implementation

[0038] The following specific examples illustrate the implementation of the present invention. Those skilled in the art can easily understand other advantages and effects of the present invention from the content disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and various details in this specification can be modified or changed based on different viewpoints and applications without departing from the spirit of the present invention. It should be noted that the illustrations provided in the following embodiments are only schematic representations of the basic concept of the present invention. Unless otherwise specified, the following embodiments and features can be combined with each other.

[0039] The accompanying drawings are for illustrative purposes only and are schematic diagrams, not actual pictures. They should not be construed as limiting the invention. To better illustrate the embodiments of the invention, some parts in the drawings may be omitted, enlarged, or reduced, and do not represent the actual product dimensions. It is understandable to those skilled in the art that some well-known structures and their descriptions may be omitted in the drawings.

[0040] In the accompanying drawings of the embodiments of the present invention, the same or similar reference numerals correspond to the same or similar components. In the description of the present invention, it should be understood that if terms such as "upper," "lower," "left," "right," "front," and "rear" indicate the orientation or positional relationship based on the orientation or positional relationship shown in the drawings, they are only for the convenience of describing the present invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, the terms used to describe positional relationships in the drawings are only for illustrative purposes and should not be construed as limiting the present invention. For those skilled in the art, the specific meaning of the above terms can be understood according to the specific circumstances.

[0041] Please see Figures 1-5 This is a method for detecting prohibited items using X-ray security images.

[0042] like Figure 1As shown, the method includes the following steps:

[0043] Step 1: Download the publicly available X-ray security inspection dataset SIXray. Since the number of images containing hammers in the SIXray dataset is small, it will cause an imbalance in the number of samples between classes. Therefore, the images containing hammers are deleted, and the adjusted SIXray dataset is divided into training set and validation set.

[0044] In this step, the adjusted SIXray dataset contains 8909 X-ray images and corresponding label information, covering five categories of prohibited items: guns, knives, scissors, wrenches, and pliers. The adjusted SIXray dataset is randomly divided into training and validation sets at a 9:1 ratio. The training set, consisting of 90% of the adjusted SIXray dataset, will be used to train the WHLAFF-YOLO security image prohibited item detection network model. The validation dataset, consisting of 10% of the adjusted SIXray dataset, will be used to validate the WHLAFF-YOLO security image prohibited item detection network model, and the validation results will be used as the model's performance metric.

[0045] Step 2: Replace CIoU Loss with WHIoU Loss as the new bounding box loss function, and improve the feature fusion method of the neck network in the YOLOv7 network model by introducing the attention feature fusion module AFFM, and construct the WHLAFF-YOLO security inspection image prohibited item detection network model.

[0046] In this step, such as Figure 2As shown, the WHLAFF-YOLO security image prohibited item detection network model consists of four parts: an image preprocessing module, a backbone network, a neck network, and a multi-layer detection head. The image preprocessing module uses Mosaic and Mixup data augmentation strategies to augment the training data before inputting it into the backbone network. The backbone network comprises a CBS module, an ELAN module, an MP module, and a SPPCSPC module, possessing powerful feature extraction capabilities. The CBS module integrates convolutional layers, batch normalization layers, and activation functions. The ELAN module, by stacking feature layers from different gradient paths, enables the model to learn more information and possess stronger robustness. The MP module contains two branches: a max-pooling branch and a convolutional branch, maximizing the retention of target item feature information during downsampling. The SPPCSPC module uses pooling kernels of different sizes to adapt the model to images of different resolutions, increasing the receptive field. The backbone network extracts features, and the extracted features are then fed into the neck network. A feature pyramid structure is used for bottom-up upsampling, and the feature layers from the backbone network are fused using AFFM. Next, a path aggregation network is used for top-down downsampling, and the feature layer information output from the feature pyramid is fused using AFFM. The fused feature layers are then fed into a multi-layer detection head, which contains three detection heads designed for different input sizes to detect large, medium, and small targets, respectively. The multi-layer detection head uses WHIoU Loss to measure the difference between the model's prediction and the true label, and then outputs the detection results.

[0047] In this step, WHIoU Loss is introduced to address the issue in CIoU Loss where the aspect ratio penalty function loses its constraint effect when the width and height of the predicted bounding box have the same linear proportional relationship with the width and height of the ground truth bounding box. The calculation method specifically consists of the following:

[0048]

[0049] In formula (1), IoU is the intersection-union ratio between the predicted bounding box and the ground truth bounding box, ρ 2 (b,b gt ) is the Euclidean distance between the center point of the predicted bounding box and the center point of the ground truth bounding box, c is the diagonal distance of the smallest bounding rectangle that simultaneously contains both the predicted bounding box and the ground truth bounding box, and w max and w min These are the larger and smaller values ​​of the widths of the predicted bounding box and the ground truth bounding box, respectively, h. max and h min These are the larger and smaller values ​​of the predicted bounding box and the ground truth bounding box, respectively.

[0050] In this step, please refer to Figure 3 The attention feature fusion module AFFM, when fusing shallow and deep features, assigns a Spatial Attention (SA) module to shallow features, which focuses more on shallow details of the target, and a Channel Attention (CA) module, which focuses more on deep semantic information of the target, to deep features. The Spatial Attention module captures orientation and positional awareness information, better learns target details, and directs the network's attention to more detailed spatial location information. It first aggregates feature information by performing average pooling and max pooling on the input feature map along its dimensional direction, and then generates a spatial attention map through convolutional layers, assigning new attention to each pixel in the feature map. The Channel Attention module uses multiple receptive fields to weight deep features to capture highly discriminative channel features. It first performs global average pooling and global max pooling on the input feature map to obtain a feature map of size 1×1×C with a global perceptual field, aggregates global contextual information, and then captures the dependencies between channels through two fully connected layers. Finally, the processed feature layers are deeply fused by stacking dimensions to fully preserve all feature information of the object, and finally output a feature map that combines detailed information and semantic information.

[0051] Step 3: Use the training set images obtained in Step 1 to train the WHLAFF-YOLO security image prohibited item detection network model.

[0052] In this step, training the WHLAFF-YOLO security image contraband detection network model specifically involves: using a Core™ i9-10920X processor, a GeForce RTX 3090 graphics card, and the torch 1.8.0 deep learning framework. The experiment employs a transfer learning strategy, using pre-trained weights on the PascalVOC2012 dataset. The input image size for the training set is adjusted to a fixed resolution of 640×640. The training period during the frozen phase is set to 50 epochs with a batch size of 16, while the training period during the unfrozen phase is set to 150 epochs with a batch size of 8. The maximum learning rate is set to 0.01, and the SGD stochastic gradient descent algorithm is used for optimization. Cosine annealing is used to decrease the learning rate. Finally, the trained weights of the WHLAFF-YOLO security image contraband detection network model are obtained.

[0053] Step 4: Use the images from the validation set obtained in Step 1 to validate the trained WHLAFF-YOLO network model obtained in Step 3.

[0054] In this step, to comprehensively evaluate the performance of the WHLAFF-YOLO security image prohibited item detection network model in automatically detecting prohibited items, Average Precision (AP), mean Average Precision (mAP), and frames per second (FPS) are selected as evaluation metrics. AP, calculated by comprehensively considering precision and recall under a fixed IoU threshold, reflects the model's detection accuracy for a single category. mAP is obtained by averaging the AP values ​​across all categories, reflecting the model's detection accuracy across all categories. FPS refers to the number of images that can be detected per second, reflecting the model's detection speed. Precision is the proportion of correctly predicted positive samples out of all predicted samples, and recall is the proportion of correctly predicted positive samples out of all positive samples. The formulas for both are as follows:

[0055]

[0056]

[0057] Where TP is the number of samples that the classifier predicts as positive and which are actually positive, i.e., the number of positive samples correctly identified. FP is the number of samples that the classifier predicts as positive but are actually negative, i.e., the number of falsely detected negative samples. FN is the number of samples that the classifier predicts as negative but are actually positive, i.e., the number of missed positive samples. The formulas for calculating AP and mAP are as follows:

[0058]

[0059]

[0060] Where P represents accuracy, R represents recall, n represents the total number of categories, and AP(n) represents the AP value of the nth category of prohibited items.

[0061] In this step, the IoU threshold was set to 0.5. Ablation experiments were conducted to test the impact of WHLAFF-YOLO Loss and the attention feature fusion module AFFM on detection performance. The results were compared with five classic mainstream object detection models, namely Faster R-CNN, SSD, RetinaNet, YOLOv3, and YOLOv7. The experimental results are shown in Table 1 (ablation experiment of WHLAFF-YOLO on SIXray) and Table 2 (comparison of detection performance of WHLAFF-YOLO with other networks).

[0062] Table 1

[0063]

[0064] Table 2

[0065] SSD 79.7 62.2 YOLOv3 86.9 58.0 RetinaNet 87.4 24.7 FasterR-CNN 88.6 30.9 YOLOv7 90.2 62.5 WHLAFF-YOLO 92.2 58.4

[0066] As shown in Table 1, compared to the original YOLOv7 model, the improved loss function WHIoU Loss achieved a 1.3% improvement in mAP50. This is because WHIoU Loss increases the overlap between the predicted bounding boxes and the ground truth bounding boxes, improving the model's detection performance. The Attention Feature Fusion (AFFM) module achieved a 1.1% improvement in mAP50. This is because the application of the attention module allows the model to focus on more useful feature information when fusing detailed and semantic information, reducing information redundancy and loss, and making the model's feature extraction capabilities more comprehensive. The WHLAFF-YOLO security image prohibited item detection network model ultimately achieved 92.2% mAP50, a 2% improvement compared to the original YOLOv7 model.

[0067] Table 2 shows the performance comparison results of WHLAFF-YOLO with other networks on the SIXray dataset. As can be seen from Table 2, the WHLAFF-YOLO model has better performance compared to mainstream object detection models, further improving detection accuracy while maintaining detection speed, making the detection of contraband more accurate.

[0068] Step 5: Use the X-ray image containing prohibited items collected during the security check as input, and perform prohibited item detection in the WHLAFF-YOLO security image prohibited item detection network model trained in Step 3, so as to accurately identify the types of prohibited items in the image and accurately locate them, thus completing the prohibited item detection task.

[0069] In this step, please refer to Figure 4 and Figure 5 The results show that the detection method of the present invention not only makes the predicted bounding box closer to the real bounding box, but also detects prohibited items that were missed by the original model, and the confidence level is significantly improved compared with the original model, thus verifying the effectiveness of the present invention.

[0070] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the present invention, and all such modifications or substitutions should be covered within the scope of the claims of the present invention.

Claims

1. A method for detecting prohibited items using X-ray security images, characterized in that: The method includes the following steps: S1: Obtain the security check dataset, preprocess the obtained security check dataset, and randomly divide the preprocessed dataset into training set and validation set according to the proportion; S2: Replace CIoU Loss with WHIoU Loss as the new bounding box loss function, introduce the attention feature fusion module AFFM to improve the feature fusion method of the neck network in the YOLOv7 network model, and construct the WHLAFF-YOLO model, a network for detecting prohibited items in security inspection images. The WHIoU Loss calculation method is expressed as follows: （1） in, It is the intersection-union ratio (IoU) between the predicted bounding box and the ground truth bounding box. It is the Euclidean distance between the center point of the predicted bounding box and the center point of the ground truth bounding box. It is the diagonal distance of the smallest bounding rectangle that simultaneously contains the predicted bounding box and the ground truth bounding box. and These are the larger and smaller values ​​of the widths of the predicted bounding box and the ground truth bounding box, respectively. and These are the larger and smaller values ​​of the predicted bounding box and the ground truth bounding box, respectively; The attention feature fusion module AFFM, when fusing shallow and deep features, assigns a spatial attention (SA) module, which focuses more on shallow details of the target, to shallow features, and a channel attention (CA) module, which focuses more on deep semantic information of the target, to deep features. The spatial attention (SA) module aggregates feature information by performing average pooling and max pooling on the input feature map along its dimensional direction, and then generates a spatial attention map through a convolutional layer, assigning new attention to each pixel of the feature map. The channel attention (CA) module performs global average pooling and global max pooling on the input feature map, respectively, to obtain a value of size [missing information]. The feature map with a global awareness field is aggregated with global context information and the dependencies between channels are captured through two fully connected layers. Finally, the attention feature fusion module AFFM performs deep fusion of the separately processed feature layers in a dimension stacking manner, which fully preserves all feature information of the object and outputs a feature map that combines detailed information and semantic information. The WHLAFF-YOLO model includes an image preprocessing module, a backbone network, a neck network, and a multi-layer detection head; The image preprocessing module uses Mosaic and Mixup data augmentation strategies to augment the training data before inputting it into the backbone network for feature extraction. The backbone network consists of a CBS module, an ELAN module, an MP module, and a SPPCSPC module. The CBS module incorporates convolutional layers, batch normalization layers, and activation functions. The ELAN module stacks feature layers from different gradient paths, enabling the model to learn more information and possess greater robustness. The MP module contains two branches: a max-pooling branch and a convolutional branch, maximizing the retention of target object feature information during downsampling. The SPPCSPC module uses different sizes of... Pooling kernels are used to adapt the model to images of different resolutions and increase the receptive field. Features are extracted from the backbone network and then fed into the neck network. A feature pyramid structure is used for bottom-up upsampling, and the feature layers from the backbone network are fused using AFFM. Then, a path aggregation network is used for top-down downsampling, and the feature layer information output from the feature pyramid is fused using AFFM. The fused feature layer is then fed into a multi-layer detection head. The multi-layer detection head contains three detection heads designed for different input sizes, respectively detecting large, medium, and small targets. The multi-layer detection head uses WHIoULoss to measure the difference between the model's prediction results and the true labels, and then outputs the detection results. S3: Input the training set in S1 into the WHLAFF-YOLO model constructed in S2, train the model, and obtain the trained WHLAFF-YOLO model. S4: Using the WHLAFF-YOLO model trained in S3, detect prohibited items in X-ray images collected during security checks.

2. The method for detecting prohibited items using X-ray security images according to claim 1, characterized in that: In step S1, the preprocessed dataset is randomly divided into a training set and a validation set in a 9:1 ratio.

Images