X-ray image detection method based on contrastive learning and attention mechanism
By employing contrastive learning and attention mechanisms, the problem of low accuracy caused by deformation and color differences in X-ray image detection was solved, achieving higher detection precision.
Patent Information
- Application Number
- CN202410498181.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-04-24
- Publication Date
- 2026-11-17
- Estimated Expiration
- 2044-04-24
AI Technical Summary
Existing technologies for X-ray image inspection suffer from problems such as X-ray projection causing object deformation and color differences in imaging from different security inspection equipment, leading to low detection accuracy.
We employ a method based on contrastive learning and attention mechanisms. We obtain foreground features through the YOLOv5 backbone feature extraction network and DAAM module, and combine it with the PAN feature extraction pyramid network and shape enhancement module to reduce background interference, enhance the capture of object shape information, and learn color invariant features.
It significantly improves the accuracy of X-ray image detection, especially in cases where deformation and color differences exist, thus enhancing the detection effect.
Smart Images

Figure CN118506054B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of X-ray image detection, and more specifically to an X-ray image detection method based on contrastive learning and attention mechanisms. Background Technology
[0002] In security screening scenarios, X-ray image target detection is widely used to detect potentially threatening items, such as weapons or explosives, in luggage, parcels, and personal belongings. With the development of deep learning, more and more security screening systems are adopting deep learning models such as CNNs, improving the accuracy and speed of target detection. Advanced security screening systems can monitor X-ray image streams in real time and automatically issue alarms when suspicious items are detected, improving the ability to respond promptly to potential threats. Unlike detection of RGB images, X-ray images present a greater challenge than traditional target detection tasks due to their inherent characteristics. Existing technologies are mostly improvements and adaptations to the characteristics of X-ray images based on traditional target detectors. Representative works include the Detection-Oriented Attention Module (DOAM) and the Local Inhibition Module (LIM).
[0003] DOAM improves deep learning models through attention mechanisms, enhancing their performance in detecting contraband. This module introduces two sub-modules to emphasize the network's extraction of edge and material information from X-ray images from an attention perspective. The edge guidance module aims to improve the focus on target edges in the image by introducing an attention mechanism for edge information. The material emphasis module aims to strengthen the extraction of material information of contraband.
[0004] LIM focuses on improving the suppression of noise in object detection models while emphasizing the most discriminative boundary information, thereby enhancing detection performance. The LIM module comprises two key sub-modules: the bidirectional propagation (BP) module and the boundary activation (BA) module. The BP module adaptively filters noise generated by the vicinity of contraband items, maximizing noise suppression. The BA module focuses on the boundary information of the image.
[0005] These technologies mostly focus on the problems of weak features of contraband and severe background interference in X-ray images. However, in real-world security inspection scenarios, X-ray image detection not only faces these technical challenges but also the complexity of practical application scenarios. In the field of security inspection, X-ray image target detection also faces the following problems:
[0006] (1) X-ray projection causes deformation of objects. Common contraband such as firearms and cosmetics often vary in shape and size. At the same time, due to the fixed X-ray scanning angle, the projection of the object on the X-ray image often causes deformation, and the target detection of X-ray images is highly dependent on the shape information of the object. Compared with RGB images, which contain rich pattern and texture information, the feature information of X-ray images in security inspection scenarios is more singular.
[0007] (2) Color differences exist in X-ray imaging. In reality, due to differences in the coloring algorithms used by X-ray manufacturers, the color and brightness of the same object may differ when it is imaged on two different security inspection devices. Even security inspection devices from the same manufacturer may produce color differences due to factors such as age and parameter settings. The most extreme example is the color difference between medical images and security inspection images.
[0008] The aforementioned existing technologies have low detection accuracy when X-ray projection causes deformation of the object, or when there are color differences in X-ray imaging. Summary of the Invention
[0009] To address the aforementioned shortcomings in existing technologies, the X-ray image detection method based on contrastive learning and attention mechanisms provided by this invention solves the problem of low detection accuracy in existing technologies when X-ray projection causes deformation of objects or when there are color differences in X-ray imaging.
[0010] To achieve the above-mentioned objectives, the technical solution adopted by this invention is as follows:
[0011] A method for X-ray image detection based on contrastive learning and attention mechanisms is provided, which includes the following steps:
[0012] S1. Acquire the X-ray image to be detected;
[0013] S2. The X-ray image to be detected is used as the input of the foreground feature comparison module to obtain three foreground features corresponding to the X-ray image to be detected; the foreground feature comparison module includes a YOLOv5 backbone feature extraction network and a DAAM module connected in series.
[0014] S3. Input the three foreground features into the PAN feature extraction pyramid network and the three shape enhancement modules respectively;
[0015] S4. Concatenate the outputs of the three shape enhancement modules with the three outputs of the PAN feature extraction pyramid network to obtain three concatenated features;
[0016] S5. The three spliced features are fed into the prediction layer to obtain the prediction box and category information corresponding to the X-ray image to be detected, and the detection is completed.
[0017] Furthermore, the foreground feature comparison module includes a first convolutional layer, a second convolutional layer, a first C3 layer, a third convolutional layer, a second C3 layer, a fourth convolutional layer, a third C3 layer, a fifth convolutional layer, a fourth C3 layer, an SPPF layer, and a DAAM module connected in sequence; wherein the output of the second C3 layer is the first output of the foreground feature comparison module, the output of the third C3 layer is the second output of the foreground feature comparison module, and the output of the DAAM module is the third output of the foreground feature comparison module.
[0018] Furthermore, the DAAM module includes three dilated convolution sub-modules, one deformable convolution sub-module, and one efficient multi-scale attention sub-module. The three dilated convolution sub-modules are connected in sequence. The output of the third dilated convolution sub-module and the input of the DAAM module are concatenated via residuals and used as the input of the deformable convolution sub-module. The output of the deformable convolution sub-module is connected to the input of the efficient multi-scale attention sub-module, and the output of the efficient multi-scale attention sub-module is the output of the DAAM module.
[0019] Furthermore, the dilatation rates of the three dilated convolutional submodules are 1, 2, and 4, respectively; each dilated convolutional submodule includes a 1×1 convolutional layer, a 3×3 dilated convolutional layer, a 1×1 convolutional layer, and a residual connection layer connected in sequence, and the other input of the residual connection layer is the input of the dilated convolutional submodule to which it belongs.
[0020] Furthermore, the PAN feature extraction pyramid network includes a sixth convolutional layer, a first upsampling layer, a first connecting layer, a fifth C3 layer, a seventh convolutional layer, a second upsampling layer, a second connecting layer, a sixth C3 layer, an eighth convolutional layer, a third connecting layer, a seventh C3 layer, a ninth convolutional layer, a fourth connecting layer, and an eighth C3 layer connected in sequence. The input of the sixth convolutional layer is connected to the output of the DAAM module; the other input of the first connecting layer is connected to the second output of the foreground feature comparison module; the other input of the second connecting layer is connected to the first output of the foreground feature comparison module; and the outputs of the sixth, seventh, and eighth C3 layers are the first, second, and third outputs of the PAN feature extraction pyramid network, respectively.
[0021] Furthermore, in step S4, the input of the first shape enhancement module is connected to the first output of the foreground feature comparison module, and the output of the first shape enhancement module is connected to the first output of the PAN feature extraction pyramid network through a connection layer to serve as the first splicing feature.
[0022] The input of the second shape enhancement module is connected to the second output of the foreground feature comparison module. The output of the second shape enhancement module is connected to the second output of the PAN feature extraction pyramid network through a connection layer and then used as the second splicing feature.
[0023] The input of the third shape enhancement module is connected to the third output of the foreground feature comparison module. The output of the third shape enhancement module is connected to the third output of the PAN feature extraction pyramid network through a connection layer and then used as the third splicing feature.
[0024] Furthermore, the shape enhancement module includes an edge activation submodule, three dilated convolution submodules, and a deformable convolution submodule; one edge activation submodule and the three dilated convolution submodules are connected in sequence, and the output of the third dilated convolution submodule is concatenated with the output residual of the edge activation submodule as the input of the deformable convolution submodule; the input of the edge activation submodule is the input of the current shape enhancement module, and the output of the deformable convolution submodule is the output of the current shape enhancement module.
[0025] The beneficial effects of this invention are: this method greatly reduces the interference of background information on the model, and solves the problems of low accuracy caused by deformation due to non-vertical projection and color differences in imaging from different security inspection machines in X-ray images. Attached Figure Description
[0026] Figure 1 This is a flowchart illustrating the method.
[0027] Figure 2 This is a schematic diagram of the network structure involved in this method;
[0028] Figure 3 This is a schematic diagram of the DAAM module.
[0029] Figure 4 This is a structural schematic diagram of the shape enhancement module;
[0030] Figure 5 This is a schematic diagram of the foreground feature comparison module during the training process;
[0031] Figure 6 This diagram illustrates the comparison between our method and the YOLOv5 model in detecting mobile phones, laptops, tablets, and a type of power bank.
[0032] Figure 7 This diagram illustrates the comparison between our method and the YOLOv5 model in detecting cosmetics, water cups, lighters, and another type of power bank. Detailed Implementation
[0033] The specific embodiments of the present invention are described below to enable those skilled in the art to understand the present invention. However, it should be understood that the present invention is not limited to the scope of the specific embodiments. For those skilled in the art, various changes are obvious as long as they are within the spirit and scope of the present invention as defined and determined by the appended claims. All inventions utilizing the concept of the present invention are protected.
[0034] like Figure 1 and Figure 2 As shown, this X-ray image detection method based on contrastive learning and attention mechanisms includes the following steps:
[0035] S1. Acquire the X-ray image to be detected;
[0036] S2. The X-ray image to be detected is used as the input of the foreground feature comparison module to obtain three foreground features corresponding to the X-ray image to be detected; the foreground feature comparison module includes a YOLOv5 backbone feature extraction network and a DAAM module connected in series.
[0037] S3. Input the three foreground features into the PAN feature extraction pyramid network and the three shape enhancement modules respectively;
[0038] S4. Concatenate the outputs of the three shape enhancement modules with the three outputs of the PAN feature extraction pyramid network to obtain three concatenated features;
[0039] S5. The three spliced features are fed into the prediction layer to obtain the prediction box and category information corresponding to the X-ray image to be detected, and the detection is completed.
[0040] The foreground feature comparison module includes a first convolutional layer, a second convolutional layer, a first C3 layer, a third convolutional layer, a second C3 layer, a fourth convolutional layer, a third C3 layer, a fifth convolutional layer, a fourth C3 layer, an SPPF layer, and a DAAM module, which are connected in sequence. The output of the second C3 layer is the first output of the foreground feature comparison module, the output of the third C3 layer is the second output of the foreground feature comparison module, and the output of the DAAM module is the third output of the foreground feature comparison module.
[0041] like Figure 3 As shown, the DAAM module includes three dilated convolution sub-modules, one deformable convolution sub-module, and one efficient multi-scale attention sub-module. The three dilated convolution sub-modules are connected in sequence. The output of the third dilated convolution sub-module and the input of the DAAM module are connected through residual concatenation and used as the input of the deformable convolution sub-module. The output of the deformable convolution sub-module is connected to the input of the efficient multi-scale attention sub-module, and the output of the efficient multi-scale attention sub-module is the output of the DAAM module.
[0042] The three dilated convolutional submodules have dilation rates of 1, 2, and 4, respectively. Each dilated convolutional submodule includes a 1×1 convolutional layer, a 3×3 dilated convolutional layer, a 1×1 convolutional layer, and a residual connection layer connected in sequence. The other input of the residual connection layer is the input of the dilated convolutional submodule to which it belongs.
[0043] The PAN feature extraction pyramid network comprises a sixth convolutional layer, a first upsampling layer, a first connecting layer, a fifth C3 layer, a seventh convolutional layer, a second upsampling layer, a second connecting layer, a sixth C3 layer, an eighth convolutional layer, a third connecting layer, a seventh C3 layer, a ninth convolutional layer, a fourth connecting layer, and an eighth C3 layer connected in sequence. The input of the sixth convolutional layer is connected to the output of the DAAM module; the other input of the first connecting layer is connected to the second output of the foreground feature comparison module; the other input of the second connecting layer is connected to the first output of the foreground feature comparison module; and the outputs of the sixth, seventh, and eighth C3 layers are the first, second, and third outputs of the PAN feature extraction pyramid network, respectively.
[0044] The input of the first shape enhancement module is connected to the first output of the foreground feature comparison module. The output of the first shape enhancement module is connected to the first output of the PAN feature extraction pyramid network through a connection layer and used as the first splicing feature.
[0045] The input of the second shape enhancement module is connected to the second output of the foreground feature comparison module. The output of the second shape enhancement module is connected to the second output of the PAN feature extraction pyramid network through a connection layer and then used as the second splicing feature.
[0046] The input of the third shape enhancement module is connected to the third output of the foreground feature comparison module. The output of the third shape enhancement module is connected to the third output of the PAN feature extraction pyramid network through a connection layer and then used as the third splicing feature.
[0047] like Figure 4 As shown, the shape enhancement module includes an edge activation submodule, three dilated convolution submodules, and a deformable convolution submodule. One edge activation submodule and the three dilated convolution submodules are connected in sequence. The output of the third dilated convolution submodule is concatenated with the output residual of the edge activation submodule and used as the input of the deformable convolution submodule. The input of the edge activation submodule is the input of the current shape enhancement module, and the output of the deformable convolution submodule is the output of the current shape enhancement module.
[0048] In practical implementation, the network model involved in this method can be trained before use. To enrich the training samples before training, random combinations of grayscale transformation, color channel swapping, image rotation, etc., can be used to expand the training images. For the same image, a pair of positive sample views is generated by randomly combining two data augmentation strategies, and the image pairs generated after data augmentation of different images are negative sample views.
[0049] The YOLO backbone feature extraction network, which shares weights between positive and negative sample views, extracts data view features F1. This extracted data view feature F1 is then input into the Deformation Adaptive Attention Module (DAAM). The DAAM module consists of three dilated convolutional sub-modules, a deformable convolutional sub-module (DCN), and an efficient multi-scale attention sub-module (EMA). In the dilated convolutional sub-module, the input feature map is first adjusted for channel dimensions using a 1×1 convolutional layer, followed by a 3×3 dilated convolutional layer to expand the receptive field, and then another 1×1 convolutional layer to adjust the channel dimensions. Finally, this is added to the input of the dilated convolutional sub-module and output to the next dilated convolutional sub-module. The DAAM module generates an output feature map F2 (the output of the third dilated convolutional sub-module) containing three different receptive fields by superimposing three dilated convolutional sub-modules with different dilation rates, effectively capturing long-range information in the feature map to adapt to contraband at different scales.
[0050] Feature map F2 is concatenated with the input residuals of the DAAM model and then input into a 3×3 deformable convolutional submodule (DCN) to reduce the negative impact of object deformation caused by non-perpendicular X-ray projection, while filtering out redundant background information caused by the increased receptive field of dilated convolution, resulting in feature map F3. Feature map F3 is then input into an efficient multi-scale attention submodule (EMA) to obtain feature map F4 with attention weights.
[0051] like Figure 5 As shown, for each pair of data views obtained after random data augmentation, the feature map F4 is mapped to the contrastive loss space. Then, the distance between the two different feature vectors of this pair of data views is calculated using a similarity metric function (cosine similarity function can be used in this embodiment). The contrastive loss function minimizes the distance between positive sample pairs to maximize the consistency between different augmented views of the same data sample, thereby driving the model to learn color-invariant features.
[0052] Simultaneously, the three output features of the Foreground Feature Comparison Module (FCM) are input in parallel into the PAN Feature Pyramid Network and the Shape Enhancement Module (SEM). The Shape Enhancement Module includes an edge activation submodule, a dilated convolution submodule, and a deformable convolution submodule. Its function is to make the model focus more on the shape features of the image, thereby weakening features such as color and texture, and strengthening the color invariance of X-ray images. The edge activation submodule enhances boundary information from four directions in the feature map output of each layer and aggregates them into a whole shape to activate the object's shape information, while weakening the object's color and brightness information. The dilated convolution submodule and the deformable convolution submodule are similar to those in the DAAM module, responsible for enhancing the model's capture of object shape information.
[0053] The parallel features from the PAN feature pyramid network and the shape enhancement module are concatenated and stacked before being input into the prediction layer. The prediction layer then infers the bounding box and contraband category information. At this point, the location loss, classification loss, and confidence loss are calculated and weighted together with the contrastive loss to form the overall loss of the entire model. This overall loss is backpropagated back into the neural network to calculate the gradient of each parameter, updating the weights and biases in the network until the overall loss function converges to its minimum, completing the training process.
[0054] In one embodiment of the present invention, to obtain the color invariance features of X-ray images, the network is trained using original training images from the HiXray dataset, and tested using test images with random color jitter to simulate color differences in real security inspection scenarios. The training set contains 36,295 X-ray images, and the test set contains 9,069 X-ray images, with a training-to-test data ratio of 8:2. The input to the entire network is the X-ray image, and the output is the location and category information of contraband.
[0055] In this embodiment, the method is compared with other methods based on the YOLOv5 model. The comparison results are shown in Table 1.
[0056] Table 1: Comparison results with other models on the HiXray dataset
[0057]
[0058] As shown in Table 1, on the test set after color dithering, this method achieves a 2.3 percentage point higher mAP than the baseline model. Except for the cosmetics category, the detection accuracy is improved for all other categories.
[0059] In this embodiment, Figure 6and Figure 7 The comparison between our method and the baseline model is shown. As can be seen from the figures, for example samples in the categories of mobile phones and tablets, our method significantly improves the confidence level compared to the baseline model. For example samples in the category of lighters, our method detected objects missed by the baseline model. This demonstrates that when there are color differences between the training and test set samples, our method can better learn the color invariance features of X-ray images, reducing the model's sensitivity to color and thus significantly improving its detection capabilities.
[0060] Meanwhile, neither our method nor the baseline model detected the target object in the example samples of the water cup category. However, the water cup object was more easily detected in the test set before color jitter. For the human eye, both X-ray images also easily distinguish the water cup object. This indicates that color difference does indeed lead to a decrease in model detection accuracy, affecting the actual performance of the algorithm.
[0061] In summary, this invention significantly reduces the interference of background information on the model and solves the problems of low accuracy caused by distortion due to non-perpendicular projection and color differences in imaging from different security inspection machines in X-ray images.
Claims
1. An X-ray image detection method based on contrastive learning and attention mechanisms, characterized in that, Includes the following steps: S1. Acquire the X-ray image to be detected; S2. The X-ray image to be detected is used as the input of the foreground feature comparison module to obtain three foreground features corresponding to the X-ray image to be detected; the foreground feature comparison module includes a YOLOv5 backbone feature extraction network and a DAAM module connected in series. S3. Input the three foreground features into the PAN feature extraction pyramid network and the three shape enhancement modules respectively; S4. Concatenate the outputs of the three shape enhancement modules with the three outputs of the PAN feature extraction pyramid network to obtain three concatenated features; S5. The three spliced features are fed into the prediction layer to obtain the prediction box and category information corresponding to the X-ray image to be detected, and the detection is completed. In step S4, the input of the first shape enhancement module is connected to the first output of the foreground feature comparison module, and the output of the first shape enhancement module is connected to the first output of the PAN feature extraction pyramid network through a connection layer to serve as the first splicing feature. The input of the second shape enhancement module is connected to the second output of the foreground feature comparison module. The output of the second shape enhancement module is connected to the second output of the PAN feature extraction pyramid network through a connection layer and then used as the second splicing feature. The input of the third shape enhancement module is connected to the third output of the foreground feature comparison module. The output of the third shape enhancement module is connected to the third output of the PAN feature extraction pyramid network through a connection layer and then used as the third stitching feature. The parallel features of the PAN feature pyramid network and the shape enhancement module are concat-stacked and then input into the prediction layer. The prediction layer infers the prediction box and the contraband category information.
2. The X-ray image detection method based on contrastive learning and attention mechanism according to claim 1, characterized in that, The foreground feature comparison module includes a first convolutional layer, a second convolutional layer, a first C3 layer, a third convolutional layer, a second C3 layer, a fourth convolutional layer, a third C3 layer, a fifth convolutional layer, a fourth C3 layer, an SPPF layer, and a DAAM module, which are connected in sequence. The output of the second C3 layer is the first output of the foreground feature comparison module, the output of the third C3 layer is the second output of the foreground feature comparison module, and the output of the DAAM module is the third output of the foreground feature comparison module.
3. The X-ray image detection method based on contrastive learning and attention mechanism according to claim 2, characterized in that, The DAAM module includes three dilated convolution submodules, one deformable convolution submodule, and one efficient multi-scale attention submodule. The three dilated convolution submodules are connected in sequence. The output of the third dilated convolution submodule and the input of the DAAM module are concatenated via residuals and used as the input of the deformable convolution submodule. The output of the deformable convolution submodule is connected to the input of the efficient multi-scale attention submodule, and the output of the efficient multi-scale attention submodule is the output of the DAAM module.
4. The X-ray image detection method based on contrastive learning and attention mechanism according to claim 3, characterized in that, The three dilated convolutional submodules have dilation rates of 1, 2, and 4, respectively. Each dilated convolutional submodule includes a 1×1 convolutional layer, a 3×3 dilated convolutional layer, a 1×1 convolutional layer, and a residual connection layer connected in sequence. The other input of the residual connection layer is the input of the dilated convolutional submodule to which it belongs.
5. The X-ray image detection method based on contrastive learning and attention mechanism according to claim 2, characterized in that, The PAN feature extraction pyramid network comprises a sixth convolutional layer, a first upsampling layer, a first connecting layer, a fifth C3 layer, a seventh convolutional layer, a second upsampling layer, a second connecting layer, a sixth C3 layer, an eighth convolutional layer, a third connecting layer, a seventh C3 layer, a ninth convolutional layer, a fourth connecting layer, and an eighth C3 layer connected in sequence. The input of the sixth convolutional layer is connected to the output of the DAAM module; the other input of the first connecting layer is connected to the second output of the foreground feature comparison module; the other input of the second connecting layer is connected to the first output of the foreground feature comparison module; and the outputs of the sixth, seventh, and eighth C3 layers are the first, second, and third outputs of the PAN feature extraction pyramid network, respectively.
6. The X-ray image detection method based on contrastive learning and attention mechanism according to claim 4, characterized in that, The shape enhancement module includes an edge activation submodule, three dilated convolution submodules, and a deformable convolution submodule; one edge activation submodule and three dilated convolution submodules are connected in sequence, and the output of the third dilated convolution submodule is concatenated with the output residual of the edge activation submodule as the input of the deformable convolution submodule; The input to the edge activation submodule is the input to the current shape enhancement module, and the output of the deformable convolution submodule is the output of the current shape enhancement module.
Citation Information
Patent Citations
A lightweight object detection method based on multiple receptive fields and attention feature pyramids
CN114937151A
Security check image display method and security check system
CN117492316A
