A contraband detection method based on weighted bidirectional fusion feature pyramid network
By introducing a weighted two-way fusion mechanism and attention module into the feature pyramid network, a multi-scale feature map is generated, which solves the missed detection and low accuracy of multi-scale contraband detection in X-ray security images, and achieves more efficient contraband detection .
Patent Information
- Application Number
- CN202210000437.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2022-01-04
- Publication Date
- 2024-11-26
- Estimated Expiration
- 2042-01-04
AI Technical Summary
The prior art is difficult to effectively detect multi-scale contraband in X-ray security images, and the detection accuracy is low. Traditional methods rely on manual design features and have poor robustness. Deep learning methods fail to fully solve the multi-scale problem.
Weighted two-way fusion feature pyramid network is adopted, and by adding attention modules and top-down and bottom-up fusion paths to the feature pyramid network, weighted feature maps are generated, which reduces useless information redundancy, enhances the lowest feature details information, and obtains Multi-scale feature map, construct a contraband detection network containing ResNet for multiple trainings.
It effectively solves the problem of missed inspection of multi-scale contraband products in the context of chaos, improves detection accuracy, and is significantly better than other contraband detection methods based on deep learning.
Smart Images

Figure CN114399482B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to a contraband detection method based on a weighted bidirectional fusion feature pyramid network, and belongs to the technical field of deep learning and image target detection. Background Art
[0002] Security checks play a vital role in ensuring public safety at transportation hubs such as airports and train stations, as well as at some major events. Currently, security checks are mainly carried out by obtaining X-ray security images through security inspection machines, and then relying on human eyes to identify contraband in the X-ray security images. However, manual identification of contraband is easily affected by factors such as visual fatigue and work pressure, and is inefficient and costly. Therefore, contraband detection technology using computer vision technology has received increasing attention.
[0003] Currently, the detection methods of contraband in X-ray images are mainly divided into two categories: traditional detection methods based on machine learning and target detection methods based on deep learning. Traditional detection methods based on machine learning need to use artificially designed features, such as Speeded Up Robust Features (SURF) and Scale Invariant Feature Transform (SIFT), and then classify and identify contraband through support vector machines (SVM) and nearest neighbor algorithms (K-Nearest Neighbor, K-NN). Traditional detection methods rely on artificially designed features, have poor robustness and are limited to small sample data sets, and cannot meet the needs of actual applications.
[0004] Deep learning methods can automatically extract target features through convolutional neural networks, making up for the defects of traditional machine learning methods that rely on manually designed features. Therefore, there are endless contraband detection methods based on deep learning. For example, Wang et al. proposed the PIDray dataset and proposed the Selective Dense Attention Network (SDANet) to detect occluded contraband in the dataset; Miao et al. collected a Security Inspection X-ray image dataset (SIXray) and proposed a Class-balanced Hierarchical Refinement (CHR); Wei et al. collected an Occluded Prohibited Items X-ray image dataset (OPIXray) and proposed the De-occlusion Attention Module (DOAM) to detect contraband under three occlusion levels, but the detection accuracy needs to be improved. However, none of the above methods consider the multi-scale problem of contraband. Therefore, the detection of multi-scale contraband in a cluttered background is still a difficult problem.
[0005] Based on the shortcomings of existing methods, this application is committed to improving the shortcomings of feature pyramid networks, combining the characteristics of contraband in X-ray security inspection images, and solving the problems of missed detection of multi-scale contraband in cluttered backgrounds and poor recognition accuracy. Summary of the invention
[0006] The purpose of the present invention is to improve the feature pyramid network, solve the problem of missed detection and low detection accuracy of multi-scale contraband in cluttered background in X-ray security inspection images, and propose a contraband detection method based on weighted bidirectional fusion feature pyramid network.
[0007] In order to achieve the above object, the present invention adopts the following technical scheme.
[0008] The contraband detection method comprises the following steps:
[0009] S10: Collect original X-ray security inspection images, create an X-ray security inspection image dataset, and divide the dataset into a training set and a test set;
[0010] The original X-ray security inspection image is an image captured and output by an X-ray security inspection machine;
[0011] S10 specifically includes: collecting original X-ray security inspection images from an X-ray security inspection machine, annotating the original X-ray security inspection images with prohibited items categories and locations, obtaining an X-ray security inspection image dataset, and then dividing the X-ray security inspection image dataset into a training set and a test set in a ratio of M:(10-M);
[0012] Among them, the value range of M is greater than or equal to 6 and less than or equal to 9;
[0013] S20: Improve the feature pyramid network and construct a weighted bidirectional fusion feature pyramid network;
[0014] Among them, the Feature Pyramid Network, or FPN for short, consists of four 1*1 convolutions, a top-down fusion path, and four 3*3 convolutions;
[0015] Among them, top-down fusion refers to upsampling fusion of feature maps;
[0016] Among them, the weighted bidirectional fusion feature pyramid network, namely WDFPN;
[0017] Step S20 includes the following sub-steps:
[0018] S21: Add three attention modules in the middle of the FPN top-down fusion path;
[0019] The specific operation of S21 is shown in formula (1):
[0020] Pj=Wi×{Conv1(Cj)}+UP(Pi) (1)
[0021] Among them, Pj and Pi are the feature maps generated by top-down fusion; Wi is the weighted coefficient output after the feature map passes through the attention module; Conv1 is a 1*1 convolution; Cj is the input feature map of WDFPN; UP represents the upsampling operation; the values of i are 5, 4, and 3 respectively, and the value of j is i-1;
[0022] S22: Add an attention module below the fourth 3*3 convolution of FPN to generate feature map D2;
[0023] The specific operation of S22 is shown in formula (2):
[0024] D2=Conv2(P2)+W2×P2 (2)
[0025] Among them, W2 is the weighted coefficient output by the feature map P2 after passing through the attention module; Conv2 is a 3*3 convolution;
[0026] S23: Add a bottom-up fusion path at the FPN output;
[0027] Among them, bottom-up fusion refers to feature map downsampling fusion;
[0028] The specific operation of S23 is shown in formula (3):
[0029] Dn=DS(Dm)+Conv2(Pn) (3)
[0030] Among them, Dn and Dm are multi-scale feature maps output by WDFPN; DS represents the downsampling operation; Conv2 is a 3*3 convolution; Pn is the feature map output by top-down fusion; the values of m are 2, 3, and 4 respectively, and the value of n is m+1;
[0031] S30: Build a contraband detection network including feature extraction network, RPN network, ROI pooling layer and classification regression network;
[0032] Among them, the feature extraction network includes N layers of ResNet and a WDFPN;
[0033] Among them, the number of layers N of ResNet is 50, 101 or 152;
[0034] S40: training the contraband detection network multiple times using the training set to obtain a contraband detection model;
[0035] S50: Testing the contraband detection model using the test set to verify the performance of the contraband detection model;
[0036] So far, from S10 to S50, the contraband detection method based on weighted bidirectional fusion feature pyramid network has been completed.
[0037] Beneficial Effects
[0038] Compared with the prior art, the improved contraband detection method based on feature pyramid network proposed in the present invention has the following beneficial effects:
[0039] 1. The method improves the feature pyramid network and uses the attention module to generate weighted coefficients to reduce the redundancy of useless information in top-down fusion, while strengthening the detailed information of contraband in the bottom-level features, and then obtains multi-scale feature maps through bottom-up feature fusion, which can effectively solve the problem of multi-scale contraband missed detection in cluttered backgrounds in X-ray security inspection images;
[0040] 2. Compared with other contraband detection methods based on deep learning, the contraband detection method effectively improves the detection accuracy. BRIEF DESCRIPTION OF THE DRAWINGS
[0041] Figure 1 It is a flow chart of the model training of the contraband detection method based on the weighted bidirectional fusion feature pyramid network of the present invention;
[0042] Figure 2 It is a structural diagram of the WDFPN in step 3 of the contraband detection method based on the weighted bidirectional fusion feature pyramid network of the present invention;
[0043] Figure 3 It is an effect diagram of detecting contraband using the contraband detection method based on weighted bidirectional fusion feature pyramid network described in the present invention;
[0044] Figure 4 It is a schematic diagram of a contraband detection platform based on a weighted bidirectional fusion feature pyramid network for realizing the present invention. DETAILED DESCRIPTION
[0045] The contraband detection method based on the weighted bidirectional fusion feature pyramid network of the present invention is described in detail below with reference to the accompanying drawings and specific embodiments.
[0046] Example 1
[0047] This embodiment describes the specific implementation of the contraband detection method based on the weighted bidirectional fusion feature pyramid network described in the present invention.
[0048] This example uses NVIDIA GeForce GTX 1080Ti GPU, pytorch deep learning framework and python language programming environment. The mean average precision (mAP) is used as the evaluation index for contraband detection. The contraband detection method based on weighted bidirectional fusion feature pyramid network proposed in this application is used to detect contraband on the datasets SIXray and OPIXray, and compared with various methods, which reflects the beneficial effects of the method described in the present invention.
[0049] Among them, the datasets SIXray and OPIXray contain a large number of X-ray security inspection images of luggage and parcels, with cluttered backgrounds, numerous items, and prohibited items of various categories and sizes.
[0050] Figure 1 This is the process of the contraband detection method based on the weighted bidirectional fusion feature pyramid network in the specific implementation mode of the present invention, which specifically includes the following steps:
[0051] S10: Collect original X-ray security inspection images, create an X-ray security inspection image dataset, and divide the dataset into a training set and a test set;
[0052] The original X-ray security inspection image is an image captured and output by an X-ray security inspection machine;
[0053] S10 specifically includes: collecting original X-ray security inspection images from an X-ray security inspection machine, annotating the original X-ray security inspection images with prohibited items categories and locations, obtaining an X-ray security inspection image dataset, and then dividing the X-ray security inspection image dataset into a training set and a test set in a ratio of M:(10-M);
[0054] Among them, the value of M is 8;
[0055] S20: Improve the feature pyramid network and construct a weighted bidirectional fusion feature pyramid network;
[0056] Among them, the Feature Pyramid Network, or FPN for short, consists of four 1*1 convolutions, a top-down fusion path, and four 3*3 convolutions;
[0057] Among them, top-down fusion refers to upsampling fusion of feature maps;
[0058] Among them, the weighted bidirectional fusion feature pyramid network, namely WDFPN;
[0059] Step S20 includes the following sub-steps:
[0060] S21: Add three attention modules in the middle of the FPN top-down fusion path;
[0061] The specific operation of S21 is shown in formula (1):
[0062] Pj=Wi×{Conv1(Cj)}+UP(Pi) (1)
[0063] Among them, Pj and Pi are the feature maps generated by top-down fusion; Wi is the weighted coefficient output after the feature map passes through the attention module; Conv1 is a 1*1 convolution; Cj is the input feature map of WDFPN; UP represents the upsampling operation; the values of i are 5, 4, and 3 respectively, and the value of j is i-1;
[0064] S22: Add an attention module below the fourth 3*3 convolution of FPN to generate feature map D2;
[0065] The specific operation of S22 is shown in formula (2):
[0066] D2=Conv2(P2)+W2×P2 (2)
[0067] Among them, W2 is the weighted coefficient output by the feature map P2 after passing through the attention module; Conv2 is a 3*3 convolution;
[0068] S23: Add a bottom-up fusion path at the FPN output;
[0069] Among them, bottom-up fusion refers to feature map downsampling fusion;
[0070] The specific operation of S23 is shown in formula (3):
[0071] Dn=DS(Dm)+Conv2(Pn) (3)
[0072] Among them, Dn and Dm are multi-scale feature maps output by WDFPN; DS represents the downsampling operation; Conv2 is a 3*3 convolution; Pn is the feature map output by top-down fusion; the values of m are 2, 3, and 4 respectively, and the value of n is m+1;
[0073] S30: Build a contraband detection network including feature extraction network, RPN network, ROI pooling layer and classification regression network;
[0074] Among them, the feature extraction network includes N layers of ResNet and a WDFPN;
[0075] Among them, the value of the number of layers N of ResNet is 101;
[0076] S40: training the contraband detection network multiple times using the training set to obtain a contraband detection model;
[0077] S50: Testing the contraband detection model using the test set to verify the performance of the contraband detection model;
[0078] At this point, from step S10 to step S50, the entire process of the contraband detection method based on the weighted bidirectional fusion feature pyramid network is completed; experimental verification shows that this method can effectively detect multi-scale contraband in a cluttered background, output the contraband category score and frame the contraband location, and the detection accuracy is significantly better than other methods.
[0079] Figure 2 The schematic diagram of the structure of the WDFPN in the contraband detection method based on the weighted bidirectional fusion feature pyramid network in the specific implementation of the present invention. The WDFPN includes four 1×1 convolutions, four 3×3 convolutions, four attention modules, three upsampling units, and three downsampling units; the WDFPN can use the attention module to weight the top-down fused feature maps to reduce the redundancy of useless information, and at the same time enhance the ability to locate contraband through bottom-up fusion.
[0080] Figure 3 This is a contraband detection effect diagram. The contraband detection method of the present invention is experimented on the public data sets SIXray and OPIXray. Figure 3 The left column is the original X-ray security inspection image, and the right column is the output image obtained by the contraband detection method described in the present invention; Figure 3It can be seen that this method can accurately detect multi-scale contraband in a cluttered background in the X-ray security inspection image, output the contraband category score and accurately frame the location of the contraband, which shows the beneficial effect of the method of the present invention in detecting multi-scale contraband in a cluttered background.
[0081] The contraband detection method based on weighted bidirectional fusion feature pyramid network described in this application was used to conduct experimental simulation on two public datasets SIXray and OPIXray.
[0082] The detection results output on SIXray are shown in Table 1 below:
[0083] Table 1 Detection results of contraband detection method based on weighted bidirectional fusion feature pyramid network on the dataset SIXray
[0084]
[0085] Table 1 shows the detection accuracy of various types of contraband obtained by Faster R-CNN+FPN, Cascade R-CNN+FPN, the method of the present invention, and the CHR method from top to bottom; the categories in the second column are the five types of contraband in the SIXray dataset from left to right: knife, pliers, gun, wrench, scissors; the last column is the overall evaluation index mAP. It can be seen from Table 1 that compared with other methods, the method of the present invention has the highest mAP for contraband detection.
[0086] The detection results output on OPIXray are shown in Table 2 below:
[0087] Table 2 Detection results of contraband detection method based on weighted bidirectional fusion feature pyramid network on the dataset OPIXray
[0088]
[0089] Table 2 shows the detection accuracy of various types of contraband obtained by Faster R-CNN+FPN, Cascade R-CNN+FPN, the method described in the present invention, and the SSD+DOAM method from top to bottom; the categories in the second column show the five types of contraband in the OPIXray dataset from left to right: folding knife, straight knife, scissors, utility knife, multi-tool knife; the last column is the overall evaluation index mAP. It can be seen from Table 2 that the mAP of the other three methods on OPIXray is lower than that of the method described in the present invention; in summary, the multi-scale contraband detection performance mAP of the method described in the present application is the best.
[0090] A contraband detection platform based on a weighted bidirectional fusion feature pyramid network, as shown in FIG4, includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the contraband detection method based on the weighted bidirectional fusion feature pyramid network; and also includes a computer-readable storage medium storing a computer program, characterized in that when the computer program is executed by the processor, the contraband detection method based on the weighted bidirectional fusion feature pyramid network is implemented.
[0091] In summary, the above are only preferred embodiments of the present invention and are not intended to limit the protection scope of the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the protection scope of the present invention.
Claims
1. A contraband detection method based on a weighted bidirectional fusion feature pyramid network, characterized in that: The steps include: S10: Collect original X-ray security inspection images, create an X-ray security inspection image dataset, and divide the dataset into a training set and a test set; S20: Improve the feature pyramid network and construct a weighted bidirectional fusion feature pyramid network; Among them, the weighted bidirectional fusion feature pyramid network, namely WDFPN; the feature pyramid network, namely FPN, contains four 1*1 convolutions, a top-down fusion path and four 3*3 convolutions; Step S20 includes the following sub-steps: S21: Add three attention modules in the middle of the FPN top-down fusion path; S22: Add an attention module below the fourth 3*3 convolution of FPN to generate feature map D2; S23: Add a bottom-up fusion path at the FPN output; S30: Build a contraband detection network including feature extraction network, RPN network, ROI pooling layer and classification regression network; S40: training the contraband detection network multiple times using the training set to obtain a contraband detection model; S50: Use the test set to test the contraband detection model to verify the performance of the contraband detection model.
2. The contraband detection method according to claim 1, characterized in that: The original X-ray security inspection image in S10 is an image captured and output by an X-ray security inspection machine.
3. The contraband detection method according to claim 1, characterized in that: S10 specifically includes: collecting original X-ray security inspection images from an X-ray security inspection machine, annotating the original X-ray security inspection images with prohibited items categories and locations, obtaining an X-ray security inspection image dataset, and then dividing the X-ray security inspection image dataset into a training set and a test set in a ratio of M:(10-M); The value range of M is greater than or equal to 6 and less than or equal to 9.
4. The contraband detection method according to claim 1, characterized in that: In S20, top-down fusion is feature map upsampling fusion; in S23, bottom-up fusion is feature map downsampling fusion.
5. The method for detecting contraband according to claim 1, characterized in that: The specific operation of S21 is shown in formula (1): Pj=Wi×{Conv1(Cj)}+UP(Pi) (1) Among them, Pj and Pi are feature maps generated by top-down fusion; Wi is the weighted coefficient output after the feature map passes through the attention module; Conv1 is a 1*1 convolution; Cj is the input feature map of WDFPN; UP represents the upsampling operation; the values of i are 5, 4, and 3 respectively, and the value of j is i-1.
6. The method for detecting contraband according to claim 1, characterized in that: The specific operation of S22 is shown in formula (2): D2=Conv2(P2)+W2×P2 (2) Among them, W2 is the weighted coefficient output by the feature map P2 after passing through the attention module; Conv2 is a 3*3 convolution.
7. The contraband detection method according to claim 1, characterized in that: The specific operation of S23 is shown in formula (3): Dn=DS(Dm)+Conv2(Pn) (3) Among them, Dn and Dm are multi-scale feature maps output by WDFPN; DS represents the downsampling operation; Conv2 is a 3*3 convolution; Pn is the feature map of the top-down fusion output; the values of m are 2, 3, and 4 respectively, and the value of n is m+1.
8. The method for detecting contraband according to claim 1, characterized in that: In S30, the feature extraction network contains N layers of ResNet and a WDFPN.
9. The method for detecting contraband according to claim 8, characterized in that: The number of layers N of ResNet can be 50, 101 or 152.
10. A contraband detection platform based on a weighted bidirectional fusion feature pyramid network, characterized by: include: at least one processor; And a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the contraband detection method based on the weighted bidirectional fusion feature pyramid network as described in any one of claims 1 to 9.
