A method, device and system for detecting prohibited items in X-ray security inspection images

Abstract

The present invention discloses a method, device and system for detecting contraband items for X-ray security inspection images. The method comprises: extracting features from X-ray security inspection images according to depthwise separable convolution to obtain an item information feature layer; pooling the high-level item information feature layer; updating the item information feature layer according to the pooled high-level item information feature layer, and performing feature fusion on the updated item information feature layer to obtain a fused feature layer; extracting category information and position information from the low-level item information feature layer to generate a small item information feature layer, and performing feature fusion on the small item information feature layer and the first fused feature layer to obtain a second fused feature layer; updating the fused feature layer according to the second fused feature layer, respectively, and setting a detection frame and item information for the X-ray security inspection image according to the updated fused feature layer to obtain a detection result. The embodiment of the present invention can make the lightweight target detection model have higher detection accuracy.

Description

Technical Field

[0001] The present invention relates to the field of image recognition, and in particular to a method, device and system for detecting contraband in X-ray security inspection images. Background Art

[0002] At present, the object detection task of contraband in X-ray security inspection images based on deep learning can be divided into contraband classification and detection. Convolutional Neural Networks (CNN) has great advantages in feature extraction of contraband and has been successfully applied to the classification of contraband in X-ray security inspection images. Object detection is the main research direction in the current X-ray security inspection image processing. The mainstream object detection algorithms include YOLO, RCNN, Faster RCNN, R-FCN, F-RCNN, SSD, etc. With the deep neural network (DNN) as the framework and the convolutional neural network as the learning method, the learning features are used to replace the manual features, which improves the generalization and robustness of the object recognition model. However, the current technical means are difficult to cope with the imaging mode and complex object background of security inspection images, and these mainstream object detection algorithm models are highly complex, with large amount of calculation and parameters, high hardware requirements for X-ray security inspection equipment, and high deployment costs; after reducing the amount of calculation or lightweighting the object detection model, the object detection model with reduced calculation depth has reduced detection accuracy for multi-size and mutually occluded contraband. Summary of the invention

[0003] The present invention provides a method, device and system for detecting contraband in X-ray security inspection images, so as to solve the technical problem of reduced detection accuracy of traditional lightweight target detection models.

[0004] In order to solve the above technical problems, an embodiment of the present invention provides a method for detecting contraband in X-ray security inspection images, comprising:

[0005] Performing feature extraction on the X-ray security inspection image according to the depthwise separable convolution to obtain an item information feature layer; wherein the item information feature layer comprises: a low-layer item information feature layer and a high-layer item information feature layer; the size of the low-layer item information feature layer is larger than the size of the high-layer item information feature layer;

[0006] Unifying the size of the high-level item information feature layer according to the pooling operation to obtain an item information feature layer of a first preset size;

[0007] According to the item information feature layer of the first preset size, data of the item information feature layer is updated respectively, and features of the updated item information feature layer are fused to obtain a fused feature layer; wherein the fused feature layer includes a first fused feature layer;

[0008] Extracting category information and location information from the low-level item information feature layer to generate a small item information feature layer, and performing feature fusion on the small item information feature layer and the first fused feature layer to obtain a second fused feature layer;

[0009] The data of the fused feature layer is updated respectively according to the second fused feature layer, and a detection frame and item information are set for the X-ray security inspection image according to the updated fused feature layer to obtain a detection result.

[0010] The present invention uses deep separable convolution to extract features from X-ray security inspection images, which has less computational complexity than traditional feature extraction networks such as VGG, ResNet, etc., while the feature extraction accuracy does not decrease, and lightweight is achieved in the feature extraction link. After the pooling operation, the feature layers of item information of different sizes obtained by feature extraction can ensure that both the item contour and position information of the large-size feature layer and the semantic information of the small-size feature layer are retained during feature fusion, so that the fused feature layer retains more item information in the X-ray security inspection image and improves the detection accuracy. The shallow item feature information has more small item information because of its large size. After the category information and position information are extracted, it is fused with the first fused feature layer to obtain a second fused feature layer, and the second fused feature layer has more small item features than the first fused feature layer. The fused feature layer is updated according to the second fused feature layer, and the detection frame and item information are added to the X-ray security inspection image according to the updated fused feature layer, which can more accurately display the category and specific location of small items, and has higher detection accuracy for small target contraband than the traditional target detection model.

[0011] Furthermore, the feature extraction of the X-ray security inspection image is performed according to the depthwise separable convolution to obtain the item information feature layer, specifically:

[0012] The depth-wise separable convolution includes depth-wise convolution and point-wise convolution;

[0013] Performing a convolution operation on the X-ray security inspection image using a unit convolution kernel according to a depthwise convolution to obtain an item information feature layer with a constant number of channels;

[0014] Increasing the number of channels of the item information feature layer whose number of channels remains unchanged according to point-by-point convolution to obtain the low-level item information feature layer and the high-level item information feature layer;

[0015] Repeating the depthwise convolution and pointwise convolution of the item information feature layer for a preset number of times to obtain a plurality of low-layer item information feature layers and a plurality of high-layer item information feature layers; the plurality of low-layer item information feature layers and the plurality of high-layer item information feature layers constitute the item information feature layer.

[0016] The feature extraction of the present invention specifically adopts deep convolution and point-by-point convolution, wherein the dimension of the feature layer obtained by operating the security inspection image according to the deep convolution remains unchanged, so that the overall amount of calculation is reduced compared with the traditional target recognition model; at the same time, the point-by-point convolution is used to increase the depth of the feature layer with unchanged dimension, ensuring that the feature layer with an increased number of channels has a larger amount of information, thereby ensuring the accuracy of the detection result.

[0017] Furthermore, the data of the item information feature layer is updated respectively according to the item information feature layer of the first preset size, specifically: the high-level item information feature layer in the item information feature layer is replaced with the item information feature layer of the first preset size to obtain the updated item information feature layer.

[0018] In the present invention, the object information feature layer of the first preset size is obtained through a pooling operation, which ensures that the proportion and shape of the object features in the original X-ray security inspection image are retained in the w object information of the first preset size while meeting the size requirements of the feature fusion operation; replacing the high-level object information feature layer with this feature layer ensures that the updated object information feature layer retains more information of the original X-ray security inspection image, so that less information is lost in the fused feature layer.

[0019] Furthermore, the extraction of category information and location information of the lower-level item information feature layer to generate the small item information feature layer is specifically as follows:

[0020] Unifying the size of the lower-level item information feature layer according to the pooling operation to obtain an item information feature layer of a second preset size;

[0021] fusing the semantic information and the resolution information of the feature layer of the second preset size according to the feature pyramid network to obtain an object information fusion feature layer;

[0022] The small object features in the item information fusion feature layer are extracted to obtain the small item information feature layer.

[0023] In the process of small target feature extraction, the present invention adopts a pooling operation to unify the size of the low-level item information feature layer, so that the item information feature layer of the second preset size can retain more item features of the original X-ray inspection image while meeting the size requirement required for the feature pyramid network fusion, so that the item information feature layer has richer item information features; after the small target feature extraction, the small item information feature layer obtained has richer small item information features, and the classification and positioning of small items according to the feature layer will be more accurate.

[0024] Furthermore, the small object features in the extracted item information fusion feature layer are obtained to obtain the small item information feature layer, specifically:

[0025] Pooling the item information fusion feature layer according to an average pooling operation to obtain a first average pooling feature layer; pooling the item information fusion feature layer according to a maximum pooling operation to obtain a first maximum pooling feature layer;

[0026] Sequentially reducing the dimensions of the first average pooling feature layer and the first maximum pooling feature layer, and adding corresponding elements to obtain an added feature layer;

[0027] Activating the added feature layer according to the sigmoid function, and multiplying the activated added feature layer and the item information fusion feature layer by corresponding elements to obtain a channel attention feature layer;

[0028] Pooling the channel attention feature layer according to an average pooling operation to obtain a second average pooling feature layer; pooling the channel attention feature layer according to a maximum pooling operation to obtain a second maximum pooling feature layer;

[0029] Performing feature fusion on the second average pooling feature layer and the second maximum pooling feature layer to obtain a fused feature layer;

[0030] The fused feature layer is activated according to the sigmoid function, and the activated fused feature layer is multiplied by corresponding elements of the channel attention feature layer to obtain the small item information feature layer.

[0031] The present invention performs dimension reduction and pooling processing on the item information fusion feature layer, thereby reducing the amount of computation while extracting category information; and performs pooling and fusion operations on the added feature layer, so that the final small item information feature layer has the category information and location information of the target item. Therefore, the process of extracting small target features of the present invention improves the accuracy of small target detection while reducing the amount of computation.

[0032] Furthermore, the data of the fused feature layer is updated respectively according to the second fused feature layer, specifically: the high-level item information feature layer in the fused feature layer is replaced by the second fused feature layer to obtain the updated fused feature layer.

[0033] The second fusion feature layer of the present invention retains more small item feature information than the first fusion feature layer. The updated fusion feature layer obtained by replacing the first fusion feature layer with this feature layer has richer small item feature information, including category information and location information, so that the final detection result has higher accuracy.

[0034] On the other hand, an embodiment of the present invention provides a contraband detection device for X-ray security inspection images, including: a feature extraction module, a pooling module, a feature fusion module, a feature enhancement module, and a result processing module;

[0035] The feature extraction module is used to extract features from the X-ray security inspection image according to the depth-separable convolution to obtain an item information feature layer; wherein the item information feature layer includes: a low-level item information feature layer and a high-level item information feature layer; the size of the low-level item information feature layer is larger than the size of the high-level item information feature layer;

[0036] The pooling module is used to unify the size of the high-level item information feature layer according to the pooling operation to obtain an item information feature layer of a first preset size;

[0037] The feature fusion module is used to update the data of the item information feature layer according to the item information feature layer of the first preset size, and perform feature fusion on the updated item information feature layer to obtain a fused feature layer; wherein the fused feature layer includes a first fused feature layer;

[0038] The small item feature extraction module is used to extract category information and location information of the low-level item information feature layer, generate a small item information feature layer, and perform feature fusion on the small item information feature layer and the first fusion feature layer to obtain a second fusion feature layer;

[0039] The result processing module is used to update the data of the fused feature layer according to the second fused feature layer, and to set a detection frame and item information for the X-ray security inspection image according to the updated fused feature layer to obtain a detection result.

[0040] Furthermore, the feature extraction module includes: a first calculation unit, a second calculation unit and a repeated calculation unit;

[0041] The first computing unit is used to perform a convolution operation on the X-ray security inspection image using a unit convolution kernel according to a depth convolution to obtain an item information feature layer with a constant number of channels;

[0042] The second calculation unit is used to increase the number of channels of the item information feature layer with the same number of channels according to point-by-point convolution to obtain the low-level item information feature layer and the high-level item information feature layer;

[0043] The repetitive calculation unit is used to repeatedly perform a preset number of depthwise convolutions and pointwise convolutions on the low-layer item information feature layer or the high-layer item information feature layer to obtain a plurality of low-layer item information feature layers and a plurality of high-layer item information feature layers; the plurality of low-layer item information feature layers and the plurality of high-layer item information feature layers constitute the item information feature layer.

[0044] Furthermore, the small item feature extraction module includes: a pooling unit, a small item feature fusion unit and a small item feature extraction unit;

[0045] The pooling unit is used to unify the size of the lower-level item information feature layer according to the pooling operation to obtain an item information feature layer of a second preset size;

[0046] The small item feature fusion unit is used to fuse the semantic information and resolution information of the feature layer of the second preset size according to the feature pyramid network to obtain an item information fusion feature layer;

[0047] The small item feature extraction unit is used to extract small target features in the item information fusion feature layer to obtain the small item information feature layer.

[0048] In another aspect, a contraband detection system for X-ray security inspection images includes: a central processing unit, an X-ray security inspection image acquisition device and a display device;

[0049] The central processing unit is used to execute any one of the contraband detection methods for X-ray security inspection images as described in the embodiments of the present invention;

[0050] The X-ray security inspection image acquisition device is used to acquire the X-ray security inspection image to be analyzed and send the X-ray security inspection image to the central processor;

[0051] The display device is used to receive the detection result output by the central processing unit and display the detection result.

[0052] The present invention uses deep separable convolution to extract features from X-ray security inspection images, which has less computational complexity than traditional feature extraction networks such as VGG, ResNet, etc., while the feature extraction accuracy does not decrease, and lightweight is achieved in the feature extraction link. After the pooling operation, the feature layers of item information of different sizes obtained by feature extraction can ensure that both the item contour and position information of the large-size feature layer and the semantic information of the small-size feature layer are retained during feature fusion, so that the fused feature layer retains more item information in the X-ray security inspection image and improves the detection accuracy. The shallow item feature information has more small item information because of its large size. After the category information and position information are extracted, it is fused with the first fused feature layer to obtain a second fused feature layer, and the second fused feature layer has more small item features than the first fused feature layer. The fused feature layer is updated according to the second fused feature layer, and the detection frame and item information are added to the X-ray security inspection image according to the updated fused feature layer, which can more accurately display the category and specific location of small items, and has higher detection accuracy for small target contraband than the traditional target detection model. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] Figure 1 It is a flow chart of an embodiment of a method for detecting contraband in X-ray security inspection images provided by the present invention;

[0054] Figure 2 It is a flow chart of another embodiment of the method for detecting contraband in X-ray security inspection images provided by the present invention;

[0055] Figure 3 It is a flow chart of another embodiment of the method for detecting contraband in X-ray security inspection images provided by the present invention;

[0056] Figure 4 It is a flow chart of another embodiment of the method for detecting contraband in X-ray security inspection images provided by the present invention;

[0057] Figure 5 It is a structural schematic diagram of an embodiment of a contraband detection device for X-ray security inspection images provided by the present invention;

[0058] Figure 6 It is a structural schematic diagram of another embodiment of the contraband detection device for X-ray security inspection images provided by the present invention;

[0059] Figure 7 It is a structural schematic diagram of another embodiment of the contraband detection device for X-ray security inspection images provided by the present invention;

[0060] Figure 8 It is a structural schematic diagram of an embodiment of a contraband detection system for X-ray security inspection images provided by the present invention; DETAILED DESCRIPTION

[0061] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0062] Embodiment 1

[0063] Please refer to Figure 1 , a flow chart of an embodiment of a method for detecting contraband in X-ray security inspection images provided by the present invention mainly includes steps 101 to 105, which are specifically as follows:

[0064] Step 101: extract features from the X-ray security inspection image according to depthwise separable convolution to obtain an item information feature layer; wherein the item information feature layer includes: a low-layer item information feature layer and a high-layer item information feature layer; the size of the low-layer item information feature layer is larger than the size of the high-layer item information feature layer.

[0065] In this embodiment, the YOLOv4 network model can be used as the algorithm basis for small target detection, and the backbone network of the original YOLOv4 network model is replaced by the MobileNetv3 network. The MobileNetv3 network uses deep separable convolution to extract features from X-ray security inspection images to obtain an item information feature layer, which is composed of three effective feature layers of different sizes, including: a low-level item information feature layer and a high-level item information feature layer, and the shallow item feature layer has a larger size than the high-level item information feature layer after a small amount of convolution and downsampling.

[0066] Step 102: unify the size of the high-level item information feature layer according to the pooling operation to obtain an item information feature layer of a first preset size.

[0067] In this embodiment, the high-level item information feature layer needs to be pooled through the pooling layer, and the spatial pyramid pooling network can be used for the pooling process to output a feature layer with uniform feature size, thereby avoiding the disadvantage of changing the original proportion of features due to adjusting the image to a fixed size. At the same time, the spatial pyramid pooling network can be used to fuse features of different sizes, realize the fusion of multiple receptive fields, and extract spatial feature information of different sizes; this is beneficial to improve the robustness of the recognition model when the target sizes in the X-ray inspection images vary greatly.

[0068] Step 103: updating the data of the item information feature layers respectively according to the item information feature layers of the first preset size, and fusing the updated item information feature layers to obtain a fused feature layer; wherein the fused feature layer includes the first fused feature layer;

[0069] In this embodiment, the feature fusion of the updated item information feature layer can be completed using a path aggregation network, and the output fusion feature layer retains more feature information of the X-ray security inspection image through top-down and bottom-up bidirectional fusion of the path aggregation network. The fusion feature layer can include three feature layers of different sizes including a first fusion feature layer, wherein the size of the first fusion feature layer is 13x13.

[0070] In this embodiment, the data of the item information feature layer is updated respectively according to the item information feature layer of the first preset size, specifically: the high-level item information feature layer in the item information feature layer is replaced with the item information feature layer of the first preset size to obtain the updated item information feature layer.

[0071] Step 104: extracting category information and location information from the lower-level item information feature layer to generate a small item information feature layer, and performing feature fusion on the small item information feature layer and the first fused feature layer to obtain a second fused feature layer.

[0072] In this embodiment, a small object feature extraction network is used to extract feature information of small items, including category information and location information of small items. At the same time, the input low-level item information feature layer has a larger size and retains more features of the X-ray security inspection image, so it has more feature information of small items, so that the small object extraction network generates a small item information feature layer with complete and rich feature information. Among them, the size of the small item information feature layer is the same as the size of the first fusion feature layer.

[0073] Step 105: updating the data of the fused feature layer respectively according to the second fused feature layer, and setting a detection frame and item information for the X-ray security inspection image according to the updated fused feature layer to obtain a detection result.

[0074] In this embodiment, the prediction layer in YOLOv4 can be used to set the detection frame and item information for the X-ray security inspection image according to the updated fusion feature layer. The detection frame marks the location of each item in the X-ray security inspection image, and the item information shows the category information of each item. The small target detection model of this embodiment can identify small-sized contraband, where small-sized contraband includes: lighters, small blades, and small scissors; it solves the problem of traditional item recognition models missing small-sized contraband.

[0075] In this embodiment, the data of the fused feature layer is updated respectively according to the second fused feature layer, specifically: the high-level item information feature layer in the fused feature layer is replaced by the second fused feature layer to obtain the updated fused feature layer.

[0076] Please refer to Figure 2 , a flow chart of another embodiment of the method for detecting contraband in X-ray security inspection images provided by the present invention mainly includes steps 201 to 203, which are as follows:

[0077] In this embodiment, step 101 is specifically steps 201 to 203 .

[0078] Step 201: Perform a convolution operation on the X-ray security inspection image using a unit convolution kernel according to depth convolution to obtain an object information feature layer with a constant number of channels.

[0079] In this embodiment, a depth convolution is first performed on the X-ray security inspection image, and the depth convolution is to use a unit convolution kernel to perform a convolution operation on the X-ray security inspection image. The unit convolution kernel size can be (3,3,1), and the number of unit convolution kernels is the same as the number of channels of the X-ray security inspection image. After the convolution operation, an item information feature layer with an unchanged number of channels is obtained.

[0080] Step 202: increasing the number of channels of the item information feature layer with the same number of channels according to point-by-point convolution to obtain the low-level item information feature layer and the high-level item information feature layer.

[0081] In this embodiment, a convolution kernel of size (1, 1, M) is used, where M is the number of channels of the item information feature layer with a constant number of channels. After performing a preset number of convolution operations on the item information feature layer with a constant number of channels according to the convolution kernel, a low-level item information feature layer or a high-level item information feature layer with a number of channels equal to the preset number is obtained.

[0082] Step 203: Repeat the depthwise convolution and pointwise convolution of the item information feature layer for a preset number of times to obtain a plurality of low-layer item information feature layers and a plurality of high-layer item information feature layers; the plurality of low-layer item information feature layers and the plurality of high-layer item information feature layers constitute the item information feature layer.

[0083] Please refer to Figure 3 , a flow chart of another embodiment of the method for detecting contraband in X-ray security inspection images provided by the present invention mainly includes steps 301 to 303, which are as follows:

[0084] In this embodiment, step 104 is specifically steps 301 to 303 .

[0085] Step 301: unify the size of the lower-level item information feature layer according to a pooling operation to obtain an item information feature layer of a second preset size.

[0086] In this embodiment, a spatial pyramid pooling network is used to perform a pooling operation on the low-level item information feature layer to unify the feature sizes in the low-level item information feature layer.

[0087] Step 302: The semantic information and resolution information of the feature layer of the second preset size are fused according to the feature pyramid network to obtain an object information fused feature layer.

[0088] In this embodiment, the feature pyramid network further extracts features from the feature layer of the second preset size, and obtains deeper feature information through several downsamplings; at the same time, the shallow feature information is retained, and the high-level semantic information and shallow resolution information are integrated, and finally the item information fusion feature layer with more small item category information and positioning information is obtained. In this embodiment, the feature pyramid network downsamples the input feature layer of the second preset size and performs 5 convolutions to obtain the next layer, repeats the same operation twice to the deep feature layer, performs upsampling and 5 convolutions on it, and fuses the features with the corresponding shallow feature layer through lateral connection to obtain the item information fusion feature layer. The feature pyramid network can obtain the features of small-sized contraband through semantic information, thereby improving the detection accuracy of small targets.

[0089] Step 303: extracting small object features in the item information fusion feature layer to obtain the small item information feature layer.

[0090] In this embodiment, a convolutional attention mechanism module is used to extract small target features in the item information fusion feature layer, and the small target features include: category features and position features.

[0091] Please refer to Figure 4 , a flow chart of another embodiment of the method for detecting contraband in X-ray security inspection images provided by the present invention mainly includes steps 401 to 406, which are as follows:

[0092] In this embodiment, step 303 is specifically steps 401 to 406.

[0093] Step 401: pooling the item information fusion feature layer according to an average pooling operation to obtain a first average pooling feature layer; pooling the item information fusion feature layer according to a maximum pooling operation to obtain a first maximum pooling feature layer.

[0094] In this embodiment, the spatial information of the item information fusion feature layer is aggregated by performing an average pooling operation and a maximum pooling operation on the item information fusion feature layer.

[0095] Step 402: sequentially perform dimensionality reduction on the first average pooling feature layer and the first maximum pooling feature layer, and add corresponding elements to obtain an added feature layer.

[0096] In this embodiment, the added feature layer retains the category information of small items in the X-ray security inspection image, and the reduction operation can reduce the number of parameters in the calculation process.

[0097] Step 403: Activate the added feature layer according to the sigmoid function, and multiply the activated added feature layer and the item information fusion feature layer by corresponding elements to obtain a channel attention feature layer.

[0098] In this embodiment, the expressions of steps 401-403 are:

[0099]

[0100]

[0101] Among them, M c (F) is the activated and added feature layer, F is the item information fusion feature layer, AvgPool is the average pooling operation, MaxPool is the maximum pooling operation, MLP is the multi-layer perceptron processing, and σ is the sigmoid function; is the first average pooling feature layer, is the first maximum pooling feature layer, W0 is the weight parameter of the first layer of the multilayer perceptron, and W1 is the weight parameter of the second layer of the multilayer perceptron. In this embodiment, a convolutional attention mechanism module is used to extract the feature information of small items; steps 401-403 are implemented according to the channel attention module to obtain a one-dimensional channel attention feature layer with classification information.

[0102] Step 404: pooling the channel attention feature layer according to the average pooling operation to obtain a second average pooling feature layer; pooling the channel attention feature layer according to the maximum pooling operation to obtain a second maximum pooling feature layer.

[0103] Step 405: performing feature fusion on the second average pooling feature layer and the second maximum pooling feature layer to obtain a fused feature layer.

[0104] In this embodiment, the second average pooling feature layer and the second maximum pooling feature layer are fused to aggregate the location information of small items.

[0105] Step 406: Activate the fused feature layer according to the sigmoid function, and multiply the activated fused feature layer and the channel attention feature layer by corresponding elements to obtain the small item information feature layer.

[0106] In this embodiment, the expressions of steps 404-406 are:

[0107]

[0108] Among them, M s(F) is the activated and fused feature layer, F′ is the channel attention feature layer, AvgPool is the average pooling operation, MaxPool is the maximum pooling operation, and f 7×7 is a 7×7 convolution filter, and σ is a sigmoid function; is the second average pooling feature layer, It is the second maximum pooling feature layer.

[0109] In this embodiment, after the item information fusion feature layer is processed by the convolutional attention module, a small item information feature layer is obtained, which is expressed as:

[0110]

[0111]

[0112] Among them, F′ is the channel attention feature layer, M c (F) is the activated and added feature layer, F is the item information fusion feature layer, F″ is the small item information feature layer, M s (F) is the activated fused feature layer. The activated added feature layer is multiplied by the corresponding elements of the item information fused feature layer to obtain the channel attention feature layer; the activated fused feature layer is multiplied by the corresponding elements of the channel attention feature layer to obtain the small item information feature layer.

[0113] Please refer to Figure 5 , a schematic diagram of the structure of an embodiment of a contraband detection device for X-ray security inspection images provided by the present invention, which mainly includes: a feature extraction module 501, a pooling module 502, a feature fusion module 503, a small item feature extraction module 504 and a result processing module 505.

[0114] In this embodiment, the feature extraction module 501 is used to perform feature extraction on the X-ray security inspection image according to the depthwise separable convolution to obtain an item information feature layer; wherein the item information feature layer includes: a low-layer item information feature layer and a high-layer item information feature layer; the size of the low-layer item information feature layer is larger than the size of the high-layer item information feature layer.

[0115] The pooling module 502 is used to unify the size of the high-level object information feature layer according to the pooling operation after the feature extraction module 501 performs feature extraction, so as to obtain the object information feature layer of the first preset size.

[0116] The feature fusion module 503 is used to update the data of the item information feature layer according to the item information feature layer of the first preset size after the pooling module 502 pools the item information feature layer, and perform feature fusion on the updated item information feature layer to obtain a fused feature layer; wherein the fused feature layer includes the first fused feature layer.

[0117] The small item feature extraction module 504 is used to extract the category information and position information of the lower-level item information feature layer after the feature fusion module 503 performs feature fusion, generate a small item information feature layer, and perform feature fusion on the small item information feature layer and the first fused feature layer to obtain a second fused feature layer.

[0118] The result processing module 505 is used to update the data of the fused feature layer according to the second fused feature layer after the small item feature extraction module 504 performs feature fusion on the small item information feature layer and the first fused feature layer to obtain a second fused feature layer, and to set a detection frame and item information for the X-ray security inspection image according to the updated fused feature layer to obtain a detection result.

[0119] Please refer to Figure 6 , a schematic structural diagram of another embodiment of the contraband detection device for X-ray security inspection images provided by the present invention. Figure 6 and Figure 5 The difference is that the feature extraction module 501 further includes: a first calculation unit 601, a second calculation unit 602 and a repeated calculation unit 603.

[0120] In this embodiment, the first computing unit 601 is used to perform a convolution operation on the X-ray security inspection image using a unit convolution kernel according to depth convolution to obtain an object information feature layer with a constant number of channels.

[0121] The second computing unit 602 is used to perform a convolution operation on the X-ray security inspection image using a unit convolution kernel in the first computing unit 601 to obtain an item information feature layer with a constant number of channels, and then increase the number of channels of the item information feature layer with a constant number of channels according to point-by-point convolution to obtain the low-level item information feature layer and the high-level item information feature layer.

[0122] The repeated calculation unit 603 is used to repeatedly perform a preset number of depthwise convolutions and pointwise convolutions on the low-layer item information feature layer or the high-layer item information feature layer to obtain a plurality of low-layer item information feature layers and a plurality of high-layer item information feature layers; the plurality of low-layer item information feature layers and the plurality of high-layer item information feature layers constitute the item information feature layer.

[0123] Please refer to Figure 7, a schematic structural diagram of another embodiment of the contraband detection device for X-ray security inspection images provided by the present invention. Figure 7 and Figure 5 The difference is that the small item feature extraction module 504 further includes: a pooling unit 701, a small item feature fusion unit 702 and a small item feature extraction unit 703.

[0124] In this embodiment, the pooling unit 701 is used to unify the size of the lower-level item information feature layer according to the pooling operation to obtain the item information feature layer of the second preset size.

[0125] The small item feature fusion unit 702 is used to unify the size of the low-level item information feature layer in the pooling unit 701 to obtain the item information feature layer of the second preset size, and then fuse the semantic information and resolution information of the feature layer of the second preset size according to the feature pyramid network to obtain the item information fusion feature layer.

[0126] The small item feature extraction unit 703 is used to fuse the semantic information and resolution information of the feature layer of the second preset size in the small item feature fusion unit 702 to obtain the item information fusion feature layer, extract the small target features in the item information fusion feature layer, and multiply the corresponding elements of the activated fused feature layer with the channel attention feature layer to obtain the small item information feature layer.

[0127] Please refer to Figure 8 , a structural schematic diagram of an embodiment of a contraband detection system for X-ray security inspection images provided by the present invention, which mainly includes: a central processing unit 801, an X-ray security inspection image acquisition device 802 and a display device 803.

[0128] In this embodiment, the central processing unit 801 is used to execute the method for detecting contraband for X-ray security inspection images described in any one of steps 101-105, steps 201-203, steps 301-303 and steps 401-406.

[0129] The X-ray security inspection image acquisition device 802 is used to acquire the X-ray security inspection image to be analyzed and send the X-ray security inspection image to the central processor.

[0130] The display device 803 is used to receive the detection result output by the central processing unit and display the detection result.

[0131] As can be seen from the above, the present invention can use MobileNetv3 as the backbone feature extraction network, select the low-level item information feature layer through the residual edge to extract the features of small target contraband, and obtain the item information fusion feature layer with rich semantic information and resolution information through the feature pyramid network, so as to obtain more feature information about small target contraband. After being processed by the convolution attention module, the small target recognition network will pay more attention to small target contraband. By fusing the feature information obtained with the backbone feature extraction network MobileNetv3, the feature information of small target contraband is further enriched, and the detection accuracy of small target contraband is effectively improved. The present invention adopts deep separable convolution to reduce the amount of model calculation, reduce the overall model parameter amount, improve the operation efficiency, and meet the requirements of lightweight. It solves the problem that the small target recognition model is difficult to deploy on mobile devices and X-ray security inspection machines with poor hardware conditions. In addition, while being lightweight, the detection of small target contraband and various types of contraband are taken into account at the same time, ensuring the detection accuracy.

[0132] The specific embodiments described above further illustrate the purpose, technical solutions and beneficial effects of the present invention. It should be understood that the above description is only a specific embodiment of the present invention and is not intended to limit the scope of protection of the present invention. It is particularly pointed out that for those skilled in the art, any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention should be included in the scope of protection of the present invention.

Claims

1. A method for detecting contraband in X-ray security inspection images, characterized in that: include: Performing feature extraction on the X-ray security inspection image according to the depthwise separable convolution to obtain an item information feature layer; wherein the item information feature layer comprises: a low-level item information feature layer and a high-level item information feature layer; the size of the low-level item information feature layer is larger than the size of the high-level item information feature layer; Unifying the size of the high-level item information feature layer according to the pooling operation to obtain an item information feature layer of a first preset size; According to the item information feature layer of the first preset size, data of the item information feature layer is updated respectively, and features of the updated item information feature layer are fused to obtain a fused feature layer; wherein the fused feature layer includes a first fused feature layer; Extracting category information and location information from the low-level item information feature layer to generate a small item information feature layer, and performing feature fusion on the small item information feature layer and the first fused feature layer to obtain a second fused feature layer; updating data of the fused feature layer respectively according to the second fused feature layer, and setting a detection frame and item information for the X-ray security inspection image according to the updated fused feature layer to obtain a detection result; The step of extracting the category information and location information of the low-level item information feature layer to generate the small item information feature layer is as follows: Unifying the size of the lower-level item information feature layer according to the pooling operation to obtain an item information feature layer of a second preset size; fusing the semantic information and the resolution information of the feature layer of the second preset size according to the feature pyramid network to obtain an object information fusion feature layer; Extracting small object features in the item information fusion feature layer to obtain the small item information feature layer; The small object features in the item information fusion feature layer are extracted to obtain the small item information feature layer, specifically: Pooling the item information fusion feature layer according to an average pooling operation to obtain a first average pooling feature layer; pooling the item information fusion feature layer according to a maximum pooling operation to obtain a first maximum pooling feature layer; Sequentially reducing the dimensions of the first average pooling feature layer and the first maximum pooling feature layer, and adding corresponding elements to obtain an added feature layer; Activating the added feature layer according to the sigmoid function, and multiplying the activated added feature layer and the item information fusion feature layer by corresponding elements to obtain a channel attention feature layer; Pooling the channel attention feature layer according to an average pooling operation to obtain a second average pooling feature layer; pooling the channel attention feature layer according to a maximum pooling operation to obtain a second maximum pooling feature layer; Performing feature fusion on the second average pooling feature layer and the second maximum pooling feature layer to obtain a fused feature layer; The fused feature layer is activated according to the sigmoid function, and the activated fused feature layer is multiplied by corresponding elements of the channel attention feature layer to obtain the small item information feature layer.

2. A method for detecting contraband items in X-ray security inspection images as claimed in claim 1, characterized in that: The feature extraction of the X-ray security inspection image is performed according to the depthwise separable convolution to obtain the item information feature layer, specifically: The depth-wise separable convolution includes depth-wise convolution and point-wise convolution; Performing a convolution operation on the X-ray security inspection image using a unit convolution kernel according to a depthwise convolution to obtain an item information feature layer with a constant number of channels; Increasing the number of channels of the item information feature layer whose number of channels remains unchanged according to point-by-point convolution to obtain the low-level item information feature layer and the high-level item information feature layer; Repeating the depthwise convolution and pointwise convolution for a preset number of times on the item information feature layer to obtain a plurality of low-level item information feature layers and a plurality of high-level item information feature layers; The plurality of lower-level item information feature layers and the plurality of higher-level item information feature layers constitute the item information feature layer.

3. The method for detecting contraband items in X-ray security inspection images according to claim 1, characterized in that: The updating of the data of the item information feature layer according to the item information feature layer of the first preset size is specifically: replacing the high-level item information feature layer in the item information feature layer with the item information feature layer of the first preset size to obtain the updated item information feature layer.

4. The method for detecting contraband items in X-ray security inspection images according to claim 1, characterized in that: The updating of data of the fused feature layer according to the second fused feature layer is specifically: replacing the high-level item information feature layer in the fused feature layer with the second fused feature layer to obtain the updated fused feature layer.

5. A contraband detection device for X-ray security inspection images, characterized in that: include: Feature extraction module, pooling module, feature fusion module, small item feature extraction module and result processing module; The feature extraction module is used to extract features from the X-ray security inspection image according to the depth-separable convolution to obtain an item information feature layer; wherein the item information feature layer includes: a low-level item information feature layer and a high-level item information feature layer; the size of the low-level item information feature layer is larger than the size of the high-level item information feature layer; The pooling module is used to unify the size of the high-level item information feature layer according to the pooling operation to obtain an item information feature layer of a first preset size; The feature fusion module is used to update the data of the item information feature layer according to the item information feature layer of the first preset size, and perform feature fusion on the updated item information feature layer to obtain a fused feature layer; wherein the fused feature layer includes a first fused feature layer; The small item feature extraction module is used to extract category information and location information of the low-level item information feature layer, generate a small item information feature layer, and perform feature fusion on the small item information feature layer and the first fusion feature layer to obtain a second fusion feature layer; The result processing module is used to update the data of the fused feature layer according to the second fused feature layer, and set a detection frame and item information for the X-ray security inspection image according to the updated fused feature layer to obtain a detection result; Wherein, the small item feature extraction module includes: a pooling unit, a small item feature fusion unit and a small item feature extraction unit; The pooling unit is used to unify the size of the lower-level item information feature layer according to the pooling operation to obtain an item information feature layer of a second preset size; The small item feature fusion unit is used to fuse the semantic information and resolution information of the feature layer of the second preset size according to the feature pyramid network to obtain an item information fusion feature layer; The small item feature extraction unit is used to extract small target features in the item information fusion feature layer, and multiply the activated fused feature layer and the channel attention feature layer by corresponding elements to obtain the small item information feature layer; Among them, the small target features in the extracted object information fusion feature layer are specifically: Pooling the item information fusion feature layer according to an average pooling operation to obtain a first average pooling feature layer; pooling the item information fusion feature layer according to a maximum pooling operation to obtain a first maximum pooling feature layer; Sequentially reducing the dimensions of the first average pooling feature layer and the first maximum pooling feature layer, and adding corresponding elements to obtain an added feature layer; Activating the added feature layer according to the sigmoid function, and multiplying the activated added feature layer and the item information fusion feature layer by corresponding elements to obtain a channel attention feature layer; Pooling the channel attention feature layer according to an average pooling operation to obtain a second average pooling feature layer; pooling the channel attention feature layer according to a maximum pooling operation to obtain a second maximum pooling feature layer; Performing feature fusion on the second average pooling feature layer and the second maximum pooling feature layer to obtain a fused feature layer; The fused feature layer is activated according to the sigmoid function, and the activated fused feature layer is multiplied by corresponding elements of the channel attention feature layer to obtain the small item information feature layer.

6. The device for detecting contraband items for X-ray security inspection images as claimed in claim 5, characterized in that: The feature extraction module includes: a first calculation unit, a second calculation unit and a repeated calculation unit; The first computing unit is used to perform a convolution operation on the X-ray security inspection image using a unit convolution kernel according to a depth convolution to obtain an item information feature layer with a constant number of channels; The second calculation unit is used to increase the number of channels of the item information feature layer with the same number of channels according to point-by-point convolution to obtain the low-level item information feature layer and the high-level item information feature layer; The repetitive calculation unit is used to repeatedly perform a preset number of depthwise convolutions and pointwise convolutions on the low-layer item information feature layer or the high-layer item information feature layer to obtain a plurality of low-layer item information feature layers and a plurality of high-layer item information feature layers; the plurality of low-layer item information feature layers and the plurality of high-layer item information feature layers constitute the item information feature layer.

7. A contraband detection system for X-ray security inspection images, characterized in that: include: Central processing unit, X-ray security image acquisition equipment and display device; Wherein, the central processing unit is used to execute the contraband detection method for X-ray security inspection images as described in any one of claims 1 to 4; The X-ray security inspection image acquisition device is used to acquire the X-ray security inspection image to be analyzed and send the X-ray security inspection image to the central processor; The display device is used to receive the detection result output by the central processing unit and display the detection result.

Images