A contraband detection method and device based on dual-view security images

Abstract

The application belongs to the security check technical field, and discloses a contraband detection method and device based on double-view security check images, which comprises the following steps: acquiring a first-view security check image and a second-view security check image of luggage; processing the first-view security check image and the second-view security check image through a pre-trained contraband detection model to obtain a contraband detection result of the luggage; wherein the contraband detection model is used for detecting whether there is contraband in the luggage according to the first-view security check image and the second-view security check image, and classifying and positioning the contraband in the luggage when the luggage contains contraband; and outputting the contraband detection result of the luggage. The application can solve the problem of poor imaging angle of security check images, mutual shielding and other factors affecting target detection accuracy through double-view feature fusion, and achieve the effect of improving detection accuracy.

Description

TECHNICAL FIELD

[0001] The present application relates to the security inspection technical field, and in particular to a contraband detection method and device based on dual-view security inspection images. BACKGROUND

[0002] With the rapid economic development, there is a high requirement for security detection in the fields of aviation, rail transportation and logistics. The current security inspection equipment that has been put into actual use mainly includes metal equipment detection and X-ray security inspection equipment. Among them, the application range of X-ray security inspection equipment is wider, and it has better detection capability for various contraband. At present, more advanced dual-view X-ray security inspection machines are used in many important occasions. Compared with the traditional X-ray security inspection machine that can only present one top view perspective, the dual-view X-ray security inspection machine can simultaneously scan out the perspective view of the top view and side view of the object, has a wider detection field of view, and is helpful to detect more small, hidden and deliberately hidden contraband.

[0003] However, the current dual-view X-ray security inspection machine still relies mainly on manual detection, which requires a large amount of manpower. Although the dual-view X-ray security inspection machine provides more information for identification, it undoubtedly puts higher working pressure on the security inspection personnel. Once the security inspection personnel work tiredly, it will lead to mis-detection or missed detection of contraband, affecting the security inspection accuracy. It can be seen that the prior art has the problem of low detection accuracy. SUMMARY

[0004] The present application provides a contraband detection method and device based on dual-view security inspection images, which can improve the detection accuracy by using dual-view feature fusion.

[0005] In a first aspect, the embodiments of the present application provide a contraband detection method based on dual-view security inspection images, which comprises:

[0006] obtaining a first-view security inspection image and a second-view security inspection image of luggage;

[0007] processing the first-view security inspection image and the second-view security inspection image through a pre-trained contraband detection model to obtain a contraband detection result of the luggage; wherein the contraband detection model is used to detect whether there is contraband in the luggage according to the first-view security inspection image and the second-view security inspection image, and when there is contraband in the luggage, to classify and locate the contraband in the luggage;

[0008] outputting the contraband detection result of the luggage.

[0009] In one of the embodiments, the contraband detection model is based on a one-stage target detection algorithm YOLOv4 target detection framework, the basic target detection framework comprising a feature extraction backbone network, an SPP spatial pyramid pooling layer, a PANet network and a detection layer; the feature extraction backbone network comprising a first primary feature extraction network, a second primary feature extraction network, a feature mapping module and a secondary feature extraction network;

[0010] The first-view security image and the second-view security image are processed by the pre-trained contraband detection model to obtain the contraband detection result of the luggage, comprising:

[0011] The first-view feature of the first-view security image is extracted by the first primary feature extraction network, and the second-view feature of the second-view security image is extracted by the second primary feature extraction network;

[0012] The first-view feature and the second-view feature are fused by the feature mapping module, and the obtained fused feature is input into the secondary feature extraction network;

[0013] The fused feature is extracted by the secondary feature extraction network to obtain the first feature layer, the second feature layer and the third feature layer arranged in descending order of scale;

[0014] The third feature layer is processed by the SPP spatial pyramid pooling layer, and the first feature layer, the second feature layer and the third feature layer processed by the SPP spatial pyramid pooling layer are input into the PANet network for processing to obtain the detection feature;

[0015] The detection feature is input into the detection layer to obtain the contraband detection result of the luggage.

[0016] In one of the embodiments, the first primary feature extraction network and the second primary feature extraction network each comprise two residual modules, and the parameters of the two residual modules included in the first primary feature extraction network are not communicated with the parameters of the two residual modules included in the second primary feature extraction network.

[0017] In one of the embodiments, the feature extraction backbone network comprises two feature mapping modules;

[0018] The first-view feature of the first-view security image is extracted by the first primary feature extraction network, and the second-view feature of the second-view security image is extracted by the second primary feature extraction network, comprising:

[0019] The first-view feature of the first-view security image is preliminarily extracted by the first residual module of the first primary feature extraction network, and the first-view feature is input into the first feature mapping module in the feature extraction backbone network;

[0020] The first residual module of the second primary feature extraction network preliminarily extracts the second-view feature of the second-view security image, and inputs the second-view feature into the first feature mapping module in the feature extraction backbone network and the second residual module of the second primary feature extraction network respectively;

[0021] The second residual module of the second primary feature extraction network reextracts the second-view feature, and inputs the second-view feature reextracted to the second feature mapping module in the feature extraction backbone network;

[0022] The feature mapping module fuses the first-view feature and the second-view feature, comprising:

[0023] The first feature mapping module in the feature extraction backbone network fuses the input first-view feature and second-view feature, inputs the primary fusion feature obtained to the second residual module of the first primary feature extraction network for re-extraction, and inputs the secondary fusion feature reextracted to the second feature mapping module in the feature extraction backbone network;

[0024] The second feature mapping module in the feature extraction backbone network fuses the secondary fusion feature and the second-view extraction feature to obtain the fusion feature.

[0025] In one of the embodiments, each feature mapping module comprises an up-down sampling module and a convolution attention mechanism module;

[0026] The first feature mapping module in the feature extraction backbone network fuses the input first-view feature and second-view feature, comprising:

[0027] The up-down sampling module of the first feature mapping module up-samples and down-samples the first-view feature and the second-view feature, and inputs the features obtained by up-sampling and down-sampling to the convolution attention mechanism module of the first feature mapping module to obtain the primary fusion feature output by the convolution attention mechanism module;

[0028] The second feature mapping module in the feature extraction backbone network fuses the secondary fusion feature and the second-view extraction feature to obtain the fusion feature, comprising:

[0029] The up-down sampling module of the second feature mapping module up-samples and down-samples the secondary fusion feature and the second-view extraction feature, and inputs the features obtained by up-sampling and down-sampling to the convolution attention mechanism module of the second feature mapping module to obtain the fusion feature output by the convolution attention mechanism module.

[0030] In one of the embodiments, the secondary feature extraction network comprises three residual modules connected in sequence; inputting the obtained fusion feature into the secondary feature extraction network comprises: inputting the obtained fusion feature into a first residual module of the secondary feature extraction network;

[0031] extracting the fusion feature through the secondary feature extraction network to obtain a first feature layer, a second feature layer and a third feature layer arranged in descending order of scale, comprising:

[0032] extracting the fusion feature through the first residual module of the secondary feature extraction network to obtain the first feature layer;

[0033] inputting the first feature layer into a second residual module of the secondary feature extraction network for feature extraction to obtain the second feature layer;

[0034] inputting the second feature layer into a third residual module of the secondary feature extraction network for feature extraction to obtain the third feature layer.

[0035] In one of the embodiments, the first-view security image is an X-ray image collected by a dual-view X-ray security inspection machine on a vertical view, and the second-view security image is an X-ray image collected by the dual-view X-ray security inspection machine on a horizontal view.

[0036] In a second aspect, the embodiments of the present application provide a contraband detection device based on dual-view security images, which comprises:

[0037] a security image acquisition module, configured to acquire a first-view security image and a second-view security image of luggage;

[0038] a contraband detection module, configured to process the first-view security image and the second-view security image through a pre-trained contraband detection model to obtain a contraband detection result of the luggage; wherein the contraband detection model is configured to detect whether there is contraband in the luggage according to the first-view security image and the second-view security image, and classify and locate the contraband in the luggage when there is contraband in the luggage;

[0039] a detection result output module, configured to output the contraband detection result of the luggage.

[0040] In a third aspect, the embodiments of the present application provide an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to perform the steps of the contraband detection method based on dual-view security images according to any of the above embodiments.

[0041] In a fourth aspect, the embodiments of the present application provide a computer readable storage medium, which stores a computer program. The computer program is executed by a processor to implement the steps of the method for detecting contraband based on dual-view security images according to any of the above embodiments.

[0042] Compared with the prior art, the technical solutions provided by the embodiments of the present application have at least the following beneficial effects:

[0043] The method for detecting contraband based on dual-view security images provided by the embodiments of the present application can obtain a first-view security image and a second-view security image of luggage. The first-view security image and the second-view security image are processed by a pre-trained contraband detection model to obtain a contraband detection result of the luggage. The contraband detection model is used to detect whether there is contraband in the luggage according to the first-view security image and the second-view security image, and classify and locate the contraband in the luggage when there is contraband in the luggage. The contraband detection result of the luggage is output. The above method can use the pre-trained contraband detection model to detect the dual-view security images to obtain the contraband detection result of the luggage. The problem of poor imaging angle and mutual shielding of the security images affecting the target detection accuracy is solved by feature fusion between the dual-view security images. Not only the labor cost can be reduced, but also the false detection or missed detection caused by fatigue during manual detection can be avoided, so as to improve the detection accuracy and reduce the missed detection rate. BRIEF DESCRIPTION OF DRAWINGS

[0044] Figure 1 An application scenario diagram of the method for detecting contraband based on dual-view security images provided by an example embodiment of the present application is shown.

[0045] Figure 2 A flowchart of the method for detecting contraband based on dual-view security images provided by an example embodiment of the present application is shown.

[0046] Figure 3 A structure diagram of the contraband detection model provided by an example embodiment of the present application is shown.

[0047] Figure 4 A flowchart of the contraband detection steps provided by an example embodiment of the present application is shown.

[0048] Figure 5 A schematic diagram of the down-sampling and up-sampling processing process provided by an example embodiment of the present application is shown.

[0049] Figure 6 A structure diagram of the device for detecting contraband based on dual-view security images provided by another example embodiment of the present application is shown. DETAILED DESCRIPTION

[0050] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0051] This application provides a method for detecting contraband based on dual-view security inspection images, which can be applied to... Figure 1 In the application scenarios shown, the above method can be specifically applied to dual-view X-ray security inspection machines. Since dual-view X-ray security inspection machines can simultaneously obtain two X-ray images from different angles for judgment, they have the advantage of more imaging information than traditional single-view X-ray security inspection machines. At the same time, because the images are taken from two perspectives, it helps to solve complex occlusion and contour problems, thereby greatly improving the detection accuracy of the entire security inspection.

[0052] This explanation will take a dual-view X-ray security inspection machine as an example. Please refer to [link / reference]. Figure 2 The method specifically includes the following steps:

[0053] Step S1: Obtain first-view security inspection images and second-view security inspection images of the luggage.

[0054] In some embodiments of this example, the first-view security inspection image can be an X-ray image of the luggage captured by the dual-view X-ray security inspection machine in the vertical view, i.e., a vertical view, and the second-view security inspection image can be an X-ray image of the luggage captured by the dual-view X-ray security inspection machine in the horizontal view, i.e., a horizontal view.

[0055] In practice, the dual-view X-ray security inspection machine simultaneously acquires first-view and second-view security inspection images of the luggage. Having an additional viewpoint means more visual information, which helps the contraband detection model extract more features, leading to more accurate target detection.

[0056] Step S2: The first-view security inspection image and the second-view security inspection image are processed by a pre-trained contraband detection model to obtain the contraband detection results of the luggage.

[0057] The contraband detection model is used for detecting whether the luggage contains contraband according to the first-view security image and the second-view security image, and classifying and positioning the contraband in the luggage when the luggage contains the contraband, that is, the contraband detection model needs to determine the category information of the contraband and the position region information of the contraband at the same time. In specific implementation, the contraband detection model in this embodiment can be implemented by using a common target detection algorithm such as YOLOv4 or SSD in the prior art. The contraband can be a knife, scissors, a dangerous explosive, a gun, etc., and the detected contraband can be marked with a bounding box according to the position region information.

[0058] In step S3, the contraband detection result of the luggage is output.

[0059] The contraband detection result can be a first detection result indicating that the luggage contains contraband or a second detection result indicating that the luggage does not contain contraband, and the first detection result includes the classification information and the positioning information of the contraband.

[0060] In some embodiments of this embodiment, step S3 can specifically include the following steps: when the contraband detection result is the first detection result indicating that the luggage contains contraband, the category and position of the contraband in the luggage are marked in the first-view security image of the luggage according to the first detection result, and the marked first-view security image is displayed.

[0061] In specific implementation, in order to timely remind the security personnel, automatic alarm can also be performed when the contraband detection result is the first detection result, and no alarm is needed when the contraband detection result is the second detection result.

[0062] The contraband detection method based on double-view security images provided in this embodiment can obtain a first-view security image and a second-view security image of luggage, process the first-view security image and the second-view security image through a pre-trained contraband detection model to obtain a contraband detection result of the luggage, wherein the contraband detection model is used for detecting whether the luggage contains contraband according to the first-view security image and the second-view security image, and classifying and positioning the contraband in the luggage when the luggage contains the contraband, and outputting the contraband detection result of the luggage. The above method can detect double-view security images through a pre-trained contraband detection model to obtain a contraband detection result of the luggage, solve the problem of poor imaging angle and mutual shielding of security images affecting the target detection accuracy through feature fusion between double-view security images, not only reduce the labor cost, but also avoid the false detection or missed detection caused by fatigue in manual detection, thereby improving the detection accuracy and reducing the missed detection rate.

[0063] In some embodiments, please refer to Figure 3The contraband detection model is based on a one-stage target detection algorithm YOLOv4, and the basic target detection framework includes a feature extraction backbone network, an SPP spatial pyramid pooling layer, a PANet network, and a detection layer.

[0064] The first primary feature extraction network, the second primary feature extraction network, and the secondary feature extraction network can be implemented by using a commonly used feature extraction network in the prior art, and the feature mapping module uses a feature mapping layer in a convolutional neural network.

[0065] In the implementation, because the use scenario of the contraband detection model is security inspection, the rapidity of detection is required to a certain extent, and therefore, in this embodiment, a one-stage target detection algorithm YOLOv4 with certain advantages in accuracy and rapidity is used as the basis, and CSPDarkNet53 is used as the feature extraction backbone network. The CSPDarkNet53 is composed of downsampling and multiple residual structure stacks, and can include 29 convolutional layers, which can fully extract useful features in the picture and is conducive to accurate detection.

[0066] Please refer to Figure 4 The step S2 specifically includes the following steps.

[0067] In step S21, a first-view feature of the first-view security inspection image is extracted by the first primary feature extraction network, and a second-view feature of the second-view security inspection image is extracted by the second primary feature extraction network.

[0068] In step S22, the first-view feature and the second-view feature are fused by the feature mapping module, and the obtained fused feature is input into the secondary feature extraction network.

[0069] The second-view feature can be mapped onto the first-view security inspection image by the feature mapping module, which can increase the information obtained by the secondary feature extraction network and enhance the target features of the first-view security inspection image, so as to fully utilize the correlation between the visual features of the two-view images, improve the detection accuracy, and reduce the probability of missed detection and false detection.

[0070] In step S23, the fused feature is extracted by the secondary feature extraction network to obtain a first feature layer, a second feature layer, and a third feature layer arranged in descending order of scale.

[0071] In some embodiments of this embodiment, the secondary feature extraction network includes three residual modules connected in sequence.

[0072] In the above embodiment, step S22 specifically comprises: inputting the obtained fusion feature into a first residual module of the secondary feature extraction network.

[0073] Specifically, step S23 can comprise the following steps:

[0074] The fusion feature is extracted by the first residual module of the secondary feature extraction network to obtain a first feature layer.

[0075] The first feature layer is input into a second residual module of the secondary feature extraction network for feature extraction to obtain a second feature layer.

[0076] The second feature layer is input into a third residual module of the secondary feature extraction network for feature extraction to obtain a third feature layer.

[0077] The three residual modules of the secondary feature extraction network can output features of different scales, and output the three feature layers in turn from large to small scales; the feature layer of large scale is conducive to positioning large targets, and the feature layer of small scale is conducive to positioning small targets.

[0078] Step S23 can use three residual blocks of different scales to extract the fusion feature after the fusion of features of different perspectives, which can enhance the performance of the feature on different scales and further fuse different features.

[0079] In step S24, the third feature layer is processed by an SPP spatial pyramid pooling layer, and the first feature layer, the second feature layer and the third feature layer processed by the SPP spatial pyramid pooling layer are input into a PANet network for processing to obtain a detection feature.

[0080] The SPP spatial pyramid pooling processing is a fusion of features of different scales and different receptive fields, which can effectively extract more spatial feature information and better fuse spatial feature information of different perspectives. Then, the features of different scales are input into the PANet network, the features of different scales are mutually up-sampled and down-sampled by the PANet network, and the up-sampled and down-sampled features are fused, so that the feature information of each scale is better fused, and the detection feature has better robustness on three scales.

[0081] In step S25, the detection feature is input into a detection layer to obtain a contraband detection result of the luggage.

[0082] The above embodiment can extract the features of two views through two primary feature extraction networks, and then fuse them through the feature mapping module, so that the features of different views are fused with each other, the visual information correlation between the two views of the dual-view X-ray security inspection machine is fully utilized, and the detection accuracy can be better improved.

[0083] Based on the above embodiments, in some embodiments, the first primary feature extraction network and the second primary feature extraction network each include two residual modules, and the parameters of the two residual modules included in the first primary feature extraction network are not communicated with the parameters of the two residual modules included in the second primary feature extraction network.

[0084] Since the first primary feature extraction network and the second primary feature extraction network are two parameter-independent feature extraction networks, different visual features can be extracted through the two feature extraction networks respectively. Specifically, the features of the dual-view security inspection images can be extracted through the two parameter-independent primary feature extraction networks, and the feature difference caused by different views is fully considered.

[0085] In specific implementation, each primary feature extraction network includes two residual modules for preliminary feature extraction, and the secondary feature extraction network performs secondary feature extraction through the three residual modules included therein.

[0086] The above-described forbidden item detection model uses five residual modules of different scales for feature extraction, which can better extract visual features due to the superiority of the residual module fitting function, and helps to improve the detection accuracy.

[0087] In some embodiments, the feature extraction backbone network includes two feature mapping modules.

[0088] Step S21 can include the following steps:

[0089] The first view feature of the first view security inspection image is preliminarily extracted through the first residual module of the first primary feature extraction network, and the first view feature is input into the first feature mapping module in the feature extraction backbone network;

[0090] The second view feature of the second view security inspection image is preliminarily extracted through the first residual module of the second primary feature extraction network, and the second view feature is input into the first feature mapping module in the feature extraction backbone network and the second residual module of the second primary feature extraction network respectively;

[0091] The second view feature is extracted again through the second residual module of the second primary feature extraction network, and the extracted second view feature is input into the second feature mapping module in the feature extraction backbone network;

[0092] Step S22 can include the following steps:

[0093] The first perspective feature and the second perspective feature are fused by a first feature mapping module in the feature extraction backbone network, the obtained primary fusion feature is input into a second residual module of the first primary feature extraction network for secondary feature extraction, and the secondary fusion feature extracted again is input into a second feature mapping module in the feature extraction backbone network.

[0094] The secondary fusion feature and the second perspective feature are fused by the second feature mapping module in the feature extraction backbone network to obtain a fusion feature.

[0095] The two residual modules included in the first primary feature extraction network not only perform preliminary feature extraction, but also integrate the mapped features to achieve better feature fusion; the second primary feature extraction network can be mapped at the positions of the first residual module and the second residual module to be fused into the first primary feature extraction network.

[0096] In some embodiments of the present embodiment, each feature mapping module includes an up-down sampling module and a convolution attention mechanism module.

[0097] The up-down sampling module is used for down-sampling and up-sampling processing of the input feature, and the specific processing process is as shown in Figure 5 The convolution attention mechanism module is an attention mechanism module of a convolution module, commonly known as a CBAM (Convolutional Block Attention Module) module, which is a spatial and channel attention mechanism module.

[0098] Specifically, the process of fusing the input feature by the feature mapping module mainly consists of up-down sampling processing and attention feature extraction. First, the feature mapping module can sort and filter the feature values of each pixel position of the input feature by the up-down sampling module, reduce the calculation amount, prevent overfitting, and increase the receptive field so that the convolution kernel in the feature mapping module can better learn the global information.

[0099] In specific implementation, the second primary feature extraction network (feature extraction network of the horizontal view) can learn the useful part of the first perspective feature (vertical view feature) through continuous training. Since the features of the two perspectives are not the same, the feature mapping module can filter the useless part of the second perspective feature (horizontal view feature) to enrich the vertical view feature information while ensuring that the useful feature information in the vertical view feature is not disturbed.

[0100] Then, the feature mapping module can filter and fuse the features extracted by sampling through the convolution attention mechanism module to obtain fused features.

[0101] In specific implementation, since the dual-view X-ray security image has certain particularity, the picture itself has perspective characteristics, and the texture features are less, and the color features and boundary features are particularly obvious; and the picture features of the two views are extracted at different angles for the same target, so they are neither completely complementary information nor completely identical information. Therefore, the convolution attention mechanism module needs to be used to filter the features. The horizontal view features after attention sorting can have a reasonable range of influence on the vertical view, and can constrain the fused features to a certain extent, preventing the spatial feature position information between the two features from being excessively fused and losing the primary and secondary relationship between the two views.

[0102] The working principle of the convolution attention mechanism module will be further described below. The convolution attention mechanism module can use the channel relationship between the features to construct a channel attention feature mapping, and regard each channel of the feature map as a feature detector, and the attention of the channel focuses on the category of the input image; meanwhile, a spatial attention feature mapping is generated by using the spatial relationship between the features, and the positioning information of the features is focused on, which effectively supplements the channel attention mechanism. The input feature map is compressed in spatial dimension, and the channel attention is effectively calculated. In the calculation process of the convolution attention mechanism module, the average pooling can be used to aggregate the spatial information, so as to effectively learn the range of the target object, and the maximum pooling can be used to obtain the specific object features to obtain better channel attention. The above calculation process can improve the performance of the convolution attention mechanism module through the sharing of the average pooling and the maximum pooling features.

[0103] For example: first, define an input feature mapping as F∈RC×H×W, wherein H and W are the height and width of the feature map respectively, C is the number of channels of the feature map, and R is a set of real numbers. The average pooling and the maximum pooling operations are used to aggregate the spatial information of the feature mapping to generate two different spatial context descriptors, the average pooling feature and the maximum pooling feature. Then, a 1D channel attention mapping M c ∈RC×1×1 is generated through a shared network composed of a multilayer perceptron (MLP, Multilayer Perceptron) and a hidden layer. In order to reduce the parameter amount, the hidden activation size is set to RC / r×1×1, and r is the reduction ratio. After applying the shared network to each descriptor, an element summation is used to combine the output feature vector.

[0104] The calculation of the channel attention A c is as follows:

[0105] The calculation of the channel attention A c(F) = σ(f 7×7 (Avgpool(F)) + f 7×7 (Maxpool(F))

[0106] Wherein: sigma is a sigmoid function, f7x7 is a 7x7 convolution filter.

[0107] In order to calculate the spatial attention, the average pooling and the maximum pooling operation are adopted along the channel axis, and they are connected to generate an effective feature descriptor. The pooling operation is applied along the channel axis, which can effectively highlight the feature information area, and two 2D spatial attention maps M s ∈R1xHxW.

[0108] The spatial attention A p is calculated in the following way:

[0109] A p (F) = sigma(f 7×7 (Avgpool(F)) + f 7×7 (Maxpool(F))

[0110] Wherein: sigma is a sigmoid function, f7x7 is a 7x7 convolution filter.

[0111] The two attention modules in the convolution attention mechanism module are connected in a sequential manner. The input feature map is F, the output after the channel attention mechanism module is F', and then the feature expression output after the spatial attention mechanism module is F''. The output expression of the convolution attention mechanism module is as follows:

[0112]

[0113] In the above embodiment, the feature map module can be sequentially connected through the up-sampling module and the convolution attention mechanism module, so that the features of the horizontal view are fused into the features of the vertical view as auxiliary features, the dual-view feature fusion is realized, and the detection accuracy is improved and the missed detection rate is reduced.

[0114] The embodiment of the application provides a kind of contraband detection device based on dual-view security image, please see Figure 6 , the device comprises:

[0115] Security image acquisition module 101 is used to acquire the first view security image and the second view security image of luggage.

[0116] The prohibited items detection module 102 is used to process the first-view security inspection image and the second-view security inspection image through a pre-trained prohibited items detection model to obtain the prohibited items detection results of the baggage; wherein, the prohibited items detection model is used to detect whether there are prohibited items in the baggage based on the first-view security inspection image and the second-view security inspection image, and when there are prohibited items in the baggage, classify and locate the prohibited items in the baggage.

[0117] The detection result output module 103 is used to output the detection results of prohibited items in luggage.

[0118] In some embodiments, the contraband detection model uses the one-stage target detection algorithm YOLOv4 as the basic target detection framework. The basic target detection framework includes a feature extraction backbone network, an SPP spatial pyramid pooling layer, a PANet network, and a detection layer. The feature extraction backbone network includes a first primary feature extraction network, a second primary feature extraction network, a feature mapping module, and a secondary feature extraction network.

[0119] The prohibited items detection module 102 specifically includes the following units:

[0120] The primary feature extraction unit is used to extract the first-view features of the first-view security inspection image through the first primary feature extraction network, and to extract the second-view features of the second-view security inspection image through the second primary feature extraction network.

[0121] The feature fusion unit is used to fuse first-view features and second-view features through the feature mapping module, and input the fused features into the secondary feature extraction network.

[0122] The secondary feature extraction unit is used to extract the fused features through the secondary feature extraction network, resulting in the first feature layer, the second feature layer, and the third feature layer arranged in descending order of scale.

[0123] The feature pooling processing unit is used to process the third feature layer through the SPP spatial pyramid pooling layer, and input the first feature layer, the second feature layer and the third feature layer after processing by the SPP spatial pyramid pooling layer into the PANet network for processing to obtain the detection features.

[0124] The prohibited items detection unit is used to input detection features into the detection layer to obtain the prohibited items detection results of the luggage.

[0125] In some embodiments, both the first primary feature extraction network and the second primary feature extraction network include two residual modules, and the parameters of the two residual modules in the first primary feature extraction network are not interchangeable with the parameters of the two residual modules in the second primary feature extraction network.

[0126] In some embodiments, the feature extraction backbone network comprises two feature mapping modules;

[0127] The primary feature extraction unit is specifically configured to: preliminarily extract first-view features of the first-view security image through a first residual module of the first primary feature extraction network, and input the first-view features into a first feature mapping module in the feature extraction backbone network; preliminarily extract second-view features of the second-view security image through a first residual module of the second primary feature extraction network, and input the second-view features into the first feature mapping module in the feature extraction backbone network and a second residual module of the second primary feature extraction network respectively; perform secondary feature extraction on the second-view features through the second residual module of the second primary feature extraction network, and input the second-view features extracted through the secondary feature extraction into a second feature mapping module in the feature extraction backbone network.

[0128] The feature fusion unit is specifically configured to: fuse the input first-view features and second-view features through the first feature mapping module in the feature extraction backbone network, input the obtained primary fusion features into the second residual module of the first primary feature extraction network for secondary feature extraction, and input the secondary fusion features extracted through the secondary feature extraction into the second feature mapping module in the feature extraction backbone network; fuse the secondary fusion features and the second-view extracted features through the second feature mapping module in the feature extraction backbone network to obtain the fusion features.

[0129] In some embodiments, each feature mapping module comprises an up-down sampling module and a convolution attention mechanism module; and the feature fusion unit is further specifically configured to: up-sample and down-sample the first-view features and the second-view features through the up-down sampling module of the first feature mapping module, and input the features obtained through the up-sampling and down-sampling into the convolution attention mechanism module of the first feature mapping module to obtain the primary fusion features output by the convolution attention mechanism module; and up-sample and down-sample the secondary fusion features and the second-view extracted features through the up-down sampling module of the second feature mapping module, and input the features obtained through the up-sampling and down-sampling into the convolution attention mechanism module of the second feature mapping module to obtain the fusion features output by the convolution attention mechanism module.

[0130] In some embodiments, the secondary feature extraction network comprises three residual modules connected in sequence; and the feature fusion unit is further specifically configured to: input the obtained fusion features into a first residual module of the secondary feature extraction network.

[0131] The secondary feature extraction unit is specifically configured to: extract the fusion feature through a first residual module of the secondary feature extraction network to obtain a first feature layer; input the first feature layer into a second residual module of the secondary feature extraction network for feature extraction to obtain a second feature layer; and input the second feature layer into a third residual module of the secondary feature extraction network for feature extraction to obtain a third feature layer.

[0132] In some embodiments, the first-view security image is an X-ray image of the luggage collected by the dual-view X-ray security inspection machine at a vertical view angle, and the second-view security image is an X-ray image of the luggage collected by the dual-view X-ray security inspection machine at a horizontal view angle.

[0133] The specific limitations of the contraband detection device based on dual-view security images provided in this embodiment can be referred to the embodiments of the contraband detection method based on dual-view security images described above, and will not be repeated here. Each module in the contraband detection device based on dual-view security images described above can be realized by software, hardware, and combinations thereof, in whole or in part. Each module described above can be embedded in or independent of the processor in the computer device in hardware form, or can be stored in the memory in the computer device in software form, so as to be called and executed by the processor to perform the operations corresponding to each module.

[0134] The electronic device provided in the embodiments of the present application can include a processor, a memory, a network interface and a database connected through a system bus. The processor of the electronic device is configured to provide computing and control capabilities. The memory of the electronic device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operating system and the computer program in the non-volatile storage medium. The network interface of the electronic device is configured to communicate with an external terminal through a network connection. The computer program is executed by the processor, so that the processor performs the steps of the contraband detection method based on dual-view security images according to any of the above embodiments.

[0135] The working process, working details and technical effects of the electronic device provided in the embodiments can be referred to the embodiments of the contraband detection method based on dual-view security images described above, and will not be repeated here.

[0136] The embodiment of the present application provides a computer readable storage medium, which stores a computer program, and the computer program is executed by a processor to realize the steps of the contraband detection method based on a dual-view security image of any one of the above embodiments. The computer readable storage medium refers to a carrier for storing data, and can include, but is not limited to, a floppy disk, an optical disk, a hard disk, a flash memory, a USB flash disk and / or a memory stick, and the computer can be a general-purpose computer, a special-purpose computer, a computer network or other programmable devices.

[0137] The working process, working details and technical effects of the computer readable storage medium provided by the embodiment can be referred to the above embodiments of the contraband detection method based on a dual-view security image, and will not be described here.

[0138] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by a computer program instructing related hardware, and the computer program can be stored in a non-volatile computer readable storage medium. When the computer program is executed, the computer program can include the processes of the above-mentioned embodiments. Any reference to memory, storage, database or other medium used in the embodiments provided by the present application can include non-volatile and / or volatile memory. The non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. The volatile memory can include random access memory (RAM) or external cache memory. As an illustration but not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDR SDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM) and memory bus dynamic RAM (RDRAM).

[0139] The technical features of the above embodiments can be combined arbitrarily. In order to make the description simple, all possible combinations of the technical features in the above embodiments are not described, however, as long as the combination of the technical features does not exist contradictory, it should be considered as the scope of the present application.

[0140] The above-described embodiments are merely illustrative of several embodiments of the present application, which are described in more detail and in a specific manner, but should not be construed as limiting the scope of the patent. It should be noted that for those skilled in the art, several modifications and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the patent of the present application should be subject to the appended claims.

Claims

1. A method for detecting contraband based on dual-view security inspection images, characterized in that, The method includes: Obtain first-person and second-person security inspection images of luggage; The first-view security inspection image and the second-view security inspection image are processed by a pre-trained prohibited items detection model to obtain the prohibited items detection result of the luggage; wherein, the prohibited items detection model is used to detect whether there are prohibited items in the luggage based on the first-view security inspection image and the second-view security inspection image, and when there are prohibited items in the luggage, to classify and locate the prohibited items in the luggage; The contraband detection model is based on the YOLOv4 one-stage target detection algorithm as its basic target detection framework. The basic target detection framework includes a feature extraction backbone network, an SPP spatial pyramid pooling layer, a PANet network, and a detection layer. The feature extraction backbone network includes a first primary feature extraction network, a second primary feature extraction network, a feature mapping module, and a secondary feature extraction network. The process of processing the first-view security inspection image and the second-view security inspection image using a pre-trained contraband detection model to obtain the contraband detection result of the luggage includes: The first-view feature of the security inspection image from the first viewpoint is extracted by the first primary feature extraction network, and the second-view feature of the security inspection image from the second viewpoint is extracted by the second primary feature extraction network. The first viewpoint features and the second viewpoint features are fused through the feature mapping module, and the resulting fused features are input into the secondary feature extraction network. The fused features are extracted through the secondary feature extraction network to obtain a first feature layer, a second feature layer, and a third feature layer arranged in descending order of scale. The third feature layer is processed by the SPP spatial pyramid pooling layer, and the first feature layer, the second feature layer, and the third feature layer processed by the SPP spatial pyramid pooling layer are input into the PANet network for processing to obtain the detection features. The detection features are input into the detection layer to obtain the contraband detection result of the luggage; Output the results of the prohibited items detection in the luggage.

2. The method according to claim 1, characterized in that, Both the first primary feature extraction network and the second primary feature extraction network include two residual modules, and the parameters of the two residual modules in the first primary feature extraction network are not interchangeable with the parameters of the two residual modules in the second primary feature extraction network.

3. The method according to claim 2, characterized in that, The feature extraction backbone network includes two feature mapping modules; The step of extracting first-view features from the first-view security inspection image using the first primary feature extraction network and extracting second-view features from the second-view security inspection image using the second primary feature extraction network includes: The first viewpoint features of the security inspection image from the first viewpoint are initially extracted by the first residual module of the first primary feature extraction network, and the first viewpoint features are input into the first feature mapping module in the feature extraction backbone network. The second-view features of the security inspection image are initially extracted through the first residual module of the second primary feature extraction network, and the second-view features are respectively input into the first feature mapping module and the second residual module of the second primary feature extraction network in the feature extraction backbone network. The second viewpoint features are extracted again through the second residual module of the second primary feature extraction network, and the extracted second viewpoint features are input into the second feature mapping module in the feature extraction backbone network. The step of fusing the first viewpoint features and the second viewpoint features through the feature mapping module includes: The first feature mapping module in the feature extraction backbone network fuses the input first viewpoint features and the second viewpoint features. The resulting primary fused features are then input into the second residual module of the first primary feature extraction network for further feature extraction. The secondary fused features obtained from the second feature extraction are then input into the second feature mapping module of the feature extraction backbone network. The secondary fusion features and the second perspective extracted features are fused by the second feature mapping module in the feature extraction backbone network to obtain the fused features.

4. The method according to claim 3, characterized in that, Each of the feature mapping modules includes an upsampling module and a convolutional attention mechanism module; The step of fusing the input first-viewpoint features and second-viewpoint features through the first feature mapping module in the feature extraction backbone network includes: The first viewpoint features and the second viewpoint features are upsampled and downsampled by the upsampling and downsampling modules of the first feature mapping module, and the features obtained by upsampling and downsampling are input into the convolutional attention mechanism module of the first feature mapping module to obtain the primary fusion features output by the convolutional attention mechanism module. The process of fusing the secondary fusion features and the second-view extracted features through the second feature mapping module in the feature extraction backbone network to obtain the fused features includes: The secondary fusion features and the second viewpoint extracted features are upsampled and downsampled by the upsampling and downsampling modules of the second feature mapping module, and the features obtained by upsampling and downsampling are input into the convolutional attention mechanism module of the second feature mapping module to obtain the fusion features output by the convolutional attention mechanism module.

5. The method according to claim 3, characterized in that, The secondary feature extraction network includes three residual modules connected in sequence; the step of inputting the obtained fused features into the secondary feature extraction network includes: inputting the obtained fused features into the first residual module of the secondary feature extraction network; The process of extracting the fused features through the secondary feature extraction network to obtain a first feature layer, a second feature layer, and a third feature layer arranged in descending order of scale includes: The fused features are extracted through the first residual module of the secondary feature extraction network to obtain the first feature layer; The first feature layer is input into the second residual module of the secondary feature extraction network for feature extraction to obtain the second feature layer; The second feature layer is input into the third residual module of the secondary feature extraction network for feature extraction to obtain the third feature layer.

6. The method according to any one of claims 1 to 5, characterized in that, The first-view security inspection image is an X-ray image of the luggage captured by the dual-view X-ray security inspection machine in a vertical view, and the second-view security inspection image is an X-ray image of the luggage captured by the dual-view X-ray security inspection machine in a horizontal view.

7. A contraband detection device based on dual-view security inspection images, characterized in that, The device includes: The security inspection image acquisition module is used to acquire first-view and second-view security inspection images of luggage. The contraband detection module is used to process the first-view security inspection image and the second-view security inspection image using a pre-trained contraband detection model to obtain the contraband detection result of the luggage. The contraband detection model is used to detect whether there are contraband items in the luggage based on the first-view and second-view security inspection images, and when contraband items are found, it classifies and locates the contraband items. The contraband detection model uses the YOLOv4 one-stage target detection algorithm as its basic target detection framework. The basic target detection framework includes a feature extraction backbone network, an SPP spatial pyramid pooling layer, a PANet network, and a detection layer. The feature extraction backbone network includes a first primary feature extraction network, a second primary feature extraction network, a feature mapping module, and a secondary feature extraction network. The process of processing the first-view security inspection image and the second-view security inspection image using a pre-trained contraband detection model to obtain the contraband detection result of the luggage includes: The first-view feature of the security inspection image from the first viewpoint is extracted by the first primary feature extraction network, and the second-view feature of the security inspection image from the second viewpoint is extracted by the second primary feature extraction network. The first viewpoint features and the second viewpoint features are fused through the feature mapping module, and the resulting fused features are input into the secondary feature extraction network. The fused features are extracted through the secondary feature extraction network to obtain a first feature layer, a second feature layer, and a third feature layer arranged in descending order of scale. The third feature layer is processed by the SPP spatial pyramid pooling layer, and the first feature layer, the second feature layer, and the third feature layer processed by the SPP spatial pyramid pooling layer are input into the PANet network for processing to obtain the detection features. The detection features are input into the detection layer to obtain the contraband detection result of the luggage; the detection result output module is used to output the contraband detection result of the luggage.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the method as described in any one of claims 1 to 6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method as described in any one of claims 1 to 6.

Images