A contraband detection method, apparatus, device, medium and program product

Abstract

This invention discloses a method, apparatus, device, medium, and program product for detecting contraband. The method includes: extracting features from X-ray security images based on a multi-branch, cross-stage local network to obtain multi-scale target feature maps. The multi-branch, cross-stage local network includes a shallow feature extraction module, multiple deep feature extraction modules, and a semantic feature extraction module. Each deep feature extraction module includes a first module and a second module. The first module uses a multi-branch structure to perform downsampling and extract features from receptive fields at different scales, and fuses the feature maps under each scale receptive field to obtain a multi-scale fused feature map. The second module uses a dual-path structure to perform cross-stage deep feature extraction and fusion processing on the multi-scale fused feature map. The multi-scale target feature map is input into a target detection network for detection to obtain the contraband detection results in the X-ray security image, thereby improving the efficiency and accuracy of contraband detection.

Description

Technical Field

[0001] The embodiments of the present invention relate to image processing technology, and more particularly to a method, apparatus, device, medium, and program product for detecting contraband. Background Technology

[0002] In transportation, logistics, and other fields, X-ray security scanners are commonly used to assist manual security checks. An X-ray security scanner is a non-invasive security inspection device that can obtain information about the contents of luggage without physical contact. When X-rays scan a bag, different materials absorb the X-rays to varying degrees. The receiver calculates a two-dimensional perspective image of the bag's interior based on the attenuation of the X-rays.

[0003] Currently, security checks are typically conducted manually by observing images generated by X-ray security scanners to determine the presence of contraband. However, this method, relying on manual visual inspection, is extremely labor-intensive, and visual fatigue from prolonged work reduces efficiency, leading to missed detections. Therefore, how to accurately and efficiently detect contraband automatically from X-ray security images has become a pressing problem to be solved. Summary of the Invention

[0004] This invention provides a method, apparatus, equipment, medium, and program product for detecting prohibited items, enabling automatic detection of prohibited items without human intervention, thereby improving the efficiency and accuracy of prohibited item detection.

[0005] In a first aspect, embodiments of the present invention provide a method for detecting prohibited items, including: Obtain X-ray security inspection images; Based on a multi-branch, cross-stage local network, feature extraction is performed on the X-ray security inspection image to obtain a multi-scale target feature map. The multi-branch, cross-stage local network includes a shallow feature extraction module, multiple deep feature extraction modules, and a semantic feature extraction module connected sequentially. Each deep feature extraction module includes a first module and a second module connected in series. The first module uses a multi-branch structure to downsample the input feature map and extract features from receptive fields at different scales, and then fuses the extracted feature maps at each scale receptive field to obtain a multi-scale fused feature map. The second module uses a dual-path structure to perform cross-stage deep feature extraction and fusion processing on the multi-scale fused feature map to obtain a target fused feature map. The multi-scale target feature map includes: at least one target fused feature map extracted by the deep feature extraction module and a semantic feature map extracted by the semantic feature extraction module. The multi-scale target feature map is input into the target detection network for cross-scale fusion and contraband detection to obtain the contraband detection results in the X-ray security inspection image.

[0006] Secondly, embodiments of the present invention also provide a contraband detection device, comprising: The image acquisition module is used to acquire X-ray security inspection images; A feature extraction module is used to extract features from the X-ray security inspection image based on a multi-branch cross-stage local network to obtain a multi-scale target feature map. The multi-branch cross-stage local network includes a shallow feature extraction module, multiple deep feature extraction modules, and a semantic feature extraction module connected in sequence. Each deep feature extraction module includes a first module and a second module connected in series. The first module uses a multi-branch structure to downsample the input feature map and extract features from receptive fields at different scales, and then fuses the extracted feature maps at each scale receptive field to obtain a multi-scale fused feature map. The second module uses a dual-path structure to perform cross-stage deep feature extraction and fusion processing on the multi-scale fused feature map to obtain a target fused feature map. The multi-scale target feature map includes at least one target fused feature map extracted by the deep feature extraction module and a semantic feature map extracted by the semantic feature extraction module. The contraband detection module is used to input the multi-scale target feature map into the target detection network for cross-scale fusion and contraband detection, so as to obtain the contraband detection results in the X-ray security inspection image.

[0007] Thirdly, embodiments of the present invention also provide an electronic device, the electronic device comprising: At least one processor; and A memory communicatively connected to the at least one processor; wherein, The memory stores a computer program that can be executed by the at least one processor, which enables the at least one processor to perform the contraband detection method provided in any embodiment of the present invention.

[0008] Fourthly, embodiments of the present invention also provide a computer-readable storage medium storing computer instructions, which are used to cause a processor to execute and implement the contraband detection method provided in any embodiment of the present invention.

[0009] Fifthly, embodiments of the present invention also provide a computer program product, including a computer program that, when executed by a processor, implements the contraband detection method provided in any embodiment of the present invention.

[0010] One embodiment of the above invention has the following advantages or beneficial effects: Feature extraction from X-ray security inspection images is performed using a multi-branch, cross-stage local network to obtain multi-scale target feature maps. The multi-branch, cross-stage local network includes a shallow feature extraction module, multiple deep feature extraction modules, and a semantic feature extraction module connected sequentially. Each deep feature extraction module includes a first module and a second module connected in series. The first module uses a multi-branch structure to downsample the input feature map and extract features from receptive fields at different scales. It then fuses the extracted feature maps at each scale receptive field to obtain a multi-scale fused feature map. This first module achieves multi-scale receptive field feature extraction while downsampling, improving feature extraction capability and parameter utilization efficiency. The second module uses a dual-path structure to perform cross-stage deep feature extraction and fusion processing on the multi-scale fused feature map to obtain a target fused feature map. The multi-scale target feature map includes the target fused feature map extracted by at least one deep feature extraction module and the semantic feature map extracted by the semantic feature extraction module. Through the collaborative processing of the first module and the second model, a multi-branch, cross-stage local network with stronger feature extraction capability and higher parameter utilization efficiency is constructed, enabling the extraction of multi-scale target feature maps with richer feature information. By inputting multi-scale target feature maps into a target detection network for cross-scale fusion and contraband detection, the detection results of contraband in X-ray security inspection images can be obtained accurately and efficiently. The entire process does not require manual intervention, realizing automatic detection of contraband and improving the efficiency and accuracy of contraband detection.

[0011] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of the present invention, nor is it intended to limit the scope of the invention. Other features of the invention will become readily apparent from the following description. Attached Figure Description

[0012] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0013] Figure 1 This is a flowchart of a method for detecting contraband provided in one embodiment of the present invention; Figure 2 This is a schematic diagram of the architecture of a multi-branch cross-stage local network according to an embodiment of the present invention; Figure 3 This is a schematic diagram of the network architecture of a semantic feature extraction module according to an embodiment of the present invention; Figure 4This is an example diagram illustrating the detection result of contraband in an X-ray security inspection image according to an embodiment of the present invention; Figure 5 This is a flowchart of a method for determining multi-scale fusion feature maps based on a first module, provided in one embodiment of the present invention; Figure 6 This is a schematic diagram of the network architecture of a first module according to an embodiment of the present invention; Figure 7 This is a flowchart of a method for determining target fusion feature maps based on a second module, provided in one embodiment of the present invention; Figure 8 This is a schematic diagram of a network architecture for a second module according to an embodiment of the present invention; Figure 9 This is a schematic diagram of the structure of a contraband detection device provided in one embodiment of the present invention; Figure 10 This is a schematic diagram of the structure of an electronic device that implements the prohibited items detection method of this invention. Detailed Implementation

[0014] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0015] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0016] Figure 1This is a flowchart illustrating a method for detecting contraband according to an embodiment of the present invention. This embodiment is applicable to situations in the security inspection field where contraband is detected by acquiring X-ray security images. The method can be executed by a contraband detection device, which can be implemented in software and / or hardware, and can be configured in an electronic device. Figure 1 As shown, the method specifically includes the following steps: S110. Obtain X-ray security inspection images.

[0017] X-ray security images refer to two-dimensional perspective images obtained by using X-ray security scanners to capture images of luggage and packages for security checks, thereby determining whether prohibited items are present. Prohibited items can refer to dangerous or controlled items that are prohibited from being carried or shipped, such as knives and guns.

[0018] For example, before feature extraction from X-ray security inspection images, preprocessing such as resizing and normalization can be performed to ensure the detection effectiveness of contraband. For instance, resizing can refer to scaling the X-ray security inspection image to a fixed input size for a multi-branch, multi-stage local area network, such as 640×640. Normalization can refer to mapping the X-ray security inspection image from its original pixels to a fixed numerical range to eliminate pixel value fluctuations caused by differences in X-ray image brightness and equipment brightness deviations.

[0019] It's important to note that natural light images offer clear details and make objects easily identifiable, facilitating the extraction of feature information. In contrast, X-ray security images exhibit poor edge texture detail, with significant loss of item detail. Not only is it impossible to obtain surface texture information of contraband, but even a complete outline of the item cannot be captured. Furthermore, the perspective properties of X-rays cause spatially non-overlapping items to overlap completely in the image, further complicating the background and making the environment cluttered. Compared to natural light images, X-ray security images not only lack sufficient foreground information but also contain more background noise, making target detection more difficult. Therefore, stronger feature extraction capabilities are needed to extract subtle features of contraband from complex backgrounds. Without extracting sufficient feature information from X-ray security images, high-precision contraband detection cannot be achieved.

[0020] S120. Based on a multi-branch cross-stage local network, feature extraction is performed on X-ray security inspection images to obtain multi-scale target feature maps. The multi-branch cross-stage local network includes a shallow feature extraction module, multiple deep feature extraction modules, and a semantic feature extraction module connected in sequence. Each deep feature extraction module includes a first module and a second module connected in series. The first module uses a multi-branch structure to downsample the input feature map and extract receptive field features at different scales, and performs fusion processing on the extracted feature maps at each scale receptive field to obtain a multi-scale fused feature map. The second module uses a dual-path structure to perform cross-stage deep feature extraction and fusion processing on the multi-scale fused feature map to obtain a target fused feature map. The multi-scale target feature map includes: a target fused feature map extracted by at least one deep feature extraction module and a semantic feature map extracted by the semantic feature extraction module.

[0021] Multi-branch cross-stage local neural networks (LCNs) are used to extract more informative feature maps of different sizes from X-ray security images. Because X-ray security images have very cluttered backgrounds and are affected by the transparency of X-rays, the features of contraband in these images are relatively weak. Therefore, in the cluttered background of X-ray security images, it is necessary to detect contraband with weak features from multiple scaled perspectives. By utilizing multi-branch cross-stage LCNs, multi-scale receptive field features can be combined during feature extraction to extract sufficiently rich features, thereby improving the accuracy of subsequent contraband detection.

[0022] Multi-branch, cross-stage local networks are specifically designed to address the unique perspective characteristics of X-ray security images, which lead to information overlap and cluttered backgrounds. (See also...) Figure 2 The multi-branch, cross-stage local network comprises a shallow feature extraction module, multiple deep feature extraction modules, and a semantic feature extraction module connected in sequence. The shallow feature extraction module is used to extract low-level feature information from X-ray security images and reduce image size. For example, the shallow feature extraction module can be a convolutional unit with a 6×6 kernel and a stride of 2. The convolutional unit is the basic feature extraction unit, used for local feature extraction and nonlinear mapping. For example, a convolutional unit can be a CBS (Convolution+BN+SiLU) unit, consisting of a convolutional layer, a batch normalization layer, and an activation function layer connected in sequence.

[0023] Multiple deep feature extraction modules extract features at different depths from the extracted shallow feature maps to obtain deep feature maps at different scales. These multiple deep feature extraction modules are connected sequentially, such as... Figure 2As shown, the multi-branch cross-stage local network includes four deep feature extraction modules. Each deep feature extraction module includes a first module and a second module connected in series. The first and second modules are alternately connected. The first module has a multi-branch structure, which allows for the extraction of receptive field features at different scales from the input feature map. This enables the first module to extract multi-scale contextual information while downsampling, achieving a balance between downsampling and feature extraction. This avoids the low parameter utilization efficiency caused by downsampling alone, thus improving the parameter utilization efficiency in the first module by simultaneously performing downsampling and extracting receptive field features at different scales. The second module uses a dual-path structure to perform cross-stage deep feature extraction and fusion processing on the multi-scale fused feature map output by the first module, obtaining the target fused feature map. The two modules work together to efficiently extract richer feature information.

[0024] It should be understood that different deep feature extraction modules correspond to different extraction depths. As the extraction depth increases, the receptive field corresponding to each deep feature extraction module will continue to expand. Furthermore, different scale receptive field features can be extracted based on the first module within the same deep feature extraction module. Thus, multi-scale receptive field features not only exist between different levels of deep feature extraction modules but also within the same level. In other words, different scale receptive field features can be extracted and interacted within the same level, thereby enabling the extraction of more and richer feature information about contraband.

[0025] The semantic feature extraction module is used to extract high-level semantic features. This module can use multiple cascaded max-pooling layers to obtain features with receptive fields of different sizes and then fuse them to obtain high-level semantic information. For example, the semantic feature extraction module includes a fifth sub-module to reduce the number of channels, multiple max-pooling layers, a concatenation unit, and a sixth sub-module to further reduce the number of channels. The fifth sub-module can be a convolutional unit with a 1×1 kernel and half the number of channels. The max-pooling layer can be a 5×5 kernel with a stride of 1 to expand the local receptive field without changing the spatial resolution. The sixth sub-module can be a convolutional unit with a 1×1 kernel and half the number of channels to keep the number of channels consistent between the input and output information. The convolutional unit can be a CBS (Convolution+BN+SiLU) unit, consisting of a convolutional layer, a batch normalization layer, and an activation function layer connected in series. For example, Figure 3 A schematic diagram of the network architecture for a semantic feature extraction module is given, such as... Figure 3 As shown, the semantic feature extraction module uses three max pooling layers to extract and fuse features with receptive fields of different sizes.

[0026] Specifically, the X-ray security inspection image is input into a multi-branch cross-stage local network for feature extraction, resulting in a more informative multi-scale target feature map. For example, the target fusion feature map extracted by each deep feature extraction module in the multi-branch cross-stage local network and the semantic feature map extracted by the semantic feature extraction module can be used as the final extracted multi-scale target feature map.

[0027] For example, a multi-branch, cross-stage local network can extract features from X-ray security inspection images to obtain multi-scale target feature maps by performing the following steps, i.e., step S120 includes: The X-ray security inspection image is input into the shallow feature extraction module for shallow feature extraction, resulting in a shallow feature map. The shallow feature map is then input into the first deep feature extraction module for deep feature extraction, and the target fusion feature map extracted by the first deep feature extraction module is input into the next deep feature extraction module for further deep feature extraction. The target fusion feature map extracted by the last deep feature extraction module is then input into the semantic feature extraction module for semantic feature extraction, resulting in a semantic feature map. The target fusion feature map extracted by at least one deep feature extraction module and the semantic feature map extracted by the semantic feature extraction module are combined to form a multi-scale target feature map.

[0028] See Figure 2When there are four deep feature extraction modules, the X-ray security inspection image is input into the shallow feature extraction module for shallow feature extraction, obtaining the extracted shallow feature map. The shallow feature map is then input into the first deep feature extraction module (i.e., deep feature extraction module 1) for deep feature extraction. The target fusion feature map extracted by the first deep feature extraction module is input into the second deep feature extraction module (i.e., deep feature extraction module 2) for deep feature extraction, and so on, until the target fusion feature map extracted by the fourth deep feature extraction module (i.e., deep feature extraction module 4) is obtained. The target fusion feature map extracted by the fourth deep feature extraction module is input into the semantic feature extraction module for semantic feature extraction, obtaining the extracted semantic feature map. The target fusion feature map extracted by at least one deep feature extraction module and the semantic feature map extracted by the semantic feature extraction module are determined as a multi-scale target feature map, thereby extracting a multi-scale target feature map with richer feature information. For example, if the input X-ray security inspection image is a 640×640 pixel, 3-channel image B0, the shallow feature extraction module extracts a 320×320 pixel, 32-channel shallow feature map, which increases the number of channels while halving the length and width. The first deep feature extraction module extracts a 160×160 pixel, 64-channel target fusion feature map, which doubles the number of channels while halving the length and width. The second deep feature extraction module extracts an 80×80 pixel, 128-channel target fusion feature map, which also doubles the number of channels while halving the length and width. The third deep feature extraction module extracts a 40×40 pixel, 256-channel target fusion feature map, which doubles the number of channels while halving the length and width. The fourth deep feature extraction module extracts a 20×20 pixel, 512-channel target fusion feature map, which also doubles the number of channels while halving the length and width. The semantic feature map extracted by the semantic feature extraction module is a 20×20 pixel, 512-channel image B5', with the number of channels, length and width remaining unchanged.

[0029] For example, when there are four deep feature extraction modules, the multi-scale target feature map includes: the target fusion feature map extracted by the second deep feature extraction module (i.e., deep feature extraction module 2), the target fusion feature map extracted by the third deep feature extraction module (i.e., deep feature extraction module 3), and the semantic feature map extracted by the semantic feature extraction module. For example, B3, B4, and B5' are used as multi-scale target feature maps. By setting four deep feature extraction modules and retaining only the target fusion feature map extracted by the more critical deep feature extraction modules, the computational load can be reduced while ensuring detection accuracy, thereby improving detection efficiency and saving computational resources.

[0030] It should be noted that each deep feature extraction module includes a first module and a second module. The deep feature extraction process is the same in each deep feature extraction module, only the input information differs. For example, the input information of the first deep feature extraction module is the shallow feature map, and the input information of subsequent deep feature extraction modules is the target fusion feature map extracted by the previous deep feature extraction module. This embodiment uses the feature extraction process of the first deep feature extraction module as an example for illustration; the feature extraction processes of other deep feature extraction modules will not be described in detail.

[0031] For example, the first deep feature extraction module can perform feature extraction by performing the following steps to obtain the extracted target fusion feature map, namely, the step "inputting the shallow feature map into the first deep feature extraction module for deep feature extraction", including: inputting the shallow feature map into the first module of the first deep feature extraction module for downsampling and extraction of receptive field features at different scales, and performing fusion processing on the feature maps under each scale receptive field to obtain a multi-scale fusion feature map; inputting the multi-scale fusion feature map into the second module of the first deep feature extraction module for cross-stage deep feature extraction and fusion processing to obtain the target fusion feature map extracted by the first deep feature extraction module.

[0032] In each deep feature extraction module, the first module employs a multi-branch structure to extract multi-scale receptive field features in parallel. Subsequent fusion and interaction facilitate the detection of subtle clues about contraband, making it more suitable for security inspection scenarios using X-ray images. This multi-branch, multi-level structure not only completes the downsampling task but also provides greater non-linearity, resulting in stronger feature extraction capabilities and improved parameter utilization efficiency. The second module uses a dual-path structure for cross-stage feature extraction, thus collaborating with the first module to extract more contraband feature information, effectively improving the accuracy of contraband detection.

[0033] For example, the semantic feature extraction module can perform semantic feature extraction by executing the following steps to obtain the extracted semantic feature map, namely, the step "inputting the target fusion feature map extracted by the last deep feature extraction module into the semantic feature extraction module for semantic feature extraction to obtain the extracted semantic feature map", includes: inputting the last deep feature extraction module into the fifth sub-module to reduce the number of channels, inputting the feature map with reduced channel number into the first max pooling layer for max pooling processing, inputting the feature map output by the first max pooling layer into the next max pooling layer for max pooling processing sequentially, inputting the feature map output by each max pooling layer into the concatenation unit for concatenation processing in the channel dimension, and inputting the concatenated feature map into the sixth sub-module for channel number reduction processing to obtain a semantic feature map with the same number of channels as the input target fusion feature map. Figure 3 As shown, Figure 3 The following describes the semantic feature extraction process of the semantic feature extraction module when it contains three max pooling layers.

[0034] S130. Input the multi-scale target feature map into the target detection network for cross-scale fusion and contraband detection to obtain the contraband detection results in the X-ray security inspection image.

[0035] The object detection network can be a network model that performs feature fusion and contraband detection on multi-scale object feature maps. The object detection network can include a cross-scale fusion module and a contraband detection module. The cross-scale fusion module can act as the neck network, used to fuse, reconstruct, and refine the multi-scale object feature maps. The contraband detection module can act as the head network, used to map the refined multi-scale object feature maps to the final contraband detection result.

[0036] The contraband detection result can include 5+N values. Among them, 4 values ​​are used to represent the position and size of the contraband prediction box, and 1 value is used to represent the probability that there is a contraband in the box, i.e., the confidence level. When there are N types of contraband, the N values ​​refer to the probability that the prediction result belongs to each type of contraband. The prediction result with the highest probability can be used as the final output contraband detection result.

[0037] For example, Figure 4 An example image of the detection results of prohibited items in X-ray security inspection images is given. Figure 4 As shown, due to the perspective properties, X-ray security images exhibit information overlap and cluttered backgrounds. Target detection networks, based on rich multi-scale target feature maps extracted from X-ray security images, can perform high-precision detection of contraband, avoiding missed detections. For example... Figure 4The system detects the presence of lighters in packages and luggage, including their location, size, and confidence level. This automatically identifies the presence of lighters as a prohibited item without human intervention, improving the efficiency and accuracy of contraband detection.

[0038] For example, before practical application, the multi-branch cross-stage local network and object detection network need to be trained based on sample data to maintain their processing performance. The sample data includes X-ray security inspection sample images and contraband label information within those images. By iteratively optimizing the multi-branch cross-stage local network and object detection network based on the sample data and loss function, a finally converged multi-branch cross-stage local network and object detection network are obtained, thereby ensuring the feature extraction performance of the multi-branch cross-stage local network and the detection performance of the object detection network.

[0039] The technical solution of this embodiment extracts features from X-ray security inspection images based on a multi-branch cross-stage local network to obtain multi-scale target feature maps. The multi-branch cross-stage local network includes a shallow feature extraction module, multiple deep feature extraction modules, and a semantic feature extraction module connected in sequence. Each deep feature extraction module includes a first module and a second module connected in series. The first module uses a multi-branch structure to downsample the input feature map and extract receptive field features at different scales. It then fuses the extracted feature maps at each scale receptive field to obtain a multi-scale fused feature map. Thus, the first module achieves the extraction of multi-scale receptive field features while downsampling, improving feature extraction capability and parameter utilization efficiency. The second module employs a dual-path structure to perform cross-stage deep feature extraction and fusion processing on the multi-scale fusion feature map, obtaining a target fusion feature map. The multi-scale target feature map includes: a target fusion feature map extracted by at least one deep feature extraction module and a semantic feature map extracted by a semantic feature extraction module. Through the collaborative processing of the first module and the second model, a multi-branch cross-stage local network with stronger feature extraction capabilities and higher parameter utilization efficiency is constructed, enabling the extraction of multi-scale target feature maps with richer feature information. By inputting the multi-scale target feature map into the target detection network for cross-scale fusion and contraband detection, the detection results of contraband in X-ray security inspection images can be obtained accurately and efficiently. The entire process requires no manual intervention, achieving automatic detection of contraband and improving the efficiency and accuracy of contraband detection.

[0040] Figure 5This is a flowchart illustrating a method for determining multi-scale fusion feature maps based on a first module, as provided in one embodiment of the present invention. Based on the aforementioned embodiments, this embodiment provides a detailed description of the specific architecture and processing procedure of the first module in each deep feature extraction module. Explanations of terms identical or corresponding to those in the aforementioned embodiments are not repeated here.

[0041] It should be noted that this embodiment uses the first module in the first deep feature extraction module as an example for illustration. The first module in other deep feature extraction modules is the same and will not be described again.

[0042] The first module includes multiple parallel branch structures and a splicing unit. Each branch structure includes a first sub-module for reducing the number of channels and a second sub-module for downsampling and adjusting the receptive field. The second sub-module adjusts the receptive field a different number of times in different branch structures.

[0043] The branching structure operates in parallel. Each branch adjusts the receptive field multiple times during downsampling, allowing for the extraction of receptive field features at different scales based on different branch structures. Each branch structure includes a first submodule and a second submodule. The first submodule can compress the feature channels to half the number of input features. The second submodule is used for simultaneous downsampling and receptive field adjustment. The second submodule can employ max-pooling layers or convolutional units with a stride of 2 for downsampling, and uses different numbers of 3×3 convolutional units to adjust the receptive field multiple times, enabling each branch structure to output features with receptive fields of different scales.

[0044] See Figure 5 The multi-scale fusion feature map is determined based on the first module by performing the following steps: S510. Input the shallow feature map into each branch structure in the first module for downsampling and extraction of receptive field features at different scales, and obtain the feature map under each scale receptive field.

[0045] Specifically, for the first module in the first deep feature extraction module, its input information is a shallow feature map. The shallow feature map is input into each branch of the first module for downsampling and extraction of receptive field features at different scales, thereby obtaining feature maps at each scale of receptive field extracted based on each branch structure.

[0046] For example, for each branch structure in the first module, the shallow feature map is input into the first sub-module of that branch structure to reduce the number of channels, and the processed feature map is obtained. The processed feature map is then input into the second sub-module of that branch structure to perform downsampling and receptive field adjustment for a corresponding number of times, and the adjusted feature map at the corresponding scale receptive field is obtained.

[0047] Figure 6 A schematic diagram of the network architecture for the first module is provided. (See diagram below.) Figure 6 As shown, the first module includes four branch structures. The first sub-module in each branch structure can be a convolutional unit with a 1×1 kernel and a stride of 1, used to reduce the number of channels. The second sub-module in the first branch structure includes a max pooling layer with a pooling size of 2×2 and a stride of 2; the second sub-module in the second branch structure includes a convolutional unit with a 3×3 kernel and a stride of 2; the second sub-module in the third branch structure includes a convolutional unit with a 3×3 kernel and a stride of 2 and a convolutional unit with a 3×3 kernel and a stride of 1; the second sub-module in the fourth branch structure includes a convolutional unit with a 3×3 kernel and a stride of 2 and two convolutional units with a 3×3 kernel and a stride of 1. Each convolutional unit includes a convolutional layer, a batch normalization layer, and an activation function layer connected in sequence.

[0048] In this approach, the number of feature channels is compressed to half that of the input features using the first submodule, while all convolutional units in the second submodule do not change the number of feature channels. The first module uses 0, 1, 2, and 3 convolutional units with 3×3 kernels in the four branch structures, respectively. In this case, the receptive fields of each element in the feature maps extracted by the four branch structures relative to the input features are 2, 3, 5, and 7, respectively. Compared to using a single receptive field of 3, this approach offers greater diversity and a significantly improved maximum receptive field. By setting up four branch structures, detection accuracy, model complexity, and computational cost can be effectively balanced, avoiding excessive computational burden caused by too many branch structures.

[0049] It should be noted that since the first module needs to adjust the feature map size as a downsampling module in the multi-branch cross-stage local network, the convolution stride of the first convolutional unit in the second sub-module of each branch structure is uniformly set to 2. Since the first branch structure does not have a 3×3 convolutional unit, max pooling layer is used to achieve the downsampling effect.

[0050] If downsampling is performed first in each branch structure, the number of channels in the input features will be high, significantly increasing the computational cost, which contradicts the design principle of constructing complex gradient paths with low computational cost and parameter count. Therefore, a first sub-module is added before downsampling to compress the number of channels in the input features. Similarly, placing downsampling at the end of each branch structure would also increase computational cost. Therefore, the downsampling module is placed after the first sub-module in each branch structure, effectively balancing computational cost and accuracy.

[0051] S520. Input the feature maps under each scale receptive field into the stitching unit for stitching processing under the channel dimension to obtain the stitched multi-scale fused feature map.

[0052] Specifically, see Figure 6 The feature maps extracted from each branch structure at different scales of the receptive field are input into the stitching unit for channel-level stitching processing, thereby obtaining a stitched multi-scale fused feature map. By directly stitching and fusing feature maps from multiple receptive fields at different scales, similar to the scaling operation when the human eye recognizes blurry objects, and combining information at different levels, subtle clues about contraband can be captured. Therefore, in scenarios such as X-ray security inspection images with complex backgrounds and indistinct target features, the ability to detect clues about contraband can be improved, thereby reducing the occurrence of missed detections of contraband.

[0053] For example, see Figure 6 The input information for the first module is C, H, and W represent the number of channels, height, and width of the input information, respectively. The output information is a multi-scale fused feature map, i.e. The specific calculation formula for this multi-scale fused feature map is as follows: In this diagram, F1, F2, F3, and F2 represent feature maps extracted from different receptive fields at four different scales, with CBS representing convolutional units. All branch structures first use 1×1 CBS convolutional units to compress the number of feature channels to half the number of input features; subsequent CBS convolutional units do not change the number of feature channels. The final output multi-scale fused feature map has twice the number of channels and half the image size compared to the input information.

[0054] The technical solution of this embodiment inputs shallow feature maps into each branch structure in the first module for downsampling and extraction of receptive field features at different scales, obtaining feature maps at each scale of receptive field. These feature maps are then input into a stitching unit for stitching processing at the channel dimension, resulting in a stitched multi-scale fused feature map. Thus, the first module uses a multi-branch structure to extract multi-scale receptive field features in parallel, and after subsequent fusion interaction, it is easier to discover clues about contraband, making it more suitable for X-ray image security inspection scenarios. Moreover, the finely designed multi-branch, multi-level structure provides more nonlinearity while completing the downsampling task, thereby possessing stronger feature extraction capabilities and improving the accuracy and precision of contraband detection.

[0055] Figure 7 This is a flowchart illustrating a method for determining target fusion feature maps based on a second module, as provided in one embodiment of the present invention. Based on the aforementioned embodiments, this embodiment provides a detailed description of the specific architecture and processing procedure of the second module in each deep feature extraction module. Explanations of terms identical or corresponding to those in the aforementioned embodiments are not repeated here.

[0056] It should be noted that this embodiment uses the second module in the first deep feature extraction module as an example for illustration. The structure of the second module in other deep feature extraction modules is similar and will not be described again.

[0057] The second module includes: a first-path structure, a second-path structure, a splicing unit, and a third sub-module for reducing the number of channels. The first-path structure includes a fourth sub-module for reducing the number of channels, and the second-path structure includes a fourth sub-module for reducing the number of channels, at least one feature extraction sub-module, and a fusion unit corresponding to each feature extraction sub-module. The number of feature extraction sub-modules is determined based on the sequence number of the deep feature extraction module to which the feature extraction sub-modules belong.

[0058] The second module employs a dual-path structure to further extract features based on the multi-scale fused feature map extracted by the first module, thereby obtaining sufficiently rich feature information. The first-path structure may only include a fourth sub-module for reducing the number of channels, thus reducing only the number of channels while retaining the features extracted by the first module. The second-path structure, in addition to the fourth sub-module, may also include at least one feature extraction sub-module and a corresponding fusion unit for each feature extraction sub-module, to extract new features based on the features extracted by the first module, obtaining at least one new feature, thereby improving feature richness and avoiding a significant increase in computational load. A concatenation unit concatenates and fuses the features output from each path. A third sub-module then adjusts the number of channels in the concatenated features to match the number of channels in the input information.

[0059] It is important to note that the number n of feature extraction sub-modules in the second path structure of the second module is related to its position in the multi-branch cross-stage local network. For example, the number n of feature extraction sub-modules in the second module of different deep feature extraction modules in the multi-branch cross-stage local network can be the same or different.

[0060] See Figure 7 The target fusion feature map is determined based on the second module by performing the following steps: S710. Input the multi-scale fused feature map into the first path structure in the second module to reduce the number of channels and obtain the reduced first feature map.

[0061] Figure 8 A schematic diagram of the network architecture for the second module is provided. For example... Figure 8 As shown, the first-path structure includes a fourth sub-module 1 for reducing the number of channels. The fourth sub-module 1 is a convolutional unit with a 1×1 kernel and a stride of 1 for reducing the number of channels. For example, the multi-scale fused feature map is input into the fourth sub-module 1 in the first-path structure to reduce the number of channels, and the first feature map output by the fourth sub-module 1 is obtained.

[0062] S720. Input the multi-scale fused feature map into the second path structure in the second module to reduce the number of channels and perform feature extraction and fusion processing at different depths to obtain the second feature map corresponding to each feature extraction sub-module.

[0063] like Figure 8 As shown, the second-path structure includes a fourth submodule 2, n feature extraction submodules, and a fusion unit corresponding to each feature extraction submodule. The fourth submodule 2 is a convolutional unit with a 1×1 kernel and a stride of 1, used to reduce the number of channels. Each feature extraction submodule includes two serially connected convolutional units with a 3×3 kernel.

[0064] For example, step S720 includes: inputting the multi-scale fused feature map into the fourth sub-module in the second path structure to reduce the number of channels, thereby obtaining a reduced fourth feature map; inputting the fourth feature map into the first feature extraction sub-module in the second path structure to perform deep feature extraction, thereby obtaining an extracted fifth feature map; inputting the fourth and fifth feature maps into the fusion unit corresponding to the first feature extraction sub-module to perform feature addition, thereby obtaining a second feature map corresponding to the first feature extraction sub-module; inputting the second feature map corresponding to the first feature extraction sub-module into the next feature extraction sub-module in the second path structure to perform deep feature extraction, and inputting the extracted feature map into the fusion unit corresponding to the next feature extraction sub-module to perform feature addition, thereby obtaining a second feature map corresponding to the next feature extraction sub-module.

[0065] See Figure 8 The multi-scale fused feature map is input into the fourth sub-module 2 in the second path structure to reduce the number of channels, and the fourth feature map output by the fourth sub-module 2 is obtained. The fourth feature map is then sequentially input into n feature extraction sub-modules and fusion units for feature extraction and fusion, and the second feature map after fusion of each feature extraction sub-module is obtained.

[0066] For example, when there are four deep feature extraction modules, the first deep feature extraction module has 1 feature extraction sub-module; the second deep feature extraction module has 2 feature extraction sub-modules; the third deep feature extraction module has 3 feature extraction sub-modules; and the fourth deep feature extraction module has 1 feature extraction sub-module.

[0067] Specifically, see Figure 2 and Figure 8 In the second path structure of the second module 1 in the deep feature extraction module 1, there is one feature extraction sub-module and a corresponding fusion unit, i.e., n=1; in the second path structure of the second module 2 in the deep feature extraction module 2, there are two feature extraction sub-modules and a corresponding fusion unit, i.e., n=2; in the second path structure of the second module 3 in the deep feature extraction module 3, there are three feature extraction sub-modules and a corresponding fusion unit, i.e., n=3; in the second path structure of the second module 4 in the deep feature extraction module 4, there is one feature extraction sub-module and a corresponding fusion unit, i.e., n=1.

[0068] S730. Input the first feature map and the second feature map corresponding to each feature extraction submodule into the stitching unit for stitching processing in the channel dimension to obtain the stitched third feature map.

[0069] See Figure 8 The first feature map and the second feature map output by each fusion unit are input into the stitching unit for stitching processing in the channel dimension to obtain the stitched third feature map.

[0070] S740. Input the third feature map into the third sub-module to reduce the number of channels, and obtain the target fusion feature map with the same number of channels as the multi-scale fusion feature map extracted by the first deep feature extraction module.

[0071] The third submodule can be a convolutional unit with a 1×1 kernel and a stride of 1, used to reduce the number of channels. See also Figure 8 The third feature map is input into the third submodule to reduce the number of channels, thereby obtaining a target fusion feature map with the same number of channels as the input multi-scale fusion feature map. This target fusion feature map is the feature map extracted by the first deep feature extraction module.

[0072] For example, the input information for the second module is C, H, and W represent the number of channels, height, and width of the input information, respectively. The second feature map corresponding to the i-th feature extraction submodule is... Output information of the second module The target fusion feature map is calculated using the following formula: For input features First, two CBS convolutional units with 1×1 kernels (i.e., the fourth submodule) are used to generate two features with C / 2 channels. and Then, n stacked feature extraction submodules and fusion units are used to process the features. Refinement is carried out to obtain characteristics. Finally, the features and The concatenation is performed along the channel direction, and a CBS convolutional unit with a 1×1 kernel (i.e., the third sub-module) is used to compress the number of channels to C, thus obtaining the output features. .

[0073] The technical solution of this embodiment reduces the number of channels by inputting the multi-scale fusion feature map into the first path structure of the second module to obtain a reduced first feature map; the multi-scale fusion feature map is then input into the second path structure of the second module to reduce the number of channels and perform feature extraction and fusion processing at different depths to obtain a second feature map corresponding to each feature extraction sub-module; the first feature map and the second feature map corresponding to each feature extraction sub-module are input into the stitching unit for stitching processing in the channel dimension to obtain a stitched third feature map; the third feature map is then input into the third sub-module to reduce the number of channels to obtain a target fusion feature map with the same number of channels as the multi-scale fusion feature map extracted by the first deep feature extraction module. This solution is suitable for detecting X-ray security images with cluttered backgrounds and weak features of contraband, and can extract more and richer feature information, thereby improving the accuracy and precision of contraband detection.

[0074] The following are embodiments of the prohibited substance detection device provided in this invention. This device and the prohibited substance detection methods in the above embodiments belong to the same inventive concept. For details not described in detail in the embodiments of the prohibited substance detection device, please refer to the embodiments of the above prohibited substance detection methods.

[0075] Figure 9 This is a schematic diagram of a prohibited item detection device provided in an embodiment of the present invention. This embodiment is applicable to situations in the security inspection field where prohibited items are detected by acquiring X-ray security inspection images. Figure 9 As shown, the device specifically includes: an image acquisition module 910, a feature extraction module 920, and a contraband detection module 930.

[0076] The image acquisition module 910 is used to acquire X-ray security inspection images; the feature extraction module 920 is used to extract features from the X-ray security inspection images based on a multi-branch cross-stage local network to obtain multi-scale target feature maps. The multi-branch cross-stage local network includes a shallow feature extraction module, multiple deep feature extraction modules, and a semantic feature extraction module connected in sequence. Each deep feature extraction module includes a first module and a second module connected in series. The first module uses a multi-branch structure to downsample the input feature map and extract receptive field features at different scales, and processes the extracted receptive field features at each scale. The first module performs fusion processing on the feature maps under the field to obtain a multi-scale fused feature map; the second module uses a dual-path structure to perform cross-stage deep feature extraction and fusion processing on the multi-scale fused feature map to obtain a target fused feature map; the multi-scale target feature map includes: at least one target fused feature map extracted by the deep feature extraction module and a semantic feature map extracted by the semantic feature extraction module; the contraband detection module 930 is used to input the multi-scale target feature map into the target detection network for cross-scale fusion and contraband detection to obtain the contraband detection result in the X-ray security inspection image.

[0077] The technical solution of this embodiment extracts features from X-ray security inspection images based on a multi-branch cross-stage local network to obtain multi-scale target feature maps. The multi-branch cross-stage local network includes a shallow feature extraction module, multiple deep feature extraction modules, and a semantic feature extraction module connected in sequence. Each deep feature extraction module includes a first module and a second module connected in series. The first module uses a multi-branch structure to downsample the input feature map and extract receptive field features at different scales. It then fuses the extracted feature maps at each scale receptive field to obtain a multi-scale fused feature map. Thus, the first module achieves the extraction of multi-scale receptive field features while downsampling, improving feature extraction capability and parameter utilization efficiency. The second module employs a dual-path structure to perform cross-stage deep feature extraction and fusion processing on the multi-scale fused feature map, obtaining a target fused feature map. The multi-scale target feature map includes: a target fused feature map extracted by at least one deep feature extraction module and a semantic feature map extracted by a semantic feature extraction module. Through the collaborative processing of the first module and the second model, a multi-branch cross-stage local network with stronger feature extraction capabilities and higher parameter utilization efficiency is constructed, thereby extracting a multi-scale target feature map with richer feature information. By inputting the multi-scale target feature map into the target detection network for cross-scale fusion and contraband detection, the detection results of contraband in X-ray security inspection images can be obtained accurately and efficiently. The entire process requires no manual intervention, achieving automatic detection of contraband and improving the efficiency and accuracy of contraband detection.

[0078] Optionally, the feature extraction module 920 includes: The shallow feature extraction submodule is used to input the X-ray security inspection image into the shallow feature extraction module to extract shallow features and obtain the extracted shallow feature map; The deep feature extraction submodule is used to input the shallow feature map into the first deep feature extraction module for deep feature extraction, and input the target fusion feature map extracted by the first deep feature extraction module into the next deep feature extraction module to continue deep feature extraction; The semantic feature extraction submodule is used to input the target fusion feature map extracted by the last deep feature extraction module into the semantic feature extraction module to extract semantic features and obtain the extracted semantic feature map. The multi-scale target feature map determination submodule is used to determine the target fusion feature map extracted by at least one deep feature extraction module and the semantic feature map extracted by the semantic feature extraction module as a multi-scale target feature map.

[0079] Optionally, the deep feature extraction submodule includes: The multi-scale fusion feature map determination unit is used to input the shallow feature map into the first module of the first deep feature extraction module for downsampling and extraction of receptive field features at different scales, and to perform fusion processing on the feature maps under each scale receptive field to obtain a multi-scale fusion feature map. The target fusion feature map determination unit is used to input the multi-scale fusion feature map into the second module in the first deep feature extraction module for cross-stage deep feature extraction and fusion processing, so as to obtain the target fusion feature map extracted by the first deep feature extraction module.

[0080] Optionally, the first module includes: multiple parallel branch structures and a splicing unit, each branch structure including a first sub-module for reducing the number of channels and a second sub-module for downsampling and adjusting the receptive field, wherein the second sub-module adjusts the receptive field a different number of times in different branch structures; The multi-scale fusion feature map determination unit is specifically used for: The shallow feature map is input into each branch structure in the first module for downsampling and extraction of receptive field features at different scales to obtain feature maps at each scale of receptive field. The feature maps at each scale of receptive field are then input into the stitching unit for stitching processing at the channel dimension to obtain a stitched multi-scale fused feature map.

[0081] Optionally, the first submodule is a convolutional unit with a kernel of 1×1 and a stride of 1; When the first module includes four branch structures, the second sub-module in the first branch structure includes a max pooling layer with a pooling size of 2×2 and a stride of 2; the second sub-module in the second branch structure includes a convolutional unit with a kernel of 3×3 and a stride of 2; the second sub-module in the third branch structure includes a convolutional unit with a kernel of 3×3 and a stride of 2 and a convolutional unit with a kernel of 3×3 and a stride of 1; the second sub-module in the fourth branch structure includes a convolutional unit with a kernel of 3×3 and a stride of 2 and two convolutional units with a kernel of 3×3 and a stride of 1. The convolutional unit comprises a convolutional layer, a batch normalization layer, and an activation function layer connected in sequence.

[0082] Optionally, the second module includes: a first path structure, a second path structure, a splicing unit, and a third sub-module for reducing the number of channels, wherein the first path structure includes a fourth sub-module for reducing the number of channels, the second path structure includes a fourth sub-module for reducing the number of channels, at least one feature extraction sub-module, and a fusion unit corresponding to each feature extraction sub-module, wherein the number of feature extraction sub-modules is determined based on the sequence number of the deep feature extraction module to which the feature extraction sub-module is located; The target fusion feature map determination unit includes: The first feature map determination subunit is used to input the multi-scale fused feature map into the first path structure in the second module to reduce the number of channels and obtain the reduced first feature map; The second feature map determination subunit is used to input the multi-scale fused feature map into the second path structure in the second module to reduce the number of channels and perform feature extraction and fusion processing at different depths, so as to obtain the second feature map corresponding to each feature extraction submodule. The third feature map determination subunit is used to input the first feature map and the second feature map corresponding to each feature extraction submodule into the stitching unit for stitching processing in the channel dimension to obtain the stitched third feature map. The target fusion feature map determination subunit is used to input the third feature map into the third submodule for channel number reduction processing to obtain the target fusion feature map extracted by the first deep feature extraction module, which has the same number of channels as the multi-scale fusion feature map.

[0083] Optionally, the second feature map determines the sub-unit, specifically for: The multi-scale fused feature map is input into the fourth sub-module of the second path structure to reduce the number of channels, obtaining a reduced fourth feature map; the fourth feature map is input into the first feature extraction sub-module of the second path structure to perform deep feature extraction, obtaining an extracted fifth feature map; the fourth and fifth feature maps are input into the fusion unit corresponding to the first feature extraction sub-module to perform feature addition, obtaining a second feature map corresponding to the first feature extraction sub-module; the second feature map corresponding to the first feature extraction sub-module is input into the next feature extraction sub-module of the second path structure to perform deep feature extraction, and the extracted feature map is input into the fusion unit corresponding to the next feature extraction sub-module to perform feature addition, obtaining a second feature map corresponding to the next feature extraction sub-module.

[0084] Optionally, both the third submodule and the fourth submodule are convolutional units with a 1×1 kernel; Each feature extraction submodule includes two 3×3 convolutional units connected in series; When there are four deep feature extraction modules, the number of feature extraction sub-modules in the first deep feature extraction module is 1; the number of feature extraction sub-modules in the second deep feature extraction module is 2; the number of feature extraction sub-modules in the third deep feature extraction module is 3; and the number of feature extraction sub-modules in the fourth deep feature extraction module is 1.

[0085] Optionally, when the number of deep feature extraction modules is four, the multi-scale target feature map includes: the target fusion feature map extracted by the second deep feature extraction module, the target fusion feature map extracted by the third deep feature extraction module, and the semantic feature map extracted by the semantic feature extraction module.

[0086] The prohibited substance detection device provided in the embodiments of the present invention can execute the prohibited substance detection method provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects for executing the prohibited substance detection method.

[0087] It is worth noting that in the embodiments of the above-mentioned prohibited items detection device, the various units and modules included are only divided according to functional logic, but are not limited to the above division, as long as the corresponding functions can be achieved; in addition, the specific names of each functional unit are only for easy differentiation and are not used to limit the scope of protection of the present invention.

[0088] Figure 10 A schematic diagram of an electronic device 10, which can be used to implement embodiments of the present invention, is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices (e.g., helmets, glasses, watches, etc.), and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the invention described and / or claimed herein.

[0089] like Figure 10 As shown, the electronic device 10 includes at least one processor 11 and a memory, such as a read-only memory (ROM) 12 or a random access memory (RAM) 13, communicatively connected to the at least one processor 11. The memory stores computer programs executable by the at least one processor. The processor 11 can perform various appropriate actions and processes based on the computer program stored in the ROM 12 or loaded from storage unit 18 into the RAM 13. The RAM 13 can also store various programs and data required for the operation of the electronic device 10. The processor 11, ROM 12, and RAM 13 are interconnected via a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.

[0090] Multiple components in electronic device 10 are connected to I / O interface 15, including: input unit 16, such as keyboard, mouse, etc.; output unit 17, such as various types of displays, speakers, etc.; storage unit 18, such as disk, optical disk, etc.; and communication unit 19, such as network card, modem, wireless transceiver, etc. Communication unit 19 allows electronic device 10 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0091] Processor 11 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. Processor 11 performs the contraband detection method described above.

[0092] In some embodiments, the contraband detection method may be implemented as a computer program tangibly contained in a computer-readable storage medium, such as storage unit 18. In some embodiments, part or all of the computer program may be loaded and / or installed on electronic device 10 via ROM 12 and / or communication unit 19. When the computer program is loaded into RAM 13 and executed by processor 11, one or more steps of the contraband detection method described above may be performed. Alternatively, in other embodiments, processor 11 may be configured to perform the contraband detection method by any other suitable means (e.g., by means of firmware).

[0093] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), payload-programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0094] Computer programs used to implement the contraband detection method of the present invention can be written in any combination of one or more programming languages. These computer programs can be provided to the processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, such that when executed by the processor, the computer programs cause the functions / operations specified in the flowcharts and / or block diagrams to be implemented. The computer programs can be executed entirely on a machine, partially on a machine, as a standalone software package partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0095] In the context of this invention, a computer-readable storage medium can be a tangible medium that may contain or store a computer program for use by or in conjunction with an instruction execution system, apparatus, or device. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination thereof. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0096] To provide interaction with a user, the systems and techniques described herein can be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the electronic device. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0097] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as data servers), or middleware components (e.g., application servers), or frontend components (e.g., user computers with graphical user interfaces or web browsers through which users can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., communication networks). Examples of communication networks include local area networks (LANs), wide area networks (WANs), blockchain networks, and the Internet.

[0098] A computing system can include clients and servers. Clients and servers are generally located far apart and typically interact through communication networks. The client-server relationship is created by computer programs running on the respective computers and having a client-server relationship with each other. The server can be a cloud server, also known as a cloud computing server or cloud host, which is a hosting product within the cloud computing service system to address the shortcomings of traditional physical hosts and VPS services, such as high management difficulty and weak business scalability.

[0099] This invention also provides a computer program product, including a computer program that, when executed by a processor, implements the contraband detection method provided in any embodiment of this invention.

[0100] In the implementation of a computer program product, computer program code for performing the operations of this invention can be written in one or more programming languages ​​or a combination thereof. Programming languages ​​include object-oriented programming languages ​​as well as conventional procedural programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0101] It should be understood that the various forms of processes shown above can be used, with steps reordered, added, or deleted. For example, the steps described in this invention can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution of this invention can be achieved, and this is not limited herein.

[0102] The specific embodiments described above do not constitute a limitation on the scope of protection of this invention. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this invention should be included within the scope of protection of this invention.

Claims

1. A method for detecting contraband, characterized in that, include: Obtain X-ray security inspection images; Based on a multi-branch, cross-stage local network, feature extraction is performed on the X-ray security inspection image to obtain a multi-scale target feature map. The multi-branch, cross-stage local network includes a shallow feature extraction module, multiple deep feature extraction modules, and a semantic feature extraction module connected sequentially. Each deep feature extraction module includes a first module and a second module connected in series. The first module uses a multi-branch structure to downsample the input feature map and extract features from receptive fields at different scales, and then fuses the extracted feature maps at each scale receptive field to obtain a multi-scale fused feature map. The second module uses a dual-path structure to perform cross-stage deep feature extraction and fusion processing on the multi-scale fused feature map to obtain a target fused feature map. The multi-scale target feature map includes: at least one target fused feature map extracted by the deep feature extraction module and a semantic feature map extracted by the semantic feature extraction module. The multi-scale target feature map is input into the target detection network for cross-scale fusion and contraband detection to obtain the contraband detection results in the X-ray security inspection image.

2. The method according to claim 1, characterized in that, The feature extraction of the X-ray security inspection image based on a multi-branch cross-stage local network to obtain a multi-scale target feature map includes: The X-ray security inspection image is input into the shallow feature extraction module to extract shallow features and obtain the extracted shallow feature map; The shallow feature map is input into the first deep feature extraction module for deep feature extraction, and the target fusion feature map extracted by the first deep feature extraction module is input into the next deep feature extraction module to continue deep feature extraction. The target fusion feature map extracted by the last deep feature extraction module is input into the semantic feature extraction module to extract semantic features, thereby obtaining the extracted semantic feature map; The target fusion feature map extracted by at least one deep feature extraction module and the semantic feature map extracted by the semantic feature extraction module are determined as a multi-scale target feature map.

3. The method according to claim 2, characterized in that, The step of inputting the shallow feature map into the first deep feature extraction module for deep feature extraction includes: The shallow feature map is input into the first module of the first deep feature extraction module for downsampling and extraction of receptive field features at different scales. The feature maps extracted at each scale of receptive field are then fused to obtain a multi-scale fused feature map. The multi-scale fused feature map is input into the second module of the first deep feature extraction module for cross-stage deep feature extraction and fusion processing to obtain the target fused feature map extracted by the first deep feature extraction module.

4. The method according to claim 3, characterized in that, The first module includes: multiple parallel branch structures and a splicing unit. Each branch structure includes a first sub-module for reducing the number of channels and a second sub-module for downsampling and adjusting the receptive field. The second sub-module adjusts the receptive field a different number of times in different branch structures. The process involves inputting the shallow feature map into the first module of the first deep feature extraction module for downsampling and extraction of receptive field features at different scales, and then fusing the extracted feature maps at each scale of receptive field to obtain a multi-scale fused feature map, including: The shallow feature map is input into each branch structure in the first module for downsampling and extraction of receptive field features at different scales, so as to obtain the feature map under each scale of receptive field. The feature maps under each scale receptive field are input into the stitching unit for stitching processing in the channel dimension to obtain the stitched multi-scale fused feature map.

5. The method according to claim 4, characterized in that, The first submodule is a convolutional unit with a kernel size of 1×1 and a stride of 1; When the first module includes four branch structures, the second sub-module in the first branch structure includes a max pooling layer with a pooling size of 2×2 and a stride of 2; the second sub-module in the second branch structure includes a convolutional unit with a kernel of 3×3 and a stride of 2; the second sub-module in the third branch structure includes a convolutional unit with a kernel of 3×3 and a stride of 2 and a convolutional unit with a kernel of 3×3 and a stride of 1; the second sub-module in the fourth branch structure includes a convolutional unit with a kernel of 3×3 and a stride of 2 and two convolutional units with a kernel of 3×3 and a stride of 1. The convolutional unit comprises a convolutional layer, a batch normalization layer, and an activation function layer connected in sequence.

6. The method according to claim 3, characterized in that, The second module includes: a first-path structure, a second-path structure, a splicing unit, and a third sub-module for reducing the number of channels. The first-path structure includes a fourth sub-module for reducing the number of channels, and the second-path structure includes a fourth sub-module for reducing the number of channels, at least one feature extraction sub-module, and a fusion unit corresponding to each feature extraction sub-module. The number of feature extraction sub-modules is determined based on the sequence number of the deep feature extraction module to which the feature extraction sub-module is located. The step of inputting the multi-scale fused feature map into the second module of the first deep feature extraction module for cross-stage deep feature extraction and fusion processing to obtain the target fused feature map extracted by the first deep feature extraction module includes: The multi-scale fused feature map is input into the first path structure in the second module to reduce the number of channels, thereby obtaining the reduced first feature map; The multi-scale fused feature map is input into the second path structure in the second module to reduce the number of channels and perform feature extraction and fusion processing at different depths to obtain the second feature map corresponding to each feature extraction sub-module; The first feature map and the second feature map corresponding to each feature extraction submodule are input into the stitching unit for stitching processing in the channel dimension to obtain the stitched third feature map. The third feature map is input into the third sub-module to reduce the number of channels, thereby obtaining the target fusion feature map extracted by the first deep feature extraction module, which has the same number of channels as the multi-scale fusion feature map.

7. The method according to claim 6, characterized in that, The step of inputting the multi-scale fused feature map into the second path structure of the second module for channel reduction and feature extraction and fusion processing at different depths to obtain the second feature map corresponding to each feature extraction sub-module includes: The multi-scale fused feature map is input into the fourth sub-module in the second path structure to reduce the number of channels, thereby obtaining the reduced fourth feature map; The fourth feature map is input into the first feature extraction submodule in the second path structure for deep feature extraction to obtain the extracted fifth feature map; The fourth feature map and the fifth feature map are input into the fusion unit corresponding to the first feature extraction submodule for feature addition processing to obtain the second feature map corresponding to the first feature extraction submodule. The second feature map corresponding to the first feature extraction submodule is input into the next feature extraction submodule in the second path structure for deep feature extraction, and the extracted feature map is input into the fusion unit corresponding to the next feature extraction submodule for feature addition processing to obtain the second feature map corresponding to the next feature extraction submodule.

8. The method according to claim 6, characterized in that, Both the third submodule and the fourth submodule are convolutional units with a 1×1 kernel; Each feature extraction submodule includes two 3×3 convolutional units connected in series; When there are four deep feature extraction modules, the number of feature extraction sub-modules in the first deep feature extraction module is 1; the number of feature extraction sub-modules in the second deep feature extraction module is 2; the number of feature extraction sub-modules in the third deep feature extraction module is 3; and the number of feature extraction sub-modules in the fourth deep feature extraction module is 1.

9. The method according to any one of claims 1-8, characterized in that, When the number of deep feature extraction modules is four, the multi-scale target feature map includes: the target fusion feature map extracted by the second deep feature extraction module, the target fusion feature map extracted by the third deep feature extraction module, and the semantic feature map extracted by the semantic feature extraction module.

10. A contraband detection device, characterized in that, include: The image acquisition module is used to acquire X-ray security inspection images; A feature extraction module is used to extract features from the X-ray security inspection image based on a multi-branch cross-stage local network to obtain a multi-scale target feature map. The multi-branch cross-stage local network includes a shallow feature extraction module, multiple deep feature extraction modules, and a semantic feature extraction module connected in sequence. Each deep feature extraction module includes a first module and a second module connected in series. The first module uses a multi-branch structure to downsample the input feature map and extract features from receptive fields at different scales, and then fuses the extracted feature maps at each scale receptive field to obtain a multi-scale fused feature map. The second module uses a dual-path structure to perform cross-stage deep feature extraction and fusion processing on the multi-scale fused feature map to obtain a target fused feature map. The multi-scale target feature map includes at least one target fused feature map extracted by the deep feature extraction module and a semantic feature map extracted by the semantic feature extraction module. The contraband detection module is used to input the multi-scale target feature map into the target detection network for cross-scale fusion and contraband detection, so as to obtain the contraband detection results in the X-ray security inspection image.

11. An electronic device, characterized in that, The electronic device includes: At least one processor; and A memory communicatively connected to the at least one processor; wherein, The memory stores a computer program that can be executed by the at least one processor, the computer program being executed by the at least one processor to enable the at least one processor to perform the contraband detection method according to any one of claims 1-9.

12. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions that cause a processor to execute the method for detecting contraband as described in any one of claims 1-9.

13. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by the processor, it implements the contraband detection method as described in any one of claims 1-9.

Images