A real-time security threat article detection method based on feature decoupling

Abstract

The present application relates to a kind of real-time security threat article detection methods based on feature decoupling, belong to image processing technical field.The method is detected using threat article detection neural network model, the model includes backbone network, detection neck network and detection decoupling head network;Backbone network is using feature pyramid to carry out feature extraction, generates multi-scale feature map to capture the object information under different scales;Detection neck network is using pyramid structure to the different scale features extracted in backbone network are fused;Detection decoupling head network includes classification branch and positioning branch;Classification branch uses semantic context coding, to enhance the attention to context features in classification task, to improve the extraction ability of the key part features of threat article;Positioning branch uses detail retention coding, to strengthen the attention of detection head to threat article outline, edge and detail texture, ensure the accuracy of detection result.

Description

Technical Field

[0001] This invention belongs to the field of image processing technology and relates to a real-time security inspection method for detecting threatening items based on feature decoupling. Background Technology

[0002] Ensuring the safe transport of objects on public transportation, such as airports and subways, is of paramount importance. However, in many cases, threatening items can only be detected by inspectors standing next to X-ray machines, requiring them to maintain high concentration for extended periods—a challenging task requiring significant manpower. Therefore, in recent years, the automated detection of threatening items in X-ray images using machines has become increasingly important. Unlike natural images, X-ray images are typically pseudo-color, their generation relying on X-ray penetration, photosensitive properties, and fluorescence effects. Furthermore, threatening items in X-ray images are often small and densely packed, and these targets frequently overlap, resulting in significant background noise.

[0003] With the rapid development of deep learning, researchers have introduced various network architectures into X-ray images for the classification and detection of threatening objects. Object detection, aiming to identify and locate targets in natural images, is a fundamental task in computer vision. In recent years, deep learning-based methods have dominated research progress in object detection, which is typically viewed as the classification of regions of interest and the localization of bounding boxes in an image. The semantic context encoded in high-level semantic features plays a crucial role in object classification and localization. Feature pyramid networks enhance the detection capability for small-scale targets by extracting high-level semantic information and fusing it into low-level detail-preserving feature maps. This design provides richer semantic context for early features, supporting the detection of multi-scale targets.

[0004] However, the semantic requirements for classification and bounding box regression differ: localization requires more texture details and edge information, while classification needs rich semantic context. To address this issue, researchers proposed using two decoupled head branches on each feature map layer to perform classification and localization tasks separately—a strategy known as parameter decoupling. While this approach has theoretical advantages, we observe that decoupling classification and localization only at the parameter level still struggles to achieve an ideal balance between the two tasks. Relying solely on the head network's learning ability to extract task-specific context from shared feature maps often leads to a bias towards one task, thus suppressing the performance of the other. This indicates that current shared feature maps are still optimized for both competing tasks and require further research and improvement. Summary of the Invention

[0005] In view of this, the purpose of this invention is to provide a real-time security inspection method for detecting threatening items based on feature decoupling, thereby improving the accuracy of threatening item detection.

[0006] To achieve the above objectives, the present invention provides the following technical solution:

[0007] A real-time security inspection method for detecting threatening items based on feature decoupling specifically includes the following steps:

[0008] S1: Construct a training set of image data of threatening items for security inspection;

[0009] S2: Construct a neural network model for threat item detection, including a backbone network, a detection neck network, and a detection decoupling head network;

[0010] The backbone network uses a feature pyramid for feature extraction to generate multi-scale feature maps to capture object information at different scales.

[0011] The detection neck network utilizes a pyramid structure to fuse features of different scales extracted from the backbone network;

[0012] The detection decoupling head network includes a classification branch and a localization branch. The classification branch uses semantic context encoding to enhance the attention to contextual features in the classification task, thereby improving the ability to extract key features of threatening items. The localization branch uses detail-preserving encoding to strengthen the detection head's attention to the outline, edges, and detailed texture of threatening items, ensuring the accuracy of the detection results.

[0013] S3: Input the training set into the threat item detection neural network model to train the optimal network model;

[0014] S4: Input the dataset to be inspected into the trained network model to perform detection and identify threatening items.

[0015] Furthermore, in step S2, the pyramid structure used in the detection neck network includes bottom-up and top-down feature extraction paths, as well as lateral connection modules. In the lateral connections, information from the feature maps is extracted from each layer of the backbone network and the number of channels is adjusted through convolution operations. The bottom-up path generates rich semantic representations from low-level feature maps through downsampling techniques. The top-down path uses upsampling and interpolation methods to obtain high-resolution features from high-level feature maps, thereby preserving more details. Finally, the feature maps from these two paths are element-wise added or concatenated with the feature maps obtained from the lateral connections to form a fused multi-scale feature representation.

[0016] Furthermore, in step S2, the specific structure of the semantic context encoding is as follows: at different levels of each threat item feature map, the semantic context encoding utilizes feature maps from two levels, namely F l and F l+1A more semantically rich classification feature map is generated through concatenation; firstly, feature map F is... l Downsampled by 2 times and compared with F l+1 The feature maps are then stitched together to obtain the final feature map.

[0017]

[0018] Here, Concat represents the concatenation operation, DConv represents the downsampling convolution, and l represents the l-th layer; the concatenation operation and the depthwise separable convolution together constitute a shared downsampling convolution.

[0019] It is important to note that the processed feature map F l cls The resolution is the original feature map F l Half of it. Next, Input into the classification branch; because It is F l A 2x downsampled version, therefore Each point at position (i,j) in the feature map will predict F. l The classification score of its four nearest neighbors is denoted as S. Where N is the number of categories; then, S is rearranged as follows: To restore resolution:

[0020]

[0021] In this way, we not only utilize the sparsity of salient features (from F...) l ), and also from higher pyramid levels (from F l+1 This benefits from rich context. This will help to infer item categories more effectively, especially for threat items that are untextured or heavily occluded.

[0022] Furthermore, in step S2, given the input feature map... Depthwise separable convolution is divided into channel-wise convolution and point-wise convolution. The specific steps are as follows:

[0023] 1) Channel-by-channel convolution: Each channel undergoes a k×k convolution operation separately, outputting a feature map. Where H, W, and C represent the height, width, and number of channels of the feature map, respectively;

[0024] The formula for calculating channel-wise convolution is:

[0025]

[0026] Among them, W m,n,c b represents the convolution kernel weights. cThe term represents the bias term, i and j represent the position indices of the output feature map (two-dimensional), m and n represent the position indices of the convolution kernel (two-dimensional), and c represents the c-th channel;

[0027] 2) Pointwise convolution: X' is converted into the target number of channels C' through 1×1 convolution to obtain the final output.

[0028] Furthermore, in step S2, the specific structure of the detail-preserving encoding is as follows: in each threat item feature layer level L, the detail-preserving encoding receives feature maps from three pyramid levels, namely F. l-1 F l and F l+1 Among them, F l-1 Provides more detail and edge features, while F l+1 This provides a more comprehensive view of the items. Our experiments show that each security check level is primarily related to its two adjacent levels, and merging too many levels may negatively impact performance.

[0029] To improve computational efficiency, we adopt a simplified U-Net structure to fuse F l-1 and F l+1 First, put F l After upsampling by 2 times and F l-1 The feature maps are aggregated and then downsampled using a 3×3 convolutional layer with a stride of 2; the entire process includes upsampling, feature aggregation, and downsampling; the final detail-preserving feature map... Represented as:

[0030]

[0031] Where DConv represents downsampling convolution, UConv represents upsampling convolution, Concat represents concatenation operation, and l represents the l-th layer.

[0032] Its specific implementation is as follows:

[0033] (1) Upsampling: Upsampling the feature map Perform upsampling by 2x to obtain the feature map.

[0034]

[0035] Where i, j, and c represent the vertical position index, horizontal position index, and channel index of the feature map after upsampling, respectively, corresponding to the c-th channel of the feature map. H, W, and C1 represent the height, width, and number of channels of the feature map before upsampling (the number of channels usually remains unchanged during the upsampling process).

[0036] (2) Feature aggregation: The upsampled feature map X' is combined with the feature map... By splicing the images together, we obtain the feature map.

[0037]

[0038] Where C2 represents the number of channels in another feature map;

[0039] (3) Downsampling: The aggregated feature map Z is downsampled by a 3×3 convolutional layer with a stride of 2 to obtain the final detail-preserving feature map.

[0040]

[0041] Among them, W m,n,c′ These are the convolutional kernel weights, with a size of 3×3; b c′ It is the corresponding bias term;

[0042] Finally, the formula for calculating the detail-preserving feature map D can be summarized as follows:

[0043]

[0044] Next, Input is fed into the localization branch to infer the bounding box.

[0045] The beneficial effects of this invention are as follows: This invention designs the backbone structure of a convolutional neural network and utilizes feature pyramids for feature extraction, generating multi-scale feature maps to capture object information at different scales. Based on the different requirements of classification and localization tasks for feature maps in security threat item detection, a detection decoupling head is designed to optimize model performance. Semantic context encoding is designed to enhance the focus on contextual features in classification tasks, thereby improving the ability to extract key features of dangerous items. Detail-preserving encoding is designed to strengthen the detection head's focus on the contours, edges, and detailed textures of threat items, ensuring the accuracy of detection results.

[0046] Other advantages, objectives, and features of the invention will be set forth in part in the description which follows, and in part will be apparent to those skilled in the art from the following examination, or may be learned from practice of the invention. The objectives and other advantages of the invention can be realized and obtained through the following description. Attached Figure Description

[0047] To make the objectives, technical solutions, and advantages of the present invention clearer, the preferred embodiments of the present invention will be described in detail below with reference to the accompanying drawings, wherein:

[0048] Figure 1 This is an overall framework diagram of the threatening item detection neural network model proposed in this invention;

[0049] Figure 2 For semantic context encoding (CE) structure;

[0050] Figure 3 Preserve the coding (DE) structure for details;

[0051] Figure 4 The results show the comparison of heatmaps detected on the SIXRAY dataset. Detailed Implementation

[0052] The following specific examples illustrate the implementation of the present invention. Those skilled in the art can easily understand other advantages and effects of the present invention from the content disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and various details in this specification can be modified or changed based on different viewpoints and applications without departing from the spirit of the present invention. It should be noted that the illustrations provided in the following embodiments are only schematic representations of the basic concept of the present invention. Unless otherwise specified, the following embodiments and features can be combined with each other.

[0053] The accompanying drawings are for illustrative purposes only and are schematic diagrams, not actual pictures. They should not be construed as limiting the invention. To better illustrate the embodiments of the invention, some parts in the drawings may be omitted, enlarged, or reduced, and do not represent the actual product dimensions. It is understandable to those skilled in the art that some well-known structures and their descriptions may be omitted in the drawings.

[0054] In the accompanying drawings of the embodiments of the present invention, the same or similar reference numerals correspond to the same or similar components. In the description of the present invention, it should be understood that if terms such as "upper," "lower," "left," "right," "front," and "rear" indicate the orientation or positional relationship based on the orientation or positional relationship shown in the drawings, they are only for the convenience of describing the present invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, the terms used to describe positional relationships in the drawings are only for illustrative purposes and should not be construed as limiting the present invention. For those skilled in the art, the specific meaning of the above terms can be understood according to the specific circumstances.

[0055] Please see Figures 1-4 This invention provides a real-time security inspection method for detecting threatening items based on feature decoupling, comprising the following steps:

[0056] S1: Construct a training set of image data of security-sensitive items to facilitate effective learning by the model.

[0057] The training image datasets used were the SIXRAY and PIDRAY datasets, with a training batch size of 32 per iteration. Multi-scale training was performed on both datasets, including image sizes ranging from 320 to 640 pixels with 10 equidistant intervals. To enhance the model's representational power with a limited number of instances, a combination of fusion, label smoothing, and randomized affine transformations was employed. The image size for testing was 640 pixels.

[0058] S2: Design the backbone network and detection neck network of the convolutional neural network, and use the feature pyramid for feature extraction to generate multi-scale feature maps to capture object information at different scales.

[0059] The convolutional neural network designed in this invention is as follows: Figure 1 As shown, the input image is first processed by a backbone network to extract multi-scale features. In this network, the outputs of stages 3, 4, and 5 are represented as P3, P4, and P5, respectively. These outputs can serve as the basis for subsequent feature fusion. The feature pyramid of the detection neck network takes the feature maps in P3, P4, and P5 as input to generate the output feature maps in the pyramid structure, which are then used by the subsequent detection decoupling head network for processing. Figure 1 In this code, "Conv-n" represents a standard n×n convolution operation, "C" represents a connection operation, and "↑" represents an upsampling operation. The CompositeAttention (CA) module aims to enhance the detection capability of small objects. Furthermore, the 3ConvolutionsandFusion (C3F) module is used to improve the model's detection efficiency. The Composite Convolution (CC) module is a composite module derived from the original C3 (Convolutions) and convolution operations. P3, P4, and P5 represent features extracted from the backbone network, used for feature fusion.

[0060] Backbone networks utilize feature pyramids for feature extraction, generating multi-scale feature maps to capture object information at different scales.

[0061] The neck detection network effectively addresses the multi-scale problem through a feature pyramid architecture. This architecture includes bottom-up and top-down feature extraction paths, as well as lateral connection modules. In the lateral connections, information from the feature maps is extracted from each layer of the backbone network and the number of channels is adjusted through convolutional operations. The bottom-up path uses downsampling techniques to generate rich semantic representations from low-level feature maps, while the top-down path uses upsampling and interpolation methods to obtain high-resolution features from high-level feature maps, thus preserving more details. Finally, the feature maps from these two paths are element-wise added or concatenated with the feature map obtained from the lateral connections to form a fused multi-scale feature representation.

[0062] S3: Based on the different requirements of the classification task and the localization task for feature maps in the detection of threatening items in security inspection, a detection decoupling head network is designed to optimize the model performance.

[0063] The detection decoupling head network consists of two branches: the localization (LOC) branch and the classification (CLS) branch.

[0064] Hazardous materials security inspection is a crucial security measure aimed at detecting and locating potentially threatening items. In this process, classification and localization are two related yet conflicting tasks. Specifically, classification relies on richer contextual information to identify item categories, while localization requires precise local information (texture details and edge information) to determine the item's exact location. Currently, typical decoupled detectors utilize feature maps F at the same level. l Based on this, we calculate the minimization of classification loss and localization loss. However, classification and localization require different semantic information in their feature maps. To alleviate the conflict between classification and localization, this invention proposes to decouple the feature encoding of these two tasks at the source, using feature maps with different contexts to employ different feature encodings for different tasks. This is in contrast to using F... l Unlike a unified input, this invention provides task-specific input features for both branches, namely F. cls and F loc .

[0065] S4: Design semantic context encoding (CE) to enhance the attention to contextual features in classification tasks, thereby improving the ability to extract key features of dangerous goods.

[0066] Specifically, at different levels of each threat item feature map, semantic context encoding utilizes feature maps from two levels, namely F l and F l+1 A more semantically rich classification feature map is generated through concatenation. First, feature map F is... l Downsampled by 2 times and compared with F l+1 The feature maps are then stitched together to obtain the final feature map.

[0067]

[0068] The concatenation operation and the depthwise separable convolution here together form a shared downsampling convolution. Given the input feature map... Depthwise separable convolution is divided into channel-wise convolution and point-wise convolution. The specific steps are as follows:

[0069] 1) Channel-by-channel convolution: Each channel undergoes a k×k convolution operation separately, outputting a feature map.

[0070] The formula for calculating channel-wise convolution is:

[0071]

[0072] Among them, W m,n,c b represents the convolution kernel weights. c This indicates the bias term.

[0073] 2) Pointwise convolution: X' is converted into the target number of channels C' through 1×1 convolution to obtain the final output.

[0074] It is important to note that the processed feature map The resolution is the original feature map F l Half of it. Next, Input into the classification branch. Because It is F l A 2x downsampled version, therefore Each point at position (i,j) in the feature map will predict F. l The classification score of its four nearest neighbors is denoted as S. Where N is the number of categories. Next, S is rearranged as follows: To restore resolution:

[0075]

[0076] In this way, we not only utilize the sparsity of salient features (from F...) l ), and also from higher pyramid levels (from F l+1 This benefits from rich context. This will help to infer item categories more effectively, especially for threat items that are untextured or heavily occluded.

[0077] S5: Design Detail Preservation Code (DE) enhances the detection head's focus on the outline, edges, and detailed textures of threatening objects, ensuring the accuracy of detection results.

[0078] Unlike classification, localization is a more fine-grained task, relying on more texture details and boundary information to predict the location of an object. However, existing methods typically regress only the corners of the target from a single-scale feature map F. Low-level pyramid feature maps respond more strongly to the outline, edges, and detailed texture of the object, which helps improve localization performance, but often comes with a huge computational cost. We reduce the computational cost of low-level feature maps through sparse convolution, but still require additional auxiliary branches and specific supervision signals. On the other hand, we note that high-level feature maps are equally important for localization because they provide the most complete view of the target possible, thus providing more information to infer the overall shape.

[0079] Based on these observations, this invention proposes a detail-preserving encoding method to achieve accurate localization without sacrificing efficiency. In each threat item feature layer level L, the detail-preserving encoding method receives feature maps from three pyramid levels, namely F. l-1 F l and F l+1 Among them, F l-1 Provides more detail and edge features, while F l+1 This provides a more comprehensive view of the items. This experiment shows that each security check level is mainly related to its two adjacent levels, and merging too many levels may have a negative impact on performance.

[0080] To improve computational efficiency, this invention employs a simplified U-Net structure fused with F l-1 and F l+1 First, put F l After upsampling by 2 times and F l-1 The features are aggregated and then downsampled using a 3×3 convolutional layer with a stride of 2. The entire process includes upsampling, feature aggregation, and downsampling. The final detail-preserving feature map... Represented as:

[0081]

[0082] Its specific implementation is as follows:

[0083] (1) Upsampling: Upsampling the feature map Perform upsampling by 2x to obtain the feature map.

[0084]

[0085] (2) Feature aggregation: The upsampled feature map X' is combined with the feature map... By splicing the images together, we obtain the feature map.

[0086]

[0087] (3) Downsampling: The aggregated feature map Z is downsampled by a 3×3 convolutional layer with a stride of 2 to obtain the final detail-preserving feature map.

[0088]

[0089] Among them, W m,n,c′ These are the convolutional kernel weights, with a size of 3×3; b c′ This is the corresponding bias term.

[0090] Finally, the formula for calculating the detail-preserving feature map D can be summarized as follows:

[0091]

[0092] Next, Input is fed into the localization branch to infer the bounding box.

[0093] S6: Implement detection on the security check dataset.

[0094] Verification experiment:

[0095] Based on this method, experiments were conducted on the challenging SIXRAY and PIDRAY datasets to evaluate the effectiveness of the proposed detection method on different backbone networks on the PIDRAY and SIXRAY datasets. As shown in Tables 1 and 2, the method of this invention achieved excellent detection performance.

[0096] Table 1. Detection results of different backbone networks on SIXRAY.

[0097]

[0098] Table 2. Detection results of different backbone networks in PIDRAY

[0099]

[0100] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the present invention, and all such modifications or substitutions should be covered within the scope of the claims of the present invention.

Claims

1. A real-time security inspection method for detecting threatening items based on feature decoupling, characterized in that, The method specifically includes the following steps: S1: Construct a training set of image data of threatening items for security inspection; S2: Construct a neural network model for threat item detection, including a backbone network, a detection neck network, and a detection decoupling head network; The backbone network uses a feature pyramid for feature extraction to generate multi-scale feature maps to capture object information at different scales. The detection neck network utilizes a pyramid structure to fuse features of different scales extracted from the backbone network; The detection decoupling head network includes a classification branch and a localization branch. The classification branch uses semantic context encoding to enhance the attention to contextual features in the classification task, thereby improving the ability to extract key features of threatening items. The localization branch uses detail-preserving encoding to strengthen the detection head's attention to the outline, edges, and detailed texture of threatening items, ensuring the accuracy of the detection results. The specific structure of the semantic context encoding is as follows: at different levels of each threat item feature map, the semantic context encoding utilizes feature maps from two levels, namely F. l and F l+1 A classification feature map is generated through a concatenation operation; first, the feature map F is... l Downsampled by 2 times and compared with F l+1 The features are then concatenated to obtain the final feature map F. l cls : F l cls =Concat(DConv(F l ),F l+1 ) Where Concat represents the concatenation operation, DConv represents the downsampling convolution, and l represents the l-th layer; the concatenation operation and the depthwise separable convolution together constitute a shared downsampling convolution; Next, F l cls Input into the classification branch; due to F l cls It is F l A 2x downsampled version, therefore F l cls Each point at position (i,j) in the feature map will predict F. l The classification score of the four nearest neighbors is denoted as S. Where N is the number of categories; then, S is rearranged as follows: To restore resolution: The specific structure of the detail-preserving encoding is as follows: In each threat item feature layer level L, the detail-preserving encoding receives feature maps from three pyramid levels, namely F. l-1 F l and F l+1 ; F is fused using a simplified U-Net structure. l-1 and F l+1 First, put F l After upsampling by 2 times and F l-1 The feature aggregation is followed by downsampling through a 3×3 convolutional layer with a stride of 2; the entire process includes upsampling, feature aggregation, and downsampling; the final detail-preserving feature map F l loc , is represented as: F l loc =DConv(Concat(F l-1 ,UConv(F l ),F l+1 )) Where DConv represents downsampling convolution, UConv represents upsampling convolution, Concat represents concatenation operation, and l represents the l-th layer; Its specific implementation is as follows: (1) Upsampling: Upsampling the feature map Perform upsampling by 2x to obtain the feature map. Where i, j, and c represent the vertical position index, horizontal position index, and channel index of the feature map after upsampling, respectively, corresponding to the c-th channel of the feature map; H, W, and C1 represent the height, width, and number of channels of the feature map before upsampling, respectively. (2) Feature aggregation: The upsampled feature map X' is combined with the feature map... By splicing the images together, we obtain the feature map. Where C2 represents the number of channels in another feature map; (3) Downsampling: The aggregated feature map Z is downsampled by a 3×3 convolutional layer with a stride of 2 to obtain the final detail-preserving feature map. Among them, W m,n,c′ These are the convolutional kernel weights, with a size of 3×3; b c′ It is the corresponding bias term; Finally, the formula for calculating the detail-preserving feature map D is: Next, F l loc Input is fed into the localization branch to infer the bounding box; S3: Input the training set into the threat item detection neural network model to train the optimal network model; S4: Input the dataset to be inspected into the trained network model to perform detection and identify threatening items.

2. The real-time security inspection method for detecting threatening items according to claim 1, characterized in that, In step S2, the pyramid structure used for neck detection includes bottom-up and top-down feature extraction paths, as well as lateral connection modules. In the lateral connections, information from the feature maps is extracted from each layer of the backbone network and the number of channels is adjusted through convolution operations. The bottom-up path uses downsampling techniques to generate rich semantic representations from low-level feature maps. The top-down path uses upsampling and interpolation methods to obtain high-resolution features from high-level feature maps, thereby preserving more details. Finally, the feature maps from these two paths are element-wise added or concatenated with the feature maps obtained from the lateral connections to form a fused multi-scale feature representation.

3. The real-time security inspection method for detecting threatening items according to claim 1, characterized in that, In step S2, given the input feature map Depthwise separable convolution is divided into channel-wise convolution and point-wise convolution. The specific steps are as follows: 1) Channel-by-channel convolution: Each channel undergoes a k×k convolution operation separately, outputting a feature map. Where H, W, and C represent the height, width, and number of channels of the feature map, respectively; The formula for calculating channel-wise convolution is: Among them, W m,n,c b represents the convolution kernel weights. c denoted as the bias term, i and j represent the position indices of the output feature map, m and n represent the position indices of the convolution kernel, and c represents the c-th channel; 2) Pointwise convolution: X' is converted into the target number of channels C' through 1×1 convolution to obtain the final output.

Images