Real-time security check threat article detection method based on feature decoupling

Abstract

The invention relates to a real-time security check threat article detection method based on feature decoupling, and belongs to the technical field of image processing. According to the method, a threat item detection neural network model is used for detection, and the model comprises a backbone network, a detection neck network and a detection decoupling head network; the backbone network performs feature extraction by using a feature pyramid to generate a multi-scale feature map so as to capture object information under different scales; the detection of the neck network is to fuse different scale features extracted from the backbone network by using a pyramid structure; the detection decoupling head network comprises a classification branch and a positioning branch; semantic context coding is adopted in classification branches to enhance the attention on context features in classification tasks, so that the capability of extracting features of key parts of threat items is improved; the positioning branch adopts detail reservation coding to strengthen the attention of the detection head on the contour, edge and detail texture of a threat object and ensure the accuracy of a detection result.

Description

Technical Field

[0001] The invention belongs to the technical field of image processing and relates to a real-time security inspection threat object detection method based on feature decoupling. Background Art

[0002] Ensuring safe transportation on public transportation vehicles such as airports and subways is of great significance. However, in many cases, threat objects can only be discovered by inspectors standing next to X-ray machines, which requires them to maintain high concentration for a long time, which is not only difficult but also requires a lot of human resources. Therefore, in recent years, it has become increasingly important to use machines to automatically detect threat objects in X-ray images. Unlike natural images, X-ray images are usually pseudo-colored, and their generation depends on the penetration, photoelectric sensitivity, and fluorescence effect of X-rays. In addition, threat objects in X-ray images are often small and dense, and these targets are often highly overlapped, resulting in more background noise.

[0003] With the rapid development of deep learning, researchers have introduced a variety of network architectures in X-ray images for the classification and detection of threatening objects. The object detection task aims to identify and locate objects in natural images and is a basic task in computer vision. In recent years, deep learning-based methods have dominated the research progress of object detection, which is usually regarded as the classification of regions of interest in images and the localization of bounding boxes. The semantic context encoded in high-level semantic features plays a key role in object classification and localization. The feature pyramid network enhances the detection ability of small-scale objects by extracting high-level semantic information and fusing it into feature maps that preserve low-level details. This design provides richer semantic context for early features and supports the detection of multi-scale objects.

[0004] However, there are differences in the semantic requirements for classification and bounding box regression: more texture details and edge information are required to locate the bounding box, while classification requires rich semantic context. To address this problem, researchers proposed using two decoupled head branches on each layer of feature maps to perform classification and localization tasks respectively. This strategy is called parameter decoupling. Although this approach has theoretical advantages, we observe that it is still difficult to achieve an ideal balance between the two tasks by decoupling classification and localization only at the parameter level. Relying solely on the learning ability of the head network to extract task-specific context from the shared feature map often leads to a bias towards one task, thereby suppressing the performance of the other task. This shows that the current shared feature map is still optimized for these two competing tasks, and further research and improvement are needed. Summary of the invention

[0005] In view of this, an object of the present invention is to provide a real-time security threat object detection method based on feature decoupling to improve the accuracy of threat object detection.

[0006] In order to achieve the above object, the present invention provides the following technical solutions:

[0007] A real-time security threat object detection method based on feature decoupling specifically comprises the following steps:

[0008] S1: Construct a training set of security threat item image data;

[0009] S2: Construct a neural network model for threat object detection, including a backbone network, a detection neck network, and a detection decoupling head network;

[0010] The backbone network uses feature pyramids to extract features and generate multi-scale feature maps to capture object information at different scales;

[0011] The neck detection network uses a pyramid structure to fuse the different scale features extracted from the backbone network;

[0012] The detection decoupling head network includes a classification branch and a positioning branch; the classification branch uses semantic context encoding to enhance the focus on context features in the classification task, thereby improving the ability to extract key features of threat objects; the positioning branch uses detail-preserving encoding to enhance the detection head's focus on the contour, edge and detailed texture of threat objects, thereby ensuring the accuracy of the detection results;

[0013] S3: Input the training set into the threat object detection neural network model training to obtain the optimal network model;

[0014] S4: Input the data set to be inspected into the trained network model for detection and identification of threatening objects.

[0015] Furthermore, in step S2, the pyramid structure used in the neck detection network includes bottom-up and top-down feature extraction paths, as well as horizontally connected modules; in the horizontal connection, the information of the feature map is extracted from each layer of the backbone network, and the number of channels is adjusted through convolution operations; the bottom-up path generates rich semantic representations from low-level feature maps through downsampling technology; and the top-down path uses upsampling and interpolation methods to obtain high-resolution features from high-level feature maps, thereby retaining more details; finally, the feature maps of these two paths are element-wise added or concatenated with the feature maps obtained by the horizontal connection to form a fused multi-scale feature representation.

[0016] Further, in step S2, the specific structure of the semantic context encoding is: at different levels of each threat object feature map, the semantic context encoding uses feature maps from two levels, namely, F l and F l+1, a more semantically rich classification feature map is generated through concatenation operation; first, the feature map F l Downsample by a factor of 2 and compare it with F l+1 Splice to get the final feature map

[0017]

[0018] Among them, Concat represents the connection operation, DConv represents the downsampling convolution, and l represents the lth layer; the concatenation operation and the depth-wise separable convolution together constitute a shared downsampling convolution;

[0019] It should be noted that the processed feature map F l cls The resolution of the original feature map F l Next, Input into the classification branch; due to Yes F l is a 2x downsampled version of Each point at position (i, j) in the feature map predicts F l The classification scores of its four nearest neighbors are denoted as S. Where N is the number of categories; then, rearrange S into To restore the resolution:

[0020]

[0021] In this way, we not only exploit the sparsity of salient features (from F l ), but also from higher pyramid levels (from F l+1 This will help to more effectively infer object categories, especially for threatening objects with no texture or severe occlusion.

[0022] Further, in step S2, given the input feature map Depth-wise separable convolution is divided into channel-by-channel convolution and point-by-point convolution. The specific steps are as follows:

[0023] 1) Channel-by-channel convolution: Each channel performs a k×k convolution operation separately and outputs a feature map Where H, W, and C represent the height, width, and number of channels of the feature map, respectively;

[0024] The calculation formula for channel-by-channel convolution is:

[0025]

[0026] Among them, W m,n,c represents the convolution kernel weight, b crepresents the bias term, i and j represent the position index (two-dimensional) of the output feature map, m and n represent the position index (two-dimensional) of the convolution kernel, and c represents the c-th channel;

[0027] 2) Point-by-point convolution: Convert X' to the target channel number C' through 1×1 convolution to get the final output

[0028] Further, in step S2, the specific structure of the detail-preserving code is as follows: in each threat object feature map level L, the detail-preserving code accepts feature maps from three pyramid levels, namely, F l-1 、F l and F l+1 Among them, F l-1 Provides more details and edge features, while F l+1 Our experiments show that each security level is mainly related to its two adjacent levels, and the fusion of too many levels may have a negative impact on performance.

[0029] In order to improve computational efficiency, we use a simplified U-Net structure to fuse F l-1 and F l+1 ; First, F l After upsampling by 2 times, l-1 Aggregation, and then down-sampled through a 3×3 convolutional layer with a stride of 2; the whole part includes upsampling, feature aggregation and downsampling; the final detail-preserving feature map It is expressed as:

[0030]

[0031] Among them, DConv represents downsampling convolution, UConv represents upsampling convolution, Concat represents connection operation, l represents the lth layer,

[0032] The specific implementation is as follows:

[0033] (1) Upsampling: feature map Perform 2x upsampling to get the feature map

[0034]

[0035] Among them, i, j, and c represent the vertical position index, horizontal position index, and channel index of the upsampled feature map, respectively, corresponding to the cth channel of the feature map, H, W, C 1 They respectively represent the height, width and number of channels of the feature map before upsampling (the number of channels usually remains unchanged during the upsampling process);

[0036] (2) Feature aggregation: The upsampled feature map X' is combined with the feature map Splice to get the feature map

[0037]

[0038] Among them, C 2 Indicates the number of channels of another feature map;

[0039] (3) Downsampling: Downsample the aggregated feature map Z through a 3×3 convolutional layer with a stride of 2 to obtain the final detail-preserving feature map

[0040]

[0041] Among them, W m,n,c′ is the convolution kernel weight, size is 3×3; b c′ is the corresponding bias term;

[0042] Finally, the calculation formula of the detail-preserving feature map D can be summarized as:

[0043]

[0044] Next, Input to the localization branch to infer the bounding box.

[0045] The beneficial effects of the present invention are as follows: the present invention designs the backbone structure of a convolutional neural network, and uses a feature pyramid for feature extraction to generate a multi-scale feature map to capture object information at different scales. According to the different requirements of the classification task and the positioning task for feature maps in security threat object detection, a detection decoupling head is designed to optimize model performance. Semantic context encoding is designed to enhance the focus on context features in classification tasks, thereby improving the ability to extract key features of dangerous goods. Detail-preserving encoding is designed to strengthen the detection head's focus on the contours, edges and detailed textures of threat objects to ensure the accuracy of the detection results.

[0046] Other advantages, objectives and features of the present invention will be described in the following description to some extent, and to some extent, will be obvious to those skilled in the art based on the following examination and study, or can be taught from the practice of the present invention. The objectives and other advantages of the present invention can be realized and obtained through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] In order to make the purpose, technical solutions and advantages of the present invention more clear, the present invention will be described in detail below in conjunction with the accompanying drawings, wherein:

[0048] Figure 1It is the overall framework diagram of the threat object detection neural network model proposed by the present invention;

[0049] Figure 2 It is the semantic context encoding (CE) structure;

[0050] Figure 3 for detail-preserving encoding (DE) structures;

[0051] Figure 4 Comparison results of heat maps detected on the SIXRAY dataset. DETAILED DESCRIPTION

[0052] The following describes the embodiments of the present invention by specific examples, and those skilled in the art can easily understand other advantages and effects of the present invention from the contents disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and the details in this specification can also be modified or changed in various ways based on different viewpoints and applications without departing from the spirit of the present invention. It should be noted that the illustrations provided in the following embodiments only illustrate the basic concept of the present invention in a schematic manner, and the following embodiments and features in the embodiments can be combined with each other without conflict.

[0053] Among them, the drawings are only used for illustrative explanations, and they only represent schematic diagrams rather than actual pictures, and should not be understood as limitations on the present invention. In order to better illustrate the embodiments of the present invention, some parts of the drawings may be omitted, enlarged or reduced, and do not represent the size of actual products. For those skilled in the art, it is understandable that some well-known structures and their descriptions in the drawings may be omitted.

[0054] The same or similar numbers in the drawings of the embodiments of the present invention correspond to the same or similar parts; in the description of the present invention, it should be understood that if the terms "upper", "lower", "left", "right", "front", "rear", etc. indicate the orientation or position relationship, they are based on the orientation or position relationship shown in the drawings, which is only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operate in a specific orientation. Therefore, the terms describing the position relationship in the drawings are only used for illustrative purposes and cannot be understood as limiting the present invention. For ordinary technicians in this field, the specific meanings of the above terms can be understood according to specific circumstances.

[0055] See also Figure 1 to Figure 4 The embodiment of the present invention provides a real-time security threat object detection method based on feature decoupling, comprising the following steps:

[0056] S1: Construct a training set of security threat item image data to facilitate effective model learning.

[0057] The image datasets used for training are SIXRAY and PIDRAY datasets. The training batch size for each iteration is set to 32. Multi-scale training is performed on the two datasets, including image sizes ranging from 320 to 640 with 10 equidistant intervals. In order to enhance the representation ability of the model under limited instances, a combination of mixing, label smoothing, and random affine transformation is used. The size of the image during testing is 640.

[0058] S2: Design the backbone network of the convolutional neural network and the neck detection network, and use the feature pyramid for feature extraction to generate multi-scale feature maps to capture object information at different scales.

[0059] The convolutional neural network designed by the present invention is as follows Figure 1 As shown in the figure, the input image first extracts multi-scale features through the backbone network. In this network, the outputs of the 3rd, 4th, and 5th stages are represented as P3, P4, and P5, respectively, which can be used as the basis for subsequent feature fusion. In the feature pyramid of the neck detection network, the feature maps in P3, P4, and P5 are used as input to generate the output feature map in the pyramid structure for subsequent detection decoupling head network processing. Figure 1 In the figure, "Conv-n" represents the standard n×n convolution operation, "C" represents the concatenation operation, and "↑" represents the upsampling operation. The Composite Attention (CA) module is designed to enhance the ability of small target detection. In addition, the 3 Convolutions and Fusion (C3F) module is used to improve the detection efficiency of the model. The Composite Convolution (CC) module is a composite module derived from the original C3 (Convolutions) and convolution operations. P3, P4, and P5 represent the features extracted by the backbone network and are used for feature fusion.

[0060] The backbone network uses feature pyramids for feature extraction and generates multi-scale feature maps to capture object information at different scales.

[0061] The neck detection network effectively solves the multi-scale problem through a feature pyramid architecture. This architecture includes bottom-up and top-down feature extraction paths, as well as lateral connection modules. In the lateral connection, the information of the feature map is extracted from each layer of the backbone network, and the number of channels is adjusted through convolution operations. The bottom-up path generates rich semantic representations from low-level feature maps through downsampling technology, while the top-down path uses upsampling and interpolation methods to obtain high-resolution features from high-level feature maps, thereby retaining more details. Finally, the feature maps of these two paths are element-wise added or concatenated with the feature maps obtained by the lateral connection to form a fused multi-scale feature representation.

[0062] S3: According to the different requirements of classification task and localization task for feature maps in security threat object detection, a detection decoupling head network is designed to optimize model performance.

[0063] The detection decoupling head network consists of two branches, namely the localization (LOC) branch and the classification (CLS) branch.

[0064] Dangerous goods inspection is a vital safety measure aimed at detecting and locating potential threatening objects. In this process, classification and localization are two interrelated but conflicting tasks. Specifically, the classification task relies on richer contextual information to identify the object category, while the localization task requires precise local information (texture details and edge information) to determine the specific location of the object. l Based on the minimization of classification loss and positioning loss calculation. However, the semantic information of the feature graphs required for classification and positioning is different. In order to alleviate the conflict between classification and positioning, this paper proposes to decouple the feature encoding of these two tasks at the source, and use different feature encodings for different tasks using feature graphs with different contexts. l As a different unified input, the present invention provides task-specific input features for both branches, namely, F cls and F loc .

[0065] S4: Design semantic context encoding (CE) to enhance the focus on contextual features in classification tasks, thereby improving the ability to extract key features of dangerous goods.

[0066] Specifically, at different levels of each threat item feature map, semantic context encoding utilizes feature maps from two levels, namely, F l and F l+1 , a more semantically rich classification feature map is generated through concatenation. First, the feature map F l Downsample by a factor of 2 and compare it with F l+1 Splice to get the final feature map

[0067]

[0068] The concatenation operation and depth-wise separable convolution together form a shared downsampling convolution. Given an input feature map Depth-wise separable convolution is divided into channel-by-channel convolution and point-by-point convolution. The specific steps are as follows:

[0069] 1) Channel-by-channel convolution: Each channel performs a k×k convolution operation separately and outputs a feature map

[0070] The calculation formula for channel-by-channel convolution is:

[0071]

[0072] Among them, W m,n,c represents the convolution kernel weight, b c Represents the bias term.

[0073] 2) Point-by-point convolution: Convert X' to the target channel number C' through 1×1 convolution to get the final output

[0074] It should be noted that the processed feature map The resolution of the original feature map F l Next, Enter into the classification branch. Yes F l is a 2x downsampled version of Each point at position (i, j) in the feature map predicts F l The classification scores of its four nearest neighbors are denoted as S. Where N is the number of categories. Next, rearrange S into To restore the resolution:

[0075]

[0076] In this way, we not only exploit the sparsity of salient features (from F l ), but also from higher pyramid levels (from F l+1 This will help to more effectively infer object categories, especially for threatening objects with no texture or severe occlusion.

[0077] S5: Design detail preservation coding (DE) to enhance the detection head's attention to the contours, edges and detailed textures of threat objects to ensure the accuracy of detection results.

[0078] Unlike classification, localization is a more fine-grained task that relies on more texture details and boundary information to predict the location of objects. However, existing methods usually only regress the corners of the target from a single-scale feature map F. The low-level pyramid feature map is more responsive to the contours, edges, and detailed textures of the object, which helps improve the localization effect, but is often accompanied by huge computational costs. The computational cost of low-level feature maps is reduced by sparse convolution, but additional auxiliary branches and specific supervisory signals are still required. On the other hand, we note that high-level feature maps are equally important for localization because they provide as complete an observation of the target as possible, thus providing more information to infer the overall shape.

[0079] Based on these observations, the present invention proposes detail-preserving coding to achieve accurate localization without sacrificing efficiency. In each threat object feature map level L, detail-preserving coding accepts feature maps from three pyramid levels, namely, F l-1 、F l and F l+1 Among them, F l-1 Provides more details and edge features, while F l+1 This provides a more comprehensive view of items. This experiment shows that each security inspection level is mainly related to its two adjacent levels, and the fusion of too many levels may have a negative impact on performance.

[0080] In order to improve the computational efficiency, this paper adopts a simplified U-Net structure to fuse F l-1 and F l+1 First, F l After upsampling by 2 times, l-1 Aggregation, and then down-sampled through a 3×3 convolution layer with a stride of 2. The whole part includes upsampling, feature aggregation and downsampling. The final detail-preserving feature map It is expressed as:

[0081]

[0082] The specific implementation is as follows:

[0083] (1) Upsampling: feature map Perform 2x upsampling to get the feature map

[0084]

[0085] (2) Feature aggregation: The upsampled feature map X' is combined with the feature map Splice to get the feature map

[0086]

[0087] (3) Downsampling: Downsample the aggregated feature map Z through a 3×3 convolutional layer with a stride of 2 to obtain the final detail-preserving feature map

[0088]

[0089] Among them, W m,n,c′ is the convolution kernel weight, size is 3×3; b c′ is the corresponding bias term.

[0090] Finally, the calculation formula of the detail-preserving feature map D can be summarized as:

[0091]

[0092] Next, Input to the localization branch to infer the bounding box.

[0093] S6: Implement detection on the security inspection dataset.

[0094] Verification experiment:

[0095] Based on this method, experiments are conducted on the challenging SIXRAY and PIDRAY datasets to evaluate the effectiveness of the detection method proposed in the present invention on the PIDRAY and SIXRAY datasets with different backbone networks. As shown in Tables 1 and 2, the method of the present invention achieves excellent detection performance.

[0096] Table 1 Detection results of different backbone networks on SIXRAY

[0097]

[0098] Table 2 Detection results of different backbone networks in PIDRAY

[0099]

[0100] Finally, it should be noted that the above embodiments are only used to illustrate the technical solution of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solution of the present invention can be modified or replaced by equivalents without departing from the purpose and scope of the technical solution, which should be included in the scope of the claims of the present invention.

Claims

1. A real-time security threat object detection method based on feature decoupling, characterized in that: The method specifically comprises the following steps: S1: Construct a training set of security threat item image data; S2: Construct a neural network model for threat object detection, including a backbone network, a detection neck network, and a detection decoupling head network; The backbone network uses feature pyramids to extract features and generate multi-scale feature maps to capture object information at different scales; The neck detection network uses a pyramid structure to fuse the different scale features extracted from the backbone network; The detection decoupling head network includes a classification branch and a positioning branch; the classification branch uses semantic context encoding to enhance the focus on context features in the classification task, thereby improving the ability to extract key features of threat objects; the positioning branch uses detail-preserving encoding to enhance the detection head's focus on the contour, edge and detailed texture of threat objects, thereby ensuring the accuracy of the detection results; S3: Input the training set into the threat object detection neural network model training to obtain the optimal network model; S4: Input the data set to be inspected into the trained network model for detection and identification of threatening objects.

2. The real-time security threat object detection method according to claim 1, characterized in that: In step S2, the pyramid structure used to detect the neck includes bottom-up and top-down feature extraction paths, as well as horizontally connected modules; in the horizontal connection, the information of the feature map is extracted from each layer of the backbone network, and the number of channels is adjusted through convolution operations; the bottom-up path generates rich semantic representations from low-level feature maps through downsampling technology; and the top-down path uses upsampling and interpolation methods to obtain high-resolution features from high-level feature maps, thereby retaining more details; finally, the feature maps of these two paths are element-wise added or concatenated with the feature maps obtained by the horizontal connection to form a fused multi-scale feature representation.

3. The real-time security threat object detection method according to claim 1, characterized in that: In step S2, the specific structure of the semantic context coding is as follows: at different levels of each threat item feature map, the semantic context coding uses feature maps from two levels, namely, F and F, to generate a classification feature map through a concatenation operation; first, the feature map F is downsampled by 2 times l l+1l, and then compared with F l+1 Splice to get the final feature map F l cls : F l cls =Concat(DConv(F l ),F l+1 ) Among them, Concat represents the connection operation, DConv represents the downsampling convolution, and l represents the lth layer; the concatenation operation and the depth-wise separable convolution together constitute a shared downsampling convolution; Next, F l cls Input into the classification branch; since F l cls Yes F l is a 2x downsampled version of l cls Each point at position (i, j) in the feature map predicts F l The classification scores of its four nearest neighbors are denoted as S. Where N is the number of categories; then, rearrange S into To restore the resolution: <h2 style=";text-align:left;direction:ltr">S'[2i+a,2j+b,c] = S[i,j,(2a+b)c],<h2 style=";text-align:left;direction:ltr"> 4. The real-time security threat object detection method according to claim 3, characterized in that: In step S2, given the input feature map Depth-wise separable convolution is divided into channel-by-channel convolution and point-by-point convolution. The specific steps are as follows: 1) Channel-by-channel convolution: Each channel performs a k×k convolution operation separately and outputs a feature map Where H, W, and C represent the height, width, and number of channels of the feature map, respectively; The calculation formula for channel-by-channel convolution is: Among them, W m,n,c represents the convolution kernel weight, b c represents the bias term, i and j represent the position index of the output feature map, m and n represent the position index of the convolution kernel, and c represents the cth channel; 2) Point-by-point convolution: Convert X' to the target channel number C' through 1×1 convolution to get the final output 5. The real-time security threat object detection method according to claim 1, characterized in that: In step S2, the specific structure of the detail-preserving coding is as follows: in each threat object feature map level L, the detail-preserving coding accepts feature maps from three pyramid levels, namely, F, F, and F; l-1l l+1 A simplified U-Net structure is used to fuse F and F. First, F is upsampled by 2 times and then aggregated with F. Then, the l-1l+1l l-1 A 3×3 convolutional layer with a stride of 2 is used for downsampling; the whole part includes upsampling, feature aggregation and downsampling; the final detail-preserving feature map F l loc , expressed as: F l loc =DConv(Concat(F l-1 ,UConv(F l ),F l+1 )) Among them, DConv represents downsampling convolution, UConv represents upsampling convolution, Concat represents connection operation, and l represents the lth layer; The specific implementation is as follows: (1) Upsampling: feature map Perform 2x upsampling to get the feature map Among them, i, j, and c represent the vertical position index, horizontal position index, and channel index of the feature map after upsampling, respectively, corresponding to the cth channel of the feature map, and H, W, and C1 represent the height, width, and number of channels of the feature map before upsampling, respectively; (2) Feature aggregation: The upsampled feature map X' is combined with the feature map Splice to get the feature map Among them, C2 represents the number of channels of another feature map; (3) Downsampling: Downsample the aggregated feature map Z through a 3×3 convolutional layer with a stride of 2 to obtain the final detail-preserving feature map Among them, W m,n,c′ is the convolution kernel weight, size is 3×3; b c′ is the corresponding bias term; Finally, the calculation formula of the detail-preserving feature map D is: Next, F l loc Input to the localization branch to infer the bounding box.

Images