Millimeter wave human body security check image target detection method based on target context features
The method addresses the challenges of detecting small targets in millimeter wave images by using a millimeter wave image processing model with CBS modules and a feature integrated progressive fusion model, enhancing target detection accuracy through contextual feature extraction and fusion.
Patent Information
- Application Number
- CN202411256521.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-09-09
- Publication Date
- 2024-12-27
AI Technical Summary
Millimeter wave images used in human body security inspection systems suffer from large noise and low resolution, making it difficult to detect small and dark targets like knives and guns, especially due to the varied shapes, sizes, and materials of such targets.
A method involving a millimeter wave image processing model with CBS modules and context feature extractors, followed by a feature integrated progressive fusion model using PAFPN and AFPN networks, to enhance target feature extraction and fusion, incorporating batch normalization and attention mechanisms to improve detection accuracy.
The method effectively enhances the ability to identify targets of different scales by leveraging contextual information, improving detection accuracy and consistency through deep interaction between low-level detail and high-level semantic features.
Smart Images
Figure CN119206781A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of millimeter wave image processing, and in particular to a method for detecting a target in a millimeter wave human body security inspection image based on target context features. Background Art
[0002] The millimeter wave body security inspection system is an emerging technology. Millimeter wave image processing is not affected by clothing. Through millimeter wave image processing technology, the shape characteristics of the human body and the characteristics of items hidden under clothing can be obtained without causing harm to the human body. It is an important tool for security inspections in public places.
[0003] In recent years, a method of target detection and recognition based on millimeter wave images has been proposed based on deep learning, aiming to improve the automation level and efficiency of security inspection. However, millimeter wave images naturally have problems such as large noise and low resolution, which makes it particularly difficult to detect small and dark targets. In addition, there are many types of targets carried by the human body, such as knives, guns, explosives, etc. The shapes, sizes and materials of various types of targets are different, which increases the difficulty of detection. Therefore, it is of great significance to study an efficient and accurate method for target detection in millimeter wave human security inspection images.
[0004] Therefore, a target detection method for millimeter-wave human body security inspection images based on target context features was proposed. Summary of the invention
[0005] In order to solve the technical problems existing in the above-mentioned prior art, the present invention provides a millimeter wave human body security inspection image target detection method based on target context features.
[0006] To solve the above technical problems, the present invention provides the following technical solution: a method for detecting a target in a millimeter wave human body security inspection image based on target context features, comprising the following steps:
[0007] Step A, performing millimeter wave image processing on the person undergoing security inspection to obtain a millimeter wave image after imaging;
[0008] Step B, establishing a millimeter wave image processing model, extracting target image features in the millimeter wave image through the millimeter wave image processing model, and extracting target image context features around the target image features;
[0009] Step C, constructing a feature integrated progressive fusion model FIPC to fuse the target image features extracted from the millimeter wave image with the target image context features;
[0010] Step D, providing the fused image features for interpretation.
[0011] Preferably, the millimeter wave image processing model in step B includes a Backbone image processing model, and the obtained millimeter wave image is imported into the Backbone image processing model.
[0012] Preferably, the Backbone image processing model is composed of a plurality of CBS modules and a plurality of context feature extractors CFE, and the plurality of context feature extractors CFE are indirectly inserted into the plurality of CBS modules.
[0013] Preferably, the context feature extractor CFE includes a Split module, a plurality of context feature blocks CFB and a Concat module.
[0014] Preferably, the step B comprises:
[0015] Step B1, the CBS module extracts target image features in the millimeter wave image;
[0016] Step B2, the target image features are transmitted to the context feature extractor CFE module, and the Split module in the context feature extractor CFE module segments the received target image features, and the segmented target image features are respectively transmitted to multiple context feature blocks CFB;
[0017] Step B3, the context feature block CFB module preprocesses the received target image features through a 1x1 convolutional layer. After preprocessing, the number of channels of the target image features is reduced from the original 1×channel to channel / 2 without changing the width and height dimensions of the image;
[0018] Step B4, after preprocessing, the target image features are divided into three parallel branches for differential processing;
[0019] Step B5, the first branch uses a standard 3×3 convolution kernel to extract local features from the preprocessed target image features;
[0020] Step B6, the second and third branches evenly divide the preprocessed target image features into two parts in the channel dimension, and each part is processed by a 3×3 dilated convolution with different dilation rates to obtain multi-level context information of the preprocessed target image features;
[0021] Step B7, concatenate the local features in the preprocessed target image features output by the first, second and third branches and the multi-level context information of the preprocessed target image features along the channel dimension to obtain concatenated features, and restore the concatenated features to the initial 1×channel number of channels.
[0022] Preferably, the step B further comprises:
[0023] Step B8, introducing a batch normalization function BN into the concatenated features to normalize the concatenated features;
[0024] Step B9, introducing activation function PReLU into the concatenated features;
[0025] Step B10, adding a multi-fully connected branch attention module MFCA to the activated and normalized concatenated features, and performing weighted processing on all channels in the concatenated features.
[0026] Preferably, the feature integration progressive fusion model FIPC in step C includes a PAFPN network and an AFPN network, and the target image features and the target image context features are input into the feature integration progressive fusion model FIPC. The feature integration progressive fusion model FIPC fuses the target image features and the target image context features extracted from the millimeter wave image.
[0027] Preferably, the step C comprises:
[0028] Step C1, the Backbone image processing model hierarchically outputs feature maps, which are divided into P3, P4 and P5 according to the scale of the feature map output, with P3 as a low-level feature, P4 as a middle-level feature, and P5 as a high-level feature;
[0029] In step C2, the three-layer features of P5, P4 and P3 are respectively input into the PAFPN network. After P5, P4 and P3 are processed by PAFPN, they are recorded as P5', P4' and P3'.
[0030] Preferably, the step C further comprises:
[0031] Step C3, P5', P4', and P3' are input into the AFPN network, and the adjacent P5' features are fused with the P4' features to generate enhanced P5' and P4';
[0032] Step C4, P5", P4" and P3' are involved in the final fusion process.
[0033] Compared with the prior art, the beneficial effects of the present invention are:
[0034] In the present invention, multiple groups of context feature extractors CFE are designed. The context feature extractors CFE include a Split module, multiple context feature blocks CFB and a Concat module. The Split module of the context feature extractor CFE segments the target image features extracted by the CBS module before the context feature extractor CFE, the context feature block CFB module processes the segmented target image features, and the multiple context feature block CFB modules respectively output the processing results, thereby improving the ability to extract target features.
[0035] In the present invention, the comprehensive feature progressive fusion network FIPC is used in the feature fusion part to promote the deep interaction between low-level detail features and high-level semantic features, so that the contextual information in the image can be fully utilized. Through this comprehensive progressive fusion strategy, the model can more accurately identify target objects of different scales in the detection task. BRIEF DESCRIPTION OF THE DRAWINGS
[0036] Figure 1 It is a structural block diagram of the present invention;
[0037] Figure 2 This is a schematic diagram of the Backbone image processing model of the present invention;
[0038] Figure 3 It is a schematic diagram of the feature integrated progressive fusion model FIPC of the present invention;
[0039] Figure 4 A schematic diagram of the joint structure of the PAFPN network and the AFPN network of the present invention;
[0040] Figure 5 Schematic diagram of the structure of the context feature extractor CFE of the present invention;
[0041] Figure 6 For the present invention Figure 2 Schematic diagram of C2f structure;
[0042] Figure 7 For the present invention Figure 6 Schematic diagram of the Bottleneck structure. DETAILED DESCRIPTION
[0043] The present invention will be further described below with reference to the accompanying drawings and embodiments, but the following embodiments are merely preferred embodiments of the present invention and are not exhaustive.
[0044] Example:
[0045] like Figure 1-Figure 7 As shown, the present invention provides a method for detecting a target in a millimeter wave human body security inspection image based on target context features, comprising the following steps:
[0046] Step A, performing millimeter wave image processing on the person undergoing security inspection to obtain a millimeter wave image after imaging;
[0047] Step B, establishing a millimeter wave image processing model, extracting target image features in the millimeter wave image through the millimeter wave image processing model, and extracting target image context features around the target image features;
[0048] The millimeter wave image processing model includes a Backbone image processing model, and the obtained millimeter wave image is imported into the Backbone image processing model, wherein the Backbone image processing model is composed of a plurality of CBS modules and a plurality of context feature extractors CFE, and the plurality of context feature extractors CFE are indirectly inserted into the plurality of CBS modules;
[0049] The CBS module extracts target image features in the millimeter wave image, and the context feature extractor CFE extracts context features of the target image features;
[0050] The context feature extractor CFE includes a Split module, a plurality of context feature blocks CFB and a Concat module;
[0051] The Split module of the context feature extractor CFE segments the target image features extracted by the CBS module before the context feature extractor CFE, the context feature block CFB module processes the segmented target image features, multiple context feature block CFB modules respectively output processing results, the Concat module stacks the processing results output by multiple context feature block CFB modules, and the stacked results are output to the CBS module after the context feature extractor CFE;
[0052] Step B1, the target image features are input into the millimeter wave image processing model, and the CBS module extracts the target image features in the millimeter wave image;
[0053] Step B2, the target image features are transmitted to the context feature extractor CFE module, and the Split module in the context feature extractor CFE module segments the received target image features, and the segmented target image features are respectively transmitted to a plurality of context feature blocks CFB;
[0054] Step B3, the context feature block CFB module preprocesses the received target image features through a 1x1 convolution layer. After preprocessing, the number of channels of the target image features is reduced from the original 1×channel to channel / 2 without changing the width and height dimensions of the image, thereby reducing the complexity of image processing and making efficient use of the target image features.
[0055] Step B4, after preprocessing, the target image features are divided into three parallel branches for differential processing;
[0056] Step B5, the first branch uses a standard 3×3 convolution kernel to extract local features from the preprocessed target image features and capture the close association between pixels in the image;
[0057] Step B6: The second and third branches evenly divide the preprocessed target image features into two parts in the channel dimension, and process each part through 3×3 dilated convolutions with dilation rates of 2 and 3, respectively, to simulate receptive fields of two different scales, thereby capturing more extensive and multi-level context information.
[0058] Step B7, concatenating the local features in the preprocessed target image features output by the first, second and third branches and the multi-level context information of the preprocessed target image features along the channel dimension to obtain concatenated features, and restoring the concatenated features to the initial 1×channel number of channels;
[0059] Step B8, introducing a batch normalization function BN into the concatenated features, normalizing the concatenated features to a distribution with a mean of 0 and a variance of 1, thereby alleviating the gradient vanishing or gradient exploding problem in the deep network;
[0060] Step B9, introducing an activation function PReLU into the concatenated features. The activation function PReLU allows to output a slope controlled by a parameter and less than 0 multiplied by the input signal when the input signal is less than 0, so as to solve the problem of neuron inactivation caused by ReLU outputting 0 when the input signal is less than 0, as well as the problem of gradient disappearance and unstable model training caused by this.
[0061] Step B10, in order to simulate the attention allocation mechanism of the human visual system when processing complex scenes, a multi-fully connected branch attention module MFCA is added to the spliced features after activation and normalization, and all channels in the spliced features are weighted. The priorities of different channels are automatically adjusted to enhance the contribution of feature channels rich in contextual feature information, so that the constructed feature comprehensive progressive fusion model FIPC can focus more on key information in subsequent fusion, thereby improving the overall performance of the millimeter wave image processing model.
[0062] Step C, constructing a feature integrated progressive fusion model FIPC, wherein the feature integrated progressive fusion model FIPC includes a PAFPN network and an AFPN network;
[0063] The PAFPN network is a dual-path processing mechanism. Path 1 is top-down. Through the lateral connection strategy, the upsampled version of the high-level feature map is fused with the low-level feature map to strengthen the semantic understanding of the low-level feature map. Path 2 is bottom-up, starting from the lowest-level feature map, and gradually moving upward to fuse the feature map of the current layer with that of the previous layer until the highest layer, so that the precise positioning information in the low-level features can be effectively transmitted to the high-level features, enhancing the detail expression of the high-level features. This fusion strategy makes each layer of feature maps rich in cross-level comprehensive information, realizing the deep integration of contextual features.
[0064] The target image features and the target image context features are input into the feature integrated progressive fusion model FIPC, and the feature integrated progressive fusion model FIPC fuses the target image features and the target image context features extracted from the millimeter wave image;
[0065] Step C1, the Backbone image processing model hierarchically outputs feature maps, which are divided into P3, P4 and P5 according to the scale of the feature map output, with P3 as a low-level feature, P4 as a middle-level feature, and P5 as a high-level feature;
[0066] Step C2, the three-layer features of P5, P4, and P3 are respectively input into the PAFPN network, and P5, P4, and P3 processed by PAFPN are recorded as P5', P4', and P3';
[0067] Step C3, P5', P4', and P3' are input into the AFPN network, and the adjacent P5' features are fused with P4' features by ASFF to generate enhanced P5' and P4'. ASFF fusion will learn a set of weights for each spatial position on feature maps of different levels. These weights determine the contribution of features from different levels at the position. In this way, ASFF can adaptively adjust the weights of feature fusion according to targets of different scales, thereby achieving better feature fusion effects.
[0068] Step C4, P5", P4" and P3' are fused by ASFF, and three-way optimized features are output, and the optimized features are used for detection;
[0069] The progressive fusion mechanism of AFPN effectively shortens the semantic gap between features at different levels, allowing features at each level to gradually converge during the fusion process, improving the consistency and effectiveness of the overall features. The feature comprehensive progressive fusion model FIPC not only inherits the flexibility of PAFPN in the feature fusion path, but also further optimizes the flow and integration of feature information through the progressive strategy of AFPN, ensuring that features at each level contain rich contextual information, providing strong support for subsequent target detection tasks;
[0070] Step D, providing the fused image features for interpretation.
[0071] The above description is only a preferred embodiment of the present invention, which is only illustrative and not restrictive of the present invention. Those skilled in the art understand that many changes, modifications, and even equivalences may be made to the present invention within the spirit and scope defined by the claims of the present invention, but all of them will fall within the scope of protection of the present invention.
Claims
1. A millimeter wave human body security inspection image target detection method based on target context features, characterized in that: The following steps are included: Step A, performing millimeter wave image processing on the person undergoing security inspection to obtain a millimeter wave image after imaging; Step B, establishing a millimeter wave image processing model, extracting target image features in the millimeter wave image through the millimeter wave image processing model, and extracting target image context features around the target image features; Step C, constructing a feature integrated progressive fusion model FIPC to fuse the target image features extracted from the millimeter wave image with the target image context features; Step D, providing the fused image features for interpretation.
2. The method for detecting targets in millimeter wave human body security inspection images based on target context features as claimed in claim 1, characterized in that: The millimeter wave image processing model in step B includes a Backbone image processing model, and the obtained millimeter wave image is imported into the Backbone image processing model.
3. The method for detecting target in millimeter wave human body security inspection images based on target context features as claimed in claim 2, characterized in that: The Backbone image processing model is composed of multiple CBS modules and multiple context feature extractors CFE, and the multiple context feature extractors CFE are indirectly inserted into the multiple CBS modules.
4. The method for detecting target in millimeter wave human body security inspection images based on target context features as claimed in claim 3, characterized in that: The context feature extractor CFE includes a Split module, a plurality of context feature blocks CFB and a Concat module.
5. The method for detecting target in millimeter wave human body security inspection images based on target context features as claimed in claim 1, characterized in that: The step B comprises: Step B1, the CBS module extracts target image features in the millimeter wave image; Step B2, the target image features are transmitted to a context feature extractor CFE module, and a Split module in the context feature extractor CFE module segments the received target image features, and the segmented target image features are respectively transmitted to a plurality of context feature blocks CFB; Step B3, the context feature block CFB module preprocesses the received target image features through a 1x1 convolution layer. After the preprocessing, the number of channels of the target image features is reduced from the original 1×channel to channel / 2 without changing the width and height dimensions of the image; Step B4, after preprocessing, the target image features are divided into three parallel branches for differential processing; Step B5, the first branch uses a standard 3×3 convolution kernel to extract local features from the preprocessed target image features; Step B6, the second and third branches evenly divide the preprocessed target image features into two parts in the channel dimension, and each part is processed by a 3×3 dilated convolution with different dilation rates to obtain multi-level context information of the preprocessed target image features; Step B7, concatenate the local features in the preprocessed target image features output by the first, second and third branches and the multi-level context information of the preprocessed target image features along the channel dimension to obtain concatenated features, and restore the concatenated features to the initial 1×channel number of channels.
6. The method for detecting target in millimeter wave human body security inspection images based on target context features as claimed in claim 1, characterized in that: The step B also includes, Step B8, introducing a batch normalization function BN into the concatenated features to normalize the concatenated features; Step B9, introducing activation function PReLU into the concatenated features; Step B10, adding a multi-fully connected branch attention module MFCA to the activated and normalized concatenated features, and performing weighted processing on all channels in the concatenated features.
7. The method for detecting target in millimeter wave human body security inspection images based on target context features as claimed in claim 1, characterized in that: The feature integrated progressive fusion model FIPC in step C includes a PAFPN network and an AFPN network. The target image features and target image context features are input into the feature integrated progressive fusion model FIPC. The feature integrated progressive fusion model FIPC fuses the target image features and target image context features extracted from the millimeter wave image.
8. The method for detecting targets in millimeter wave human body security inspection images based on target context features as claimed in claim 7, characterized in that: The step C comprises: Step C1, the Backbone image processing model hierarchically outputs feature maps, which are divided into P3, P4 and P5 according to the scale of the feature map output, with P3 as a low-level feature, P4 as a middle-level feature, and P5 as a high-level feature; In step C2, the three-layer features of P5, P4 and P3 are respectively input into the PAFPN network. After P5, P4 and P3 are processed by PAFPN, they are recorded as P5', P4' and P3'.
9. The method for detecting targets in millimeter wave human body security inspection images based on target context features as claimed in claim 7, characterized in that: The step C also includes, Step C3, P5', P4', and P3' are input into the AFPN network, and the adjacent P5' features are fused with the P4' features to generate enhanced P5' and P4'; Step C4, P5", P4" and P3' are involved in the final fusion process.