Terahertz target detection model training method using novel attention mechanism
By embedding an adaptive context-aware attention network and wavelet transform into the YOLOv8 framework, the problems of low detection efficiency and insufficient accuracy in terahertz security inspection equipment are solved, and efficient and accurate identification of concealed targets is achieved.
Patent Information
- Application Number
- CN202411137470.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-19
- Publication Date
- 2024-11-26
- Estimated Expiration
- 2044-08-19
AI Technical Summary
Existing terahertz security inspection equipment cannot effectively detect non-metallic dangerous items, and traditional target detection algorithms suffer from low detection efficiency, inaccurate recognition, and inability to capture the relationships between objects in a global view in terahertz images.
An adaptive context-aware attention network is embedded in the YOLOv8 object detection framework. Image features are enhanced through wavelet transform, and a self-attention mechanism and an improved loss function are introduced to establish a global context-aware attention mechanism, thereby improving feature representation and localization accuracy.
It improves the real-time and accurate positioning and recognition capabilities of concealed targets in terahertz security inspection images, enhances target feature representation, suppresses irrelevant information, and improves pixel-level recognition rate and detection accuracy.
Smart Images

Figure CN119027770A_ABST
Abstract
Description
Technical Field
[0001] The present invention belongs to the field of terahertz science and technology, and in particular relates to a terahertz target detection model training method using a novel attention mechanism. Background Art
[0002] Traditional security inspection facilities include X-ray security inspection machines, security gates, handheld metal object detectors, etc., which require manual assistance to complete the inspection, are very inefficient, and can only detect dangerous items in backpacks and metal objects carried by the human body. X-rays will also cause certain radiation to the human body. In addition, these security inspection equipment cannot effectively detect non-metallic dangerous items such as chemical agents and ceramic knives, and also bring high labor costs and increase travel time.
[0003] Terahertz waves are between millimeter waves and far infrared in the electromagnetic spectrum, with a frequency range of 0.1THz~10THz (wavelength 3mm~30μm). Electromagnetic waves in this frequency band have both the penetrability of microwaves and millimeter waves and the collimation of light waves, and the spectrum resources are extremely rich. Since the terahertz band is difficult to detect by conventional electronic and optical methods, it is called the last blank of the electromagnetic spectrum, that is, the terahertz gap. In the past two or three decades, radar imaging technology based on terahertz waves has achieved rapid development. In particular, it has excellent performance in the field of security inspection equipment: (1) Low radiation: terahertz waves have low radiation energy. When biological tissues are irradiated by terahertz beams, they will not produce the photoionization effect of high-energy rays. Therefore, they are suitable for security inspections in crowded public places; (2) Good penetration: terahertz waves can penetrate non-polar materials such as cardboard, clothing, plaster, foam, etc., and the loss is relatively small, which can effectively meet the object security inspection needs in most scenarios; (3) High resolution: Compared with the microwave band, terahertz waves are in a higher frequency band, so the wavelength is shorter. Combined with its good broadband characteristics, higher imaging resolution can be obtained, which is conducive to improving the accuracy of target recognition. Therefore, terahertz security inspection systems have been widely used in security inspections in various scenarios.
[0004] According to whether it is region-based or not, current object detection algorithms are mainly divided into anchor-based detectors and anchor-free detectors. Among them, the anchor-free algorithm does not rely on the region proposal generation network, but directly extracts the location and features of the target from the image, usually through dense sampling or adaptive methods to locate the target, and adopts an end-to-end training method, making the model simpler and more efficient, and achieving faster reasoning speed when detecting targets.
[0005] In addition, the attention mechanism has been shown to be helpful for various computer vision tasks. It can help the model selectively focus on important areas or features in the image, and enhance the feature representation of the target area and suppress the interference of irrelevant features by adjusting the weights of different channels or spatial positions in the feature map, thereby improving the model's perception ability.
[0006] With the development of deep learning, methods based on convolutional neural networks (CNNs) have improved the accuracy and speed of target detection in THz images, but the following technical defects still exist: (1) Unlike natural images, original THz security images have shortcomings such as insufficient texture information, background image noise, incomplete imaging of hidden targets, low contrast, and variable shapes of hidden objects, which pose challenges to the detection and identification of suspicious targets hidden in clothing; (2) Although convolutional neural networks can extract high-level semantic information of images, directly transplanting the detection algorithm used for natural images to THz security images will lead to serious detection performance degradation and cannot effectively detect and identify suspicious targets; (3) Existing hidden target detection algorithms still have problems with low detection efficiency and inaccurate recognition; (4) The previous attention mechanism using multi-scale context fusion cannot capture the relationship between hidden objects in the global view, nor can it adaptively perceive global context dependencies. In addition, recursive neural networks have the problem of relying too much on prior knowledge. Summary of the invention
[0007] To solve the above technical problems, the present invention provides a terahertz target detection model training method using a new attention mechanism. Based on the most advanced target detection framework YOLOv8, the proposed adaptive context-aware attention network is embedded. This network solves the problem that multi-scale context fusion cannot capture the relationship between objects in the global view in previous work, and also solves the problem that recursive neural networks rely too much on prior knowledge.
[0008] To achieve the above purpose, the technical solution adopted by the present invention is as follows: A terahertz target detection model training method using a novel attention mechanism, the method comprising the following steps: Step 1, obtaining a terahertz image dataset; Step 2: Decompose the terahertz image in the horizontal, vertical and diagonal directions using the wavelet transform method to obtain the high-frequency components in the corresponding directions, reconstruct the image according to the wavelet coefficients, and obtain an enhanced image; Step 3: input the enhanced image into a backbone feature extraction network to obtain a multi-scale feature map; Step 4: Input the multi-scale feature map into the adaptive context-aware attention network to obtain an enhanced context-related attention feature map processed by the adaptive context-aware attention network; Step 5: Input the enhanced context-associated attention feature map processed by the adaptive context-aware attention network into the feature fusion module for upsampling and downsampling, and then perform a connection operation to obtain a multi-scale fusion feature map; Step 6: Input the multi-scale fusion feature map into the decoupling prediction module to obtain a prediction result, train the target detection network model, calculate the loss value through the loss function based on the prediction result and the annotation information of the terahertz image, use the loss value to calculate the gradient value of back propagation through the optimizer, and use the gradient value to update the target detection network parameters.
[0009] The beneficial effects of the present invention are: 1) By establishing a global context-aware attention mechanism model from the spatial and channel dimensions, the semantic association of the image context can be established, and the guidance of the attention map rich in semantic information can be obtained. Then, similar features of any scale can be adaptively integrated from a global perspective, and features can be enhanced by perceiving and integrating global context-weighted features, so that related feature points can be adaptively fused together. This efficient and multi-dimensional context feature information fusion enhances the target feature representation, suppresses information irrelevant to the target feature, and effectively simulates context dependency, thereby improving the pixel-level recognition rate.
[0010] 2) The present invention introduces a self-attention mechanism to capture the spatial dependency between any two positions in the feature map. For a feature at a certain position, it is updated by weighted summing the features at all positions, where the weight is determined by the feature correlation between the two corresponding positions. In other words, any two positions with associated features can promote each other's improvement, no matter how far apart they are in the spatial dimension.
[0011] 3) The present invention introduces an improved loss function, replaces the binary cross entropy function of the classification loss term, and increases the modulation factor; the modulation coefficient is added to the IoU loss term to increase the proportion of the center point distance in training and improve the positioning accuracy. The experimental results show that the present invention can accurately locate and identify hidden targets in terahertz security inspection images in real time, and has important application value in terahertz intelligent security inspection systems. BRIEF DESCRIPTION OF THE DRAWINGS
[0012] Figure 1 This is a flow chart of a terahertz target detection model training method using a novel attention mechanism of the present invention; Figure 2 The structure diagram of the adaptive context-aware attention network; Figure 3 It is a schematic diagram of each component image after wavelet transform decomposition; Figure 4Schematic diagram of enhanced image reconstructed according to wavelet coefficients; Figure 5 A schematic diagram of the test results. DETAILED DESCRIPTION
[0013] The present invention will be further described below in conjunction with the accompanying drawings and embodiments.
[0014] like Figure 1 As shown, it is a flow chart of a terahertz target detection model training method using a novel attention mechanism of the present invention, which specifically includes the following steps: Step 1: Acquire a terahertz image dataset; the terahertz image dataset includes an image and image annotation information; the annotation information includes location information, size information, quantity information and category information of a hidden target in the terahertz image.
[0015] Step 2: Perform image enhancement preprocessing on each terahertz image in the terahertz image dataset using wavelet transform to obtain an enhanced image. The terahertz image has low resolution and insufficient texture information. It contains background image noise, incomplete imaging of hidden targets, and low contrast. Therefore, the original image is enhanced before inputting into the detection network. The original image is decomposed in the horizontal, vertical, and diagonal directions using wavelet transform to obtain high-frequency components in the corresponding directions (horizontal, vertical, and diagonal directions from left to right), such as Figure 3 As shown, the image is then reconstructed according to the wavelet coefficients to obtain an enhanced image, as shown in Figure 4 shown.
[0016] Step 3: Set the size to The enhanced image is first uniformly adjusted to , input into the backbone feature extraction network, through the convolution kernel , the stride is The convolution operation is performed down-sampling operation to obtain feature maps of three different scales, namely multi-scale feature maps; Step 4: Input the multi-scale feature map into the adaptive context-aware attention network to obtain an enhanced context-related attention feature map (hereinafter referred to as the enhanced attention feature map) processed by the adaptive context-aware attention network. Figure 2 As shown in FIG. 1 , the adaptive context-aware attention network structure diagram includes a spatial attention submodule and a channel attention submodule. The spatial attention submodule is as follows: Figure 2 As shown in the upper part, assuming that the input multi-scale feature map is ,in Represents the number of channels of the multi-scale feature map, represents the height of the multi-scale feature map, Represents the width of the multi-scale feature map. First, the convolution kernels are The first and second convolutional layers of The first output feature map and size are also The second output feature map of , and then reshape them into , where the intermediate parameter Then, the first output feature map is applied with softmax to calculate the weight vector, which carries the spatial global context attention information. The weight vector is matrix multiplied with the second output feature map to adaptively generate the feature relationship dependency matrix between pixels. , that is, its size is At the same time, the multi-scale feature map The size of , and then compare it with the feature relationship dependency matrix Perform matrix multiplication to obtain the final spatial contextual attention feature map . The process can be expressed as: (1) in, and They represent the convolution kernels in the spatial attention submodule respectively. The weights of the first and second convolutional layers, spatial contextual attention feature map Contains the first output feature map The pixel position and the second output feature map The interdependence of pixel positions, represents the matrix multiplication operation, and They represent the first output feature map The pixel position and the second output feature map pixel positions, Represents all pixels.
[0017] Furthermore, the spatial context is associated with the attention feature map Input the convolution kernels in sequence as The third convolutional layer and Bachnorm layer (BN) are further sampled to extract high-level features, and the scale of the spatial branch output is adjusted. Finally, the ReLU activation function is used to activate the final output spatial attention feature map. The whole process can be expressed as: (2) in, Represents multi-scale feature maps The spatial attention applied, Conv represents a regular convolution operation.
[0018] The channel attention submodule is as follows Figure 2 As shown in the lower part, since each high-level feature map extracted by the backbone network contains specific channel feature responses, and the semantic responses between any two channels are related to each other, the feature dependencies between different channels in the channel branch are used to model the channel global contextual association information features. Assume that the input multi-scale feature map is , in order to summarize the feature maps of each channel, the input multi-scale feature map Perform global average pooling (GAP) processing to generate a channel feature relationship dependency vector, that is, a channel context-related attention feature map, whose size is , this vector soft-encodes the global contextual information in each channel. In order to obtain the attention between channels from the channel feature relationship dependency vector, the channel feature relationship dependency vector is input into the convolution kernel as The fourth convolutional layer downsamples the channel dimension to obtain a size of The feature map of is the reduction rate, and then inputs the BachNorm layer (BN), adjusts the ratio of the channel branch output, continues to use the ReLU activation function to activate, and finally inputs the convolution kernel as The fifth convolutional layer restores the channel context-related attention feature map to The size of the channel attention feature map is obtained and then expanded it to The size of the input multi-scale feature map Perform matrix multiplication operation, the process can be expressed as: (3) in, Represents a multi-scale feature map of the input The channel attention applied, and They represent the convolution kernels respectively. The weights of the fourth and fifth convolutional layers of , and AvgPool represents the average pooling operation.
[0019] Furthermore, the spatial attention feature map is obtained from the two attention branches And channel attention feature map After that, they are merged to obtain the enhanced contextual attention correlation feature map Among various combination methods such as element summation, element multiplication and maximum operation, element summation is selected to obtain efficient gradient flow. It can be expressed as: (4) Step 5: Input the enhanced contextual attention associated feature map processed by the adaptive context-aware attention module into the feature fusion module and perform a connection operation to obtain a multi-scale fusion feature map.
[0020] Step 6. Input the multi-scale fusion feature map into the decoupled prediction module to obtain the prediction result, and train the target detection network model based on the prediction result and the annotation information of the sample image. The prediction result and the annotation information are used to calculate the error, that is, the loss value, through the loss function, and the loss value is further used to calculate the gradient value of back propagation through the optimizer, and the gradient value is used to update the target detection network parameters; the loss function includes the complete intersection-over-union loss (CIoU), confidence loss and classification loss. The CIoU loss consists of overlapping area, center point distance and aspect ratio, and is used for fitting positioning; the confidence loss is used to determine whether the target is contained; the classification loss is used to fit the object category information; wherein, the classification loss function is expressed as follows: (5) in, represents the classification loss, represents the sample value, is the modulation factor, usually taken as , and used the binary cross entropy loss, Indicates the number of grids into which the input image is divided. Indicates that there is an object in the nth predicted bounding box in the mth grid. t indicates the category to which the object belongs, and classes indicates all categories. Represents the predicted result value; The complete intersection-over-union loss (CIoU) It is expressed as follows: (6) in, Represents the distance between the predicted center point and the true center point of the target, represents the center point of the predicted bounding box, represents the center point of the ground-truth bounding box, and are the width and height of the predicted bounding box, respectively. and are the width and height of the true bounding box, respectively. IoU represents the intersection over union of the predicted bounding box and the true bounding box. c represents the diagonal distance of the minimum closure area that can contain both the predicted bounding box and the true bounding box. v represents the trade-off parameter. is the modulation coefficient, which is generally . The modulation coefficient Multiplying the aspect ratio term can increase the ratio of the center point distance during training and improve positioning accuracy.
[0021] The confidence loss It is expressed as follows: (7) In the formula, Indicates that there is an object in the nth predicted bounding box in the mth grid, Indicates that there is no object in the nth predicted bounding box in the mth grid. represents the confidence of the anchor box corresponding to the mth grid, represents the sample label value of the mth grid, and M represents the number of predicted bounding boxes.
[0022] Furthermore, the total loss function Loss can be expressed as: (8) After obtaining the loss value through the loss function Loss, the network starts backpropagation to update the network's weights and bias parameters. The network training is repeated in this way until the loss value reaches the minimum and the network model parameters are saved.
[0023] Example In this example, through the training method of the target detection model using the adaptive context-aware attention mechanism, the network can learn to use the global information of the terahertz image to selectively emphasize the hidden target information features and suppress features unrelated to the target, effectively improving the accuracy of target detection in terahertz images.
[0024] Step 1: Acquire a terahertz image dataset: the terahertz image dataset includes an image and image annotation information; the annotation information includes location information, size information, quantity information, and category information of a hidden target in the terahertz image.
[0025] Step 2: Image enhancement: Use wavelet transform to decompose the original image in the horizontal, vertical and diagonal directions to obtain the high-frequency components in the corresponding directions. Finally, reconstruct the image according to the wavelet coefficients to obtain an enhanced image.
[0026] Step 3: Input the enhanced image into the backbone feature extraction network to obtain a multi-scale feature map. The backbone feature extraction network includes four alternately connected CSPLayer_2Conv and five convolutional layers and a spatial pyramid pooling module (SPPF), where the convolution kernel size of the convolutional layer is , the convolution stride is , and performs padding operations; CSPLayer_2Conv transforms the input feature map through the first convolutional layer, and the output feature map is divided into two parts. The two parts are processed by different convolutional layers, activated by the SiLU activation function, and then merged. This process helps the model capture more contextual information, so as to more accurately identify the target; the merged feature map is then processed by the second convolutional layer and finally output; the SPPF module samples the input feature map through three maximum pooling layers respectively, and then connects the output. The CSPdarknet53 downsamples the input enhanced image, and inputs the three different scale feature maps obtained by the CSPLayer_2Conv module and the SPPF module into the adaptive context-aware attention module to capture the feature dependencies of the spatial dimension and the channel dimension.
[0027] Step 4: Input the multi-scale feature map into the adaptive context-aware attention network to obtain an enhanced context-related attention feature map processed by the adaptive context-aware attention network.
[0028] Step 5: Input the context-associated attention feature map processed by the adaptive context-aware attention network into the feature fusion module for upsampling and downsampling, and then perform a connection operation to obtain a multi-scale fusion feature map.
[0029] Step 6: Input the multi-scale fusion feature map into the decoupled prediction module to obtain the prediction result, and train the target detection network model based on the prediction result and the annotation information of the sample image. Calculate the error, i.e., the loss value, through the loss function using the prediction result and the annotation information, and further use the loss value to calculate the gradient value of back propagation through the optimizer, and use the gradient value to update the target detection network parameters.
[0030] Finally, the trained target detection network model is used for detection. The detection results are as follows: Figure 5 As shown in the figure, from left to right, it respectively represents the situations where a human body carries a single target, two targets, and three targets. The objects carried include a pistol, a water bottle, a wallet, a ceramic knife, etc. (in the sample figure, WB represents a water bottle, GA represents a pistol model, KK represents a kitchen knife, SS represents scissors, LW represents a leather wallet, MD represents a metal dagger, and CK represents a ceramic knife). It can be seen intuitively that the Adaptation-YOLO method adopted in the present invention realizes accurate classification and high detection accuracy of hidden objects carried by the human body.
[0031] The specific embodiments described above further illustrate the objectives, technical solutions and beneficial effects of the present invention in detail. It should be understood that the above description is only a specific embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.
Claims
1. A terahertz target detection model training method using a novel attention mechanism, It is characterized in that The method comprises the following steps: Step 1, obtaining a terahertz image dataset; Step 2: Decompose the terahertz image in the horizontal, vertical and diagonal directions using the wavelet transform method to obtain the high-frequency components in the corresponding directions, reconstruct the image according to the wavelet coefficients, and obtain an enhanced image; Step 3: input the enhanced image into a backbone feature extraction network to obtain a multi-scale feature map; Step 4: Input the multi-scale feature map into the adaptive context-aware attention network to obtain an enhanced context-related attention feature map processed by the adaptive context-aware attention network; Step 5: Input the enhanced context-associated attention feature map processed by the adaptive context-aware attention network into the feature fusion module for upsampling and downsampling, and then perform a connection operation to obtain a multi-scale fusion feature map; Step 6: Input the multi-scale fusion feature map into the decoupling prediction module to obtain a prediction result, train the target detection network model, calculate the loss value through the loss function based on the prediction result and the annotation information of the terahertz image, use the loss value to calculate the gradient value of back propagation through the optimizer, and use the gradient value to update the target detection network parameters.
2. A terahertz target detection model training method using a novel attention mechanism according to claim 1, It is characterized in that The terahertz image data set includes an image and image annotation information, and the annotation information includes position information, size information, quantity information and category information of a hidden target in the terahertz image.
3. According to claim 1, a terahertz target detection model training method using a novel attention mechanism, It is characterized in that In step 4, the adaptive context-aware attention network includes a spatial attention submodule and a channel attention submodule, and the spatial attention feature map is obtained from the spatial attention submodule and the channel attention submodule respectively. And channel attention feature map Then merge them to get the enhanced context attention correlation feature map : (4)。 4. A terahertz target detection model training method using a novel attention mechanism according to claim 3, It is characterized in that The spatial attention feature map Obtained through the following steps: Assume that the input multi-scale feature map is ,in Represents the number of channels of the multi-scale feature map, represents the height of the multi-scale feature map, Represents the width of the multi-scale feature map, which is respectively passed through the convolution kernel as The first and second convolutional layers of The first output feature map and size are also The second output feature map of , and then reshape the first output feature map and the second output feature map into , where the intermediate parameter ; Apply softmax to the first output feature map to calculate the weight vector, and perform matrix multiplication with the second output feature map to generate the feature relationship dependency matrix between pixels. ; Multi-scale feature map The size of Then, the feature relationship dependency matrix Perform matrix multiplication to obtain the final spatial contextual attention feature map : (1) in, and They represent the convolution kernels in the spatial attention submodule respectively. The weights of the first and second convolutional layers, represents the matrix multiplication operation, and They represent the first output feature map The pixel position and the second output feature map pixel positions, Represents all pixels; Associating spatial context with attention feature map Input the convolution kernels in sequence: The third convolutional layer and BN layer extract high-level features and use the ReLU activation function to activate and obtain the final spatial attention feature map : (2) in, Represents multi-scale feature maps The spatial attention applied, Conv represents the convolution operation, ReLU represents the activation function, and BN represents the normalization layer.
5. A terahertz target detection model training method using a novel attention mechanism according to claim 3, It is characterized in that The channel attention feature map Obtained through the following steps: Assume that the input multi-scale feature map is , for the multi-scale feature map of the input Perform global average pooling to generate channel context-related attention feature maps, which are input to the convolution kernel as follows: The fourth convolutional layer downsamples the channel dimension and then inputs the BN layer. After activation using the ReLU activation function, the input convolution kernel is The fifth convolutional layer restores the channel context-related attention feature map to The size of the channel attention feature map is obtained , expand the size to After that, with the input multi-scale feature map Perform matrix multiplication: (3) in, Represents a multi-scale feature map of the input The channel attention applied, and They represent the convolution kernel sizes respectively. The weights of the fourth and fifth convolutional layers of , AvgPool represents the average pooling operation, Conv represents the convolution operation, ReLU represents the activation function, and BN represents the normalization layer.
6. A terahertz target detection model training method using a novel attention mechanism according to claim 1, It is characterized in that The loss function in step 6 includes complete intersection-over-union loss, confidence loss and classification loss, where the classification loss is expressed as : (5) in, represents the sample value, is the modulation factor, Indicates the number of grids into which the input image is divided, Indicates that there is an object in the nth predicted bounding box in the mth grid; t indicates the category to which it belongs, and classes indicates all categories. Represents the predicted result value; The complete intersection-over-union loss is expressed as : (6) in, Represents the distance between the predicted center point and the true center point of the target, represents the center point of the predicted bounding box, represents the center point of the ground-truth bounding box, and are the width and height of the predicted bounding box, respectively. and are the width and height of the true bounding box, respectively. IoU represents the intersection over union of the predicted bounding box and the true bounding box. c represents the diagonal distance of the minimum closure area that can contain both the predicted bounding box and the true bounding box. v represents the trade-off parameter. is the modulation coefficient; The confidence loss is expressed as : (7) In the formula, Indicates that there is an object in the nth predicted bounding box in the mth grid, Indicates that there is no object in the nth predicted bounding box in the mth grid. represents the confidence of the anchor box corresponding to the mth grid, Represents the sample label value of the mth grid, and M represents the number of predicted bounding boxes; The total loss function Loss is expressed as: (8)。
Citation Information
Patent Citations
Method for detecting small target in attention-oriented terahertz image
CN112183583A
SAR image ship target detection method based on multistage feature fusion and mixed attention
CN116469002A
Security check contraband detection method based on multi-scale attention and data enhancement
CN116883933A
Terahertz image hazardous article detection method based on multi-scale decomposition convolution
CN117095158A
Dangerous article detection method and device based on cross fusion attention mechanism
CN117115583A
Cited By
Subway hazardous article terahertz detection method based on adaptive downsampling and multi-dimensional attention mechanism
CN120635557A
Terahertz defect detection method for battery insulation coating layer
CN122084566A
Target positioning photoelectric detection system and method based on multiband fusion
CN122150237A
A multi-band fusion-based target positioning photoelectric detection system and method
CN122150237B
