A terahertz target detection model training method using a new attention mechanism

Abstract

The application discloses a terahertz target detection model training method using a novel attention mechanism and belongs to the field of terahertz science and technology. The method comprises the following steps: obtaining an enhanced image by processing a terahertz image using a wavelet transform method; inputting the enhanced image into a backbone feature extraction network to obtain a multi-scale feature map; inputting the multi-scale feature map into an adaptive context-aware attention network to obtain enhanced context-related attention feature information; inputting the enhanced context-related attention feature information into a feature fusion module to obtain multi-scale fusion feature information; and inputting the multi-scale fusion feature information into a decoupling prediction module to obtain a prediction result, training the target detection network model and updating parameters. The application improves the pixel-level recognition rate, can accurately and timely locate and identify hidden targets in a terahertz image, and has important application value in a terahertz intelligent security inspection system.

Description

Technical Field

[0001] This invention belongs to the field of terahertz science and technology, specifically relating to a training method for a terahertz target detection model using a novel attention mechanism. Background Technology

[0002] Traditional security screening facilities, including X-ray scanners, security gates, and handheld metal detectors, require manual assistance, resulting in low efficiency. Furthermore, they can only detect dangerous items in backpacks and metal objects carried by the person, and X-rays can cause radiation exposure. In addition, these devices cannot effectively detect non-metallic dangerous items such as chemical agents and ceramic knives, and they also incur high labor costs and increase processing time.

[0003] Terahertz waves lie between millimeter waves and far-infrared in the electromagnetic spectrum, with a frequency range of 0.1 THz to 10 THz (wavelength 3 mm to 30 μm). Electromagnetic waves in this band possess both the penetrability of microwaves and millimeter waves and the collimation of light waves, and their spectral resources are extremely rich. Because the terahertz band is difficult to detect using conventional electronic and optical methods, it is called the last blank space in the electromagnetic spectrum, or the terahertz gap. In the past two or three decades, radar imaging technology based on terahertz waves has developed rapidly. In particular, it has excellent performance in the field of security inspection equipment: (1) Low radiation: Terahertz waves have low radiation energy. When biological tissue is irradiated by terahertz wave beams, it will not produce photoionization effect of high-energy rays. Therefore, it is suitable for security inspection in crowded public places; (2) Good penetration: Terahertz waves can penetrate non-polar materials such as cardboard, clothing, plaster, and foam, and the loss is relatively small. It can effectively meet the security inspection needs of objects in most scenarios; (3) High resolution: Compared with microwave band, terahertz waves are in a higher frequency band and therefore have a shorter wavelength. Combined with its good characteristics of wide bandwidth, it can obtain higher imaging resolution, which is conducive to improving the accuracy of target recognition. Therefore, terahertz security inspection systems have been widely used in security inspection in various scenarios.

[0004] Based on whether it is region-based, current object detection algorithms are mainly divided into anchor-based detectors and anchor-free detectors. Among them, anchor-free algorithms do not rely on region proposal generation networks, but directly extract the location and features of the target from the image. They usually use dense sampling or adaptive methods for target localization and adopt an end-to-end training approach, making the model simpler and more efficient, and achieving faster inference speed when detecting targets.

[0005] Furthermore, attention mechanisms have been shown to be helpful for various computer vision tasks, enabling models to selectively focus on important regions or features in an image. By adjusting the weights of different channels or spatial locations in the feature map, the model can enhance the feature representation of the target region, suppress interference from irrelevant features, and thus improve its perceptual capabilities.

[0006] With the development of deep learning, the method based on convolutional neural networks (CNN) has improved the accuracy and speed of target detection in THz images, but there are still the following technical defects: (1) Unlike natural images, the original terahertz security inspection images have the disadvantages of insufficient texture information, background image noise, incomplete imaging of hidden targets, low contrast, and variable shape of hidden objects, which pose a challenge to the detection and identification of suspicious targets hidden in clothing; (2) Although convolutional neural networks can extract high-level semantic information of images, directly transplanting detection algorithms used for natural images to terahertz security inspection images will lead to serious degradation of detection performance and will not be able to effectively detect and identify suspicious targets; (3) Existing hidden target detection algorithms still have the problems of low detection efficiency and insufficient identification accuracy; (4) Previous attention mechanisms using multi-scale context fusion cannot capture the interrelationships of hidden objects in the global view, nor can they adaptively perceive global contextual dependencies, and recurrent neural networks have the problem of relying too much on prior knowledge. Summary of the Invention

[0007] To address the aforementioned technical problems, this invention provides a training method for a terahertz object detection model using a novel attention mechanism. Based on the state-of-the-art object detection framework YOLOv8, it embeds a proposed adaptive context-aware attention network. This network solves the problem in previous works where multi-scale context fusion could not capture the relationships between objects in the global view, and also addresses the problem of recurrent neural networks relying too much on prior knowledge.

[0008] To achieve the above objectives, the technical solution adopted by the present invention is as follows:

[0009] A training method for a terahertz target detection model using a novel attention mechanism, the method comprising the following steps:

[0010] Step 1: Obtain the terahertz image dataset;

[0011] Step 2: Use wavelet transform to decompose the terahertz image in the horizontal, vertical and diagonal directions to obtain the high-frequency components in the corresponding directions, and reconstruct the image based on the wavelet coefficients to obtain the enhanced image;

[0012] Step 3: Input the enhanced image into the backbone feature extraction network to obtain multi-scale feature maps;

[0013] Step 4: Input the multi-scale feature map into the adaptive context-aware attention network to obtain the enhanced context-related attention feature map processed by the adaptive context-aware attention network;

[0014] Step 5: Input the enhanced context-related attention feature map processed by the adaptive context-aware attention network into the feature fusion module for upsampling and downsampling, and then perform a connection operation to obtain a multi-scale fused feature map;

[0015] Step 6: Input the multi-scale fused feature map into the decoupled prediction module to obtain the prediction result, train the target detection network model, calculate the loss value through the loss function based on the prediction result and the annotation information of the terahertz image, calculate the gradient value of backpropagation through the optimizer using the loss value, and update the target detection network parameters using the gradient value.

[0016] The beneficial effects of this invention are as follows:

[0017] 1) By establishing a global context-aware attention mechanism model from both spatial and channel dimensions, semantic associations within the image context can be established, and guidance from semantically rich attention maps can be obtained. This allows for the adaptive integration of similar features at any scale from a global view, enhancing features by perceiving and integrating weighted features based on global contextual relationships, thus adaptively fusing related feature points together. This efficient and multi-dimensional fusion of contextual feature information enhances the target feature representation, suppresses information irrelevant to the target features, effectively simulates contextual dependencies, and thus improves pixel-level recognition rates.

[0018] 2) This invention introduces a self-attention mechanism to capture the spatial dependencies between any two locations in the feature map. For a feature at a certain location, it is updated by weighted summation of features at all locations, where the weights are determined by the feature correlation between the corresponding two locations. That is, any two locations with correlated features can mutually promote and improve each other, regardless of how far apart they are in the spatial dimension.

[0019] 3) This invention introduces an improved loss function, replacing the binary cross-entropy function in the classification loss term and adding a modulation factor; a modulation coefficient is added to the IoU loss term to increase the proportion of center point distance during training and improve positioning accuracy. Experimental results show that this invention can accurately locate and identify concealed targets in terahertz security inspection images in real time, and has important application value in terahertz intelligent security inspection systems. Attached Figure Description

[0020] Figure 1 This is a flowchart of a terahertz target detection model training method using a novel attention mechanism according to the present invention.

[0021] Figure 2 Diagram of the adaptive context-aware attention network structure;

[0022] Figure 3 This is a schematic diagram of the components after wavelet transform decomposition.

[0023] Figure 4 This is a schematic diagram of the enhanced image reconstructed based on wavelet coefficients;

[0024] Figure 5 This is a schematic diagram of the test results. Detailed Implementation

[0025] The present invention will be further described below with reference to the accompanying drawings and embodiments.

[0026] like Figure 1 The diagram shown is a flowchart of a terahertz target detection model training method using a novel attention mechanism according to the present invention, which specifically includes the following steps:

[0027] Step 1: Obtain the terahertz image dataset; the terahertz image dataset includes images and image annotation information; the annotation information includes the location information, size information, quantity information, and category information of hidden targets in the terahertz images.

[0028] Step 2: Perform wavelet transform on each terahertz image in the terahertz image dataset to obtain the enhanced image. Since the original input size is... Terahertz images have low resolution, insufficient texture information, contain background noise, and exhibit incomplete imaging of hidden targets with low contrast. Therefore, enhancement processing is performed on the original image before inputting it into the detection network. Wavelet transform is used to decompose the original image in the horizontal, vertical, and diagonal directions to obtain the high-frequency components in the corresponding directions (from left to right: horizontal, vertical, and diagonal). Figure 3 As shown, the image is then reconstructed based on the wavelet coefficients to obtain the enhanced image, as shown below. Figure 4 As shown.

[0029] Step 3, make the size as The enhanced images are first uniformly adjusted to The input is fed into the backbone feature extraction network and processed by convolutional kernels. stride is The convolution is downsampled to obtain feature maps of three different scales, i.e., multi-scale feature maps.

[0030] Step 4: Input the multi-scale feature map into the adaptive context-aware attention network to obtain the enhanced context-related attention feature map (hereinafter referred to as the enhanced attention feature map) processed by the adaptive context-aware attention network. For example... Figure 2 The diagram shows the structure of an adaptive context-aware attention network. The adaptive context-aware attention network includes a spatial attention submodule and a channel attention submodule. The spatial attention submodule is shown below. Figure 2 As shown in the upper part, assuming the input multi-scale feature map is... ,in This represents the number of channels in the multi-scale feature map. Indicates the height of the multi-scale feature map. This represents the width of the multi-scale feature map. First, it is processed by convolutional kernels... The first and second convolutional layers yield a size of The first output feature map and its size are also the same. The second output feature map is then resized to... Among them, intermediate parameters Then, a softmax operation is applied to the first output feature map to calculate a weight vector, which carries spatial global context attention information. This weight vector is then multiplied with the second output feature map to adaptively generate a feature dependency matrix between pixels. That is, its size is At the same time, multi-scale feature maps Resize to Then it is compared with the feature dependency matrix. Perform matrix multiplication to obtain the final spatial context-related attention feature map. This process can be represented as:

[0031] (1)

[0032] in, and These represent the convolution kernels in the spatial attention submodule as follows: The weights of the first and second convolutional layers, and the spatial context-related attention feature map. The first output feature map contains the first... The pixel position and the second output feature map of the second output feature map. Interdependence of pixel positions This represents the matrix multiplication operation. and These represent the first output feature map, respectively. The pixel position and the second output feature map of the second output feature map. 1 pixel position This represents all pixels.

[0033] Furthermore, the spatial context is associated with the attention feature map. The convolution kernels are input sequentially as follows The third convolutional layer and Bachnorm (BN) layer are used for further sampling to extract high-level features, and the scale of the spatial branch output is adjusted. Finally, the ReLU activation function is used to obtain the final output spatial attention feature map. The entire process can be represented as:

[0034] (2)

[0035] in, Represents the multi-scale feature map Spatial attention is applied, and Conv represents the regular convolution operation.

[0036] The channel attention submodule is as follows: Figure 2 As shown in the lower part, since each backbone network's extracted high-level feature map contains specific channel feature responses, and the semantic responses between any two channels are correlated, the feature dependencies between different channels in the channel branches are used to model the channel global contextual information features. Assume the input multi-scale feature map is... To aggregate the feature maps of each channel, the input multi-scale feature maps are processed. Global average pooling (GAP) is performed to generate channel feature dependency vectors, also known as channel context-related attention feature maps, with a size of [size missing]. This vector soft-encodes the global contextual association information in each channel. To obtain the inter-channel attention from the channel feature dependency vector, the channel feature dependency vector is input into the convolution kernel. The fourth convolutional layer downsamples the channel dimension to obtain a size of The feature map, where The reduction rate is then input into a BachNorm (BN) layer, the output ratio of the channel branches is adjusted, and ReLU activation is continued. Finally, the input convolution kernel is... The fifth convolutional layer restores the channel context-related attention feature map to... The size is used to obtain the final channel attention feature map. And then expand it to The size, and the input multi-scale feature map. The matrix multiplication operation can be represented as follows:

[0037] (3)

[0038] in, Represents the multi-scale feature map of the input. Applying channel attention, and These represent the convolution kernel as follows: The weights of the fourth and fifth convolutional layers, AvgPool represents the average pooling operation.

[0039] Furthermore, spatial attention feature maps are obtained from the two attention branches. Channel attention feature map Then, they are merged to obtain the enhanced context attention associated feature map. Among various combinations of methods such as element-wise summation, element-wise multiplication, and maximum value calculation, element-wise summation is chosen to obtain efficient gradient flow. This can be represented as:

[0040] (4)

[0041] Step 5: Input the enhanced context attention associated feature map processed by the adaptive context-aware attention module into the feature fusion module and perform a connection operation to obtain a multi-scale fused feature map.

[0042] Step 6: Input the multi-scale fused feature map into the decoupled prediction module to obtain the prediction result. Based on the prediction result and the annotation information of the sample image, train the target detection network model. Calculate the error, i.e., the loss value, using the prediction result and the annotation information through a loss function. Further, use the loss value to calculate the gradient value of backpropagation through an optimizer, and use the gradient value to update the target detection network parameters. The loss function includes the complete intersection-union loss (CIoU), confidence loss, and classification loss. The CIoU loss consists of the overlap area, center point distance, and aspect ratio, and is used for fitting the localization. The confidence loss is used to determine whether a target is contained. The classification loss is used to fit the object category information. The classification loss function is expressed as follows:

[0043] (5)

[0044] in, Represents classification loss, Indicates sample value, The modulation factor is typically set to a value of [value missing]. And it used binary cross-entropy loss. This indicates the number of grids in the input image. This indicates that the nth predicted bounding box in the mth grid contains a target. t represents the target's class, and classes represents all classes. Indicates the predicted result value;

[0045] The complete intersection-union ratio loss (CIoU) It is expressed as follows:

[0046] (6)

[0047] in, This represents the distance between the predicted center point and the true center point of the target. Indicates the center point of the predicted bounding box. This represents the center point of the actual bounding box. and These are the predicted width and height of the bounding box, respectively. and , where are the width and height of the ground truth bounding box, respectively; IoU represents the intersection-over-union ratio of the predicted and ground truth bounding boxes; c represents the diagonal distance of the smallest closure region that can simultaneously contain both the predicted and ground truth bounding boxes; and v represents the tradeoff parameter. This is the modulation coefficient, and its value is generally 1. Modulation coefficients Multiplying by the aspect ratio term can increase the proportion of the center point distance during training, thereby improving positioning accuracy.

[0048] The confidence loss It is expressed as follows:

[0049] (7)

[0050] In the formula, This indicates that the nth predicted bounding box in the mth grid contains a target. This indicates that the nth predicted bounding box in the mth grid does not contain a target. This represents the confidence level of the anchor frame corresponding to the m-th grid. This represents the sample label value of the m-th grid, and M represents the number of predicted bounding boxes.

[0051] Furthermore, the overall loss function Loss can be expressed as:

[0052] (8)

[0053] After obtaining the loss value through the loss function Loss, the network begins backpropagation to update the network's weights and bias parameters. This process is repeated until the loss value reaches its minimum, at which point the network model parameters are saved.

[0054] Example

[0055] In this example, by using the training method of the target detection model with the adaptive context-aware attention mechanism, the network can learn to use global information of the terahertz image to selectively emphasize hidden target information features and suppress features irrelevant to the target, effectively improving the accuracy of target detection in terahertz images.

[0056] Step 1: Obtain the terahertz image dataset: The terahertz image dataset includes images and image annotation information; the annotation information includes the location information, size information, quantity information, and category information of hidden targets in the terahertz images.

[0057] Step 2: Image enhancement. Wavelet transform is used to decompose the original image in the horizontal, vertical and diagonal directions to obtain the high-frequency components in the corresponding directions. Finally, the image is reconstructed based on the wavelet coefficients to obtain the enhanced image.

[0058] Step 3: Input the enhanced image into the backbone feature extraction network to obtain multi-scale feature maps. The backbone feature extraction network includes four alternately connected CSPlayer_2Conv layers, five convolutional layers, and a spatial pyramid pooling module (SPPF). The kernel size of the convolutional layers is [missing information]. The convolution stride is The input feature map is processed by a first convolutional layer and then padded. The CSPrayer_2Conv module transforms the input feature map through the first convolutional layer, resulting in a feature map divided into two parts. These two parts are processed by different convolutional layers and activated using the SiLU activation function before being merged. This process helps the model capture more contextual information, thus more accurately identifying the target. The merged feature map is then processed by a second convolutional layer for final output. The SPPF module samples the input feature map through three max-pooling layers and then concatenates the outputs. The CSPdarknet53 downsamples the input augmented image and inputs the three feature maps of different scales obtained from the CSPrayer_2Conv and SPPF modules into an adaptive context-aware attention module to capture feature dependencies in both spatial and channel dimensions.

[0059] Step 4: Input the multi-scale feature map into the adaptive context-aware attention network to obtain the enhanced context-related attention feature map processed by the adaptive context-aware attention network.

[0060] Step 5: Input the context-related attention feature map processed by the adaptive context-aware attention network into the feature fusion module for upsampling and downsampling, and then perform a connection operation to obtain a multi-scale fused feature map.

[0061] Step 6: Input the multi-scale fused feature map into the decoupled prediction module to obtain the prediction result. Based on the prediction result and the annotation information of the sample image, train the target detection network model. Calculate the error, i.e., the loss value, using the prediction result and the annotation information through a loss function. Further, use the loss value to calculate the gradient value of backpropagation through an optimizer, and use the gradient value to update the target detection network parameters.

[0062] Finally, the trained object detection network model is used for detection, and the detection results are shown in the following example. Figure 5 As shown, from left to right, the images depict a person carrying a single target, two targets, and three targets. The objects carried include a pistol, a water bottle, a wallet, and a ceramic knife (in the example images, WB represents a water bottle, GA represents a pistol model, KK represents a kitchen knife, SS represents scissors, LW represents a leather wallet, MD represents a metal dagger, and CK represents a ceramic knife). It can be clearly seen that the Adaptation-YOLO method used in this invention achieves accurate classification and high detection accuracy for hidden objects carried by the human body.

[0063] The specific embodiments described above further illustrate the purpose, technical solution, and beneficial effects of the present invention. It should be understood that the above descriptions are merely specific embodiments of the present invention and are not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A training method for a terahertz target detection model using a novel attention mechanism, characterized in that, The method includes the following steps: Step 1: Obtain the terahertz image dataset; Step 2: Use wavelet transform to decompose the terahertz image in the horizontal, vertical and diagonal directions to obtain the high-frequency components in the corresponding directions, and reconstruct the image based on the wavelet coefficients to obtain the enhanced image; Step 3: Input the enhanced image into the backbone feature extraction network to obtain multi-scale feature maps; Step 4: Input the multi-scale feature map into the adaptive context-aware attention network to obtain an enhanced context-related attention feature map processed by the adaptive context-aware attention network; the adaptive context-aware attention network includes a spatial attention submodule and a channel attention submodule, and the spatial attention feature map is obtained from the spatial attention submodule and the channel attention submodule, respectively. Channel attention feature map The features are then merged to obtain an enhanced contextual attention-related feature map. : (4)， Among them, the spatial attention feature map Obtained through the following steps: Let the input multi-scale feature map be... ,in This represents the number of channels in the multi-scale feature map. Indicates the height of the multi-scale feature map. The width of the multi-scale feature map is represented by the convolution kernel. The first and second convolutional layers yield a size of The first output feature map and its size are also the same. The second output feature map, and then the sizes of the first and second output feature maps are resized to... Among them, intermediate parameters ; The first output feature map is used to calculate a weight vector using softmax, which is then multiplied with the second output feature map to generate a feature dependency matrix between pixels. ; Multi-scale feature maps Resize to Then, the feature relation dependency matrix Perform matrix multiplication to obtain the final spatial context-related attention feature map. Spatial context is associated with attention feature maps The convolution kernels are input sequentially as follows The third convolutional layer and BN layer extract high-level features, which are then activated using the ReLU activation function to obtain the final spatial attention feature map. ; The channel attention feature map Obtained through the following steps: Let the input multi-scale feature map be... For the input multi-scale feature map Global average pooling is performed to generate channel context-related attention feature maps, which are then input into the convolutional kernel. The fourth convolutional layer downsamples the channel dimension, then inputs it into the BN layer, and after activation using the ReLU activation function, the input convolutional kernel is... The fifth convolutional layer restores the channel context-related attention feature map to... The size is used to obtain the final channel attention feature map. Expand size to Then, it is compared with the input multi-scale feature map. Perform matrix multiplication operations; Step 5: Input the enhanced context-related attention feature map processed by the adaptive context-aware attention network into the feature fusion module for upsampling and downsampling, and then perform a connection operation to obtain a multi-scale fused feature map; Step 6: Input the multi-scale fused feature map into the decoupled prediction module to obtain the prediction result, train the target detection network model, calculate the loss value based on the prediction result and the annotation information of the terahertz image using a loss function, calculate the gradient value of backpropagation using the loss value through the optimizer, and update the target detection network parameters using the gradient value; the loss function includes perfect intersection-union loss, confidence loss, and classification loss, wherein the classification loss is expressed as... : (5) in, Indicates sample value, For modulation factor, This indicates the number of grids in the input image. This indicates that the nth predicted bounding box in the mth grid contains a target; t represents the class, and classes represents all classes. This represents the predicted result value.

2. The method for training a terahertz target detection model using a novel attention mechanism according to claim 1, characterized in that, The terahertz image dataset includes images and image annotation information. The annotation information includes the location, size, quantity, and category information of hidden targets in the terahertz images.

3. The method for training a terahertz target detection model using a novel attention mechanism according to claim 1, characterized in that, The final spatial context-related attention feature map for: (1) in, and These represent the convolution kernels in the spatial attention submodule as follows: The weights of the first and second convolutional layers, This represents the matrix multiplication operation. and These represent the first output feature map, respectively. The pixel position and the second output feature map of the second output feature map. 1 pixel position Represents all pixels; The final spatial attention feature map for: (2) in, Represents the multi-scale feature map Spatial attention is applied, Conv represents convolution operation, ReLU represents activation function, and BN represents normalization layer.

4. The method for training a terahertz target detection model using a novel attention mechanism according to claim 1, characterized in that, The channel attention feature map for: (3) in, Represents the multi-scale feature map of the input. Applying channel attention, and These represent the kernel size as follows: The weights of the fourth and fifth convolutional layers are: AvgPool represents the average pooling operation, Conv represents the convolution operation, ReLU represents the activation function, and BN represents the normalization layer.

5. The method for training a terahertz target detection model using a novel attention mechanism according to claim 1, characterized in that, In step 6, the complete intersection-union ratio loss is expressed as: : (6) in, This represents the distance between the predicted center point and the true center point of the target. Indicates the center point of the predicted bounding box. This represents the center point of the actual bounding box. and These are the predicted width and height of the bounding box, respectively. and , where are the width and height of the ground truth bounding box, respectively; IoU represents the intersection-over-union ratio of the predicted and ground truth bounding boxes; c represents the diagonal distance of the smallest closure region that can simultaneously contain both the predicted and ground truth bounding boxes; and v represents the tradeoff parameter. The modulation coefficient; Confidence loss is expressed as : (7) In the formula, This indicates that the nth predicted bounding box in the mth grid contains a target. This indicates that the nth predicted bounding box in the mth grid does not contain a target. This represents the confidence level of the anchor frame corresponding to the m-th grid. This represents the sample label value of the m-th grid, where M represents the number of predicted bounding boxes; The overall loss function, Loss, is expressed as: (8)。

Images