A target detection method, apparatus, electronic device, medium, and program product

Abstract

The present application provides a kind of target detection method, device, electronic equipment, medium and program product, the method comprises: obtaining target infrared image, the shallow feature map of target infrared image is extracted by pre-set codec, wherein shallow feature map indicates local detail feature in target infrared image;Deep feature map of target infrared image is extracted by enhancement module in codec, wherein deep feature map indicates the deep relationship of small target and background relationship in target infrared image;Shallow feature map and deep feature map are fused by hybrid attention feature fusion module in codec, and target detection result is obtained.The present application, by embedding enhancement module, obtains the feature information of global joint local of infrared small target in series form, makes full use of the local detail extraction capability of codec and the long-term global information attention capability of enhancement module, promotes global and local information attention fusion, and further strengthens network detail feature capture capability.

Description

Technical Field

[0001] The embodiments of the present invention relate to the field of artificial intelligence technology, and in particular to a target detection method, apparatus, electronic device, medium and program product. Background Technology

[0002] With the upgrading of infrared sensors and UAV photography equipment, infrared imaging detection systems have become the mainstream vision system for target detection in scenarios such as long distances, dark scenes, and interference resistance. Infrared detection technology utilizes the difference in infrared radiation between the target object and background noise to detect and track the target object, offering significant advantages over active radar imaging and visible light imaging. However, most infrared small target images suffer from extremely small target sizes and inconsistent target scales in sequential images, placing high demands on the algorithm's detail attention capabilities. Furthermore, due to harsh imaging environments, image quality is often poor, containing significant clutter noise, making infrared small target detection a challenging problem that remains to be solved.

[0003] Existing solutions include traditional model-driven methods and deep learning data-driven methods. Among them, infrared small target detection algorithms based on traditional models include image filter-based methods, local information-based methods, and data structure-based methods. Image filter methods have poor robustness, are prone to false positives or false negatives in complex backgrounds, and have poor versatility; local information-based methods have poor adaptability, high computational complexity, and easily overlook target details; data structure-based methods have limited computational power, are difficult to process, and have poor real-time performance; deep learning methods have poor adaptability to different scenarios, are time-consuming to train, have poor robustness, and struggle to maintain effectiveness in complex environments. Summary of the Invention

[0004] This invention provides a target detection method, apparatus, electronic device, medium, and program product to solve the problems of weak global information focus capability and poor joint global and local information in existing infrared small target detection methods.

[0005] To solve the above-mentioned technical problems, the present invention is implemented as follows:

[0006] In a first aspect, embodiments of the present invention provide a target detection method, comprising:

[0007] Acquire a target infrared image, and extract a shallow feature map of the target infrared image using a preset codec, wherein the shallow feature map represents local detail features in the target infrared image;

[0008] The deep feature map of the target infrared image is extracted by the enhancement module in the codec, wherein the deep feature map represents the deep relationship between small targets and the background in the target infrared image;

[0009] The shallow feature map and the deep feature map are fused by the hybrid attention feature fusion module in the codec to obtain the target detection result.

[0010] Optionally, the preset codec is a U-shaped codec, and the enhancement module is a visual converter attention mechanism module with an enhanced feedforward neural network.

[0011] Optionally, the step of extracting the deep feature map of the target infrared image through the enhancement module in the codec includes:

[0012] The enhancement module in the codec extracts a first feature map from the target infrared image and encodes the first feature map to obtain a first feature representation;

[0013] The enhancement module processes the first feature representation through a multi-head attention network to obtain a second feature representation;

[0014] The enhancement module uses an enhanced feedforward neural network to strengthen the second feature representation to obtain a third feature representation;

[0015] The enhancement module iteratively extracts long-distance information tokens from the third feature representation to obtain the deep feature map.

[0016] Optionally, the step of using an enhanced feedforward neural network to strengthen the second feature representation to obtain a third feature representation includes:

[0017] The second feature representation is restored to the first three-dimensional feature map through the first fully connected layer;

[0018] The first three-dimensional feature map is input into the first convolutional layer for convolution processing, then into the normalization layer for normalization processing, and then into the second convolutional layer for convolution processing to obtain the second three-dimensional feature map.

[0019] The second 3D feature map is fully connected through a second fully connected layer to obtain the third feature representation.

[0020] Optionally, the step of fusing the shallow feature map and the deep feature map through the hybrid attention feature fusion module in the codec to obtain the target detection result includes:

[0021] The shallow feature maps of multiple levels of the codec are obtained, and the shallow feature maps of multiple levels are fused by pyramid fusion to obtain the fused shallow features.

[0022] The hybrid attention feature fusion module performs point-by-point and channel-by-channel attention interaction between the fused shallow features and the deep feature map to obtain the target detection result.

[0023] Optionally, the hybrid attention feature fusion module includes: a point-by-point attention module, a channel attention module, and a shallow-deep feature summing module. The point-by-point attention module focuses on the infrared small target detail bounding box information from shallow to deep features and from bottom to top, outputting a fused shallow output feature map with strong local detail awareness. The channel attention module focuses on the relationship between the infrared small target background and the target from deep to shallow features and from top to bottom, outputting a deep global output feature map of important weight channels. The shallow-deep feature summing module sums the shallow features and the deep features to balance network features.

[0024] Secondly, embodiments of the present invention provide a target detection device, comprising:

[0025] The first extraction module is used to acquire a target infrared image and extract a shallow feature map of the target infrared image through a preset codec, wherein the shallow feature map represents local detail features in the target infrared image;

[0026] The second extraction module is used to extract the deep feature map of the target infrared image through the enhancement module in the codec, wherein the deep feature map represents the deep relationship between small targets and the background in the target infrared image;

[0027] The first processing module is used to fuse the shallow feature map and the deep feature map through the hybrid attention feature fusion module in the codec to obtain the target detection result.

[0028] Optionally, the preset codec is a U-shaped codec, and the enhancement module is a visual converter attention mechanism module with an enhanced feedforward neural network.

[0029] Optionally, the second extraction module includes:

[0030] The first processing submodule is used to extract the first feature map of the target infrared image from the enhancement module in the codec, and encode the first feature map to obtain the first feature representation;

[0031] The second processing submodule is used by the enhancement module to process the first feature representation through a multi-head attention network to obtain the second feature representation;

[0032] The third processing submodule is used by the enhancement module to strengthen the second feature representation using an enhanced feedforward neural network to obtain a third feature representation;

[0033] The fourth processing submodule is used by the enhancement module to iteratively extract long-distance information tokens from the third feature representation to obtain the deep feature map.

[0034] Optionally, the third processing submodule includes:

[0035] The fifth processing submodule is used to restore the second feature representation into the first three-dimensional feature map through the first fully connected layer;

[0036] The sixth processing submodule is used to input the first three-dimensional feature map into the first convolutional layer for convolution processing, then into the normalization layer for normalization processing, and then into the second convolutional layer for convolution processing to obtain the second three-dimensional feature map.

[0037] The seventh processing submodule is used to fully connect the second three-dimensional feature map through the second fully connected layer to obtain the third feature representation.

[0038] Optionally, the first processing module includes:

[0039] The eighth processing submodule is used to obtain shallow feature maps of multiple levels of the codec, and to perform pyramid fusion on the shallow feature maps of multiple levels to obtain fused shallow features.

[0040] The ninth processing submodule is used to perform point-by-point and channel attention interaction between the fused shallow features and the deep feature map through the hybrid attention feature fusion module to obtain the target detection result.

[0041] Optionally, the hybrid attention feature fusion module includes: a point-by-point attention module, a channel attention module, and a shallow-deep feature summing module. The point-by-point attention module focuses on the infrared small target detail bounding box information from shallow to deep features and from bottom to top, outputting a fused shallow output feature map with strong local detail awareness. The channel attention module focuses on the relationship between the infrared small target background and the target from deep to shallow features and from top to bottom, outputting a deep global output feature map of important weight channels. The shallow-deep feature summing module sums the shallow features and the deep features to balance network features.

[0042] Thirdly, embodiments of the present invention provide an electronic device, including a processor, a memory, and a program or instructions stored in the memory and executable on the processor, wherein the program or instructions, when executed by the processor, implement the steps of the target detection method as described in any one of the first aspects.

[0043] Fourthly, embodiments of the present invention provide a readable storage medium storing a program or instructions that, when executed by a processor, implement the steps of the target detection method as described in any one of the first aspects.

[0044] Fifthly, embodiments of the present invention provide a computer program product including computer instructions that, when executed by a processor, implement the steps of the target detection method as described in any one of the first aspects.

[0045] In this invention, a target infrared image is acquired, and a shallow feature map of the target infrared image is extracted using a preset codec. This shallow feature map represents local detail features in the target infrared image. A deep feature map of the target infrared image is extracted using an enhancement module in the codec. This deep feature map represents the deep relationship between the small target and the background in the target infrared image. The shallow and deep feature maps are then fused using a hybrid attention feature fusion module in the codec to obtain the target detection result. This invention acquires global and local joint feature information of infrared small targets in a cascaded manner by embedding an enhancement module. It fully utilizes the local detail extraction capability of the codec and the long-term global information attention capability of the enhancement module, promoting the fusion of global and local information attention, further strengthening the network's ability to capture detailed features, and solving the problems of weak global information attention capability and poor joint global and local information in infrared small target detection methods. Attached Figure Description

[0046] Various other advantages and benefits will become apparent to those skilled in the art upon reading the following detailed description of preferred embodiments. The accompanying drawings are for illustrative purposes only and are not intended to limit the invention. Furthermore, the same reference numerals denote the same parts throughout the drawings. In the drawings:

[0047] Figure 1 This is a flowchart of a target detection method provided in an embodiment of the present invention;

[0048] Figure 2 This is a schematic diagram of the structure of a target detection method provided in an embodiment of the present invention;

[0049] Figure 3 This is a schematic diagram of the structure of a multi-layer point-by-point channel attention fusion module in a target detection method provided in an embodiment of the present invention;

[0050] Figure 4 This is a schematic diagram of the structure of a multi-layer point-by-point channel attention fusion module in another target detection method provided in this embodiment of the invention;

[0051] Figure 5 This is a schematic diagram of the pointwise convolution structure of a target detection method provided in an embodiment of the present invention;

[0052] Figure 6 This is a schematic diagram of the structure of a target detection device provided in an embodiment of the present invention;

[0053] Figure 7 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention. Detailed Implementation

[0054] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0055] Please refer to Figure 1 This invention provides a target detection method, comprising:

[0056] Step 11: Acquire the target infrared image, and extract the shallow feature map of the target infrared image through a preset codec, wherein the shallow feature map represents the local detail features in the target infrared image;

[0057] In this embodiment of the invention, the preset codec is a U-shaped codec. Please refer to [reference needed]. Figure 2 This study employs a Resblock-based U-shaped codec to obtain the output of a convolutional neural network. Specifically, it combines ResNet and U-Net structures, utilizing Resblock staggered blocks to construct the codec and then employing U-Net-like jump connections to combine deep and shallow information. The Resblock-based U-shaped codec can be configured with different levels of symmetric codecs based on the characteristics of the dataset and experimental conditions. Figure 2 The above description uses a three-layer depth (each layer contains 4 Resblocks) as an example. The encoder module corresponds to Layer 1, Layer 2, and Layer 3 from top to bottom, and the decoder corresponds to Up-Layer 1, Up-Layer 2, and Up-Layer 3. It is particularly important to note that in order to prevent small infrared targets in the target infrared image from disappearing during downsampling of the current backbone network, downsampling is only performed between each adjacent layer, that is, between Layer 1 and Layer 2, and between Layer 2 and Layer 3. Upsampling is designed between Up-Layer 1 and Up-Layer 2, and between Up-Layer 2 and Up-Layer 3, thereby improving the detection accuracy of local detailed features in the target infrared image.

[0058] Step 12: Extract the deep feature map of the target infrared image through the enhancement module in the codec, wherein the deep feature map represents the deep relationship between small targets and the background in the target infrared image;

[0059] In the embodiments of this invention, please refer to Figure 2 The codec embeds an enhancement module, specifically an enhanced Vision Transformer attention mechanism module incorporating an enhanced feedforward neural network. Utilizing embedded token blocks, it focuses on the deep relationship between small targets and the background. The small targets, whose size is smaller than or significantly smaller than a preset size threshold in the overall background, are easily ignored or difficult to distinguish. Since infrared imaging often occurs in harsh environments with poor image quality, the texture information of small targets is weakened by radiation propagation noise. Especially when the ambient infrared radiation signal and the radiation signal of the small target are similar, the algorithm struggles to distinguish the difference between the target and the background, leading to a high false alarm rate. Furthermore, in highly complex backgrounds, the noise in small target images is highly similar to the target itself. Therefore, the algorithm needs to maintain its detail extraction capabilities while also focusing on the overall semantic information of the image to obtain the dependencies between the foreground and background. Furthermore, during the detection process, noise information related to the background and target information of key interest are distinguished. In this embodiment of the invention, an enhanced visual converter attention mechanism network with enhanced feedforward neural network is embedded on the U-shaped convolutional neural network structure. Combining the local information extraction capability of the convolutional neural network with the global information attention capability of the Transformer, the high robustness infrared small target detection algorithm based on the U-shaped encoder and decoder is improved, which further reduces the false alarm rate, improves the detection rate, and more accurately segments the infrared small target details. This enables the identification of weak textured target objects even in the case of severe noise, and has good robustness and high accuracy.

[0060] Step 13: The shallow feature map and the deep feature map are fused by the hybrid attention feature fusion module in the codec to obtain the target detection result.

[0061] In this embodiment of the invention, the shallow feature map and the deep feature map are fused by the hybrid attention feature fusion module in the codec. The idea of ​​local and global information attention fusion is introduced. A multi-layer point-by-point channel attention fusion module is used to fuse the output feature information of multiple shallow networks with the global feature information of deep networks. Before the information fusion at different levels, effective details and channel information are filtered by point-by-point attention and channel attention respectively. Important semantic information is fused instead of directly splicing and fusion. The shallow details and deep semantic information of infrared small target images are obtained respectively, taking into account the advantages of high accuracy and low false alarm rate, further enhancing the network's ability to capture detailed features and focusing on small targets with weak texture.

[0062] In this embodiment of the invention, a target infrared image is acquired, and a shallow feature map of the target infrared image is extracted using a preset codec. The shallow feature map represents local detail features in the target infrared image. A deep feature map of the target infrared image is extracted using an enhancement module in the codec. The deep feature map represents the deep relationship between the small target and the background in the target infrared image. The shallow and deep feature maps are fused using a hybrid attention feature fusion module in the codec to obtain the target detection result. In this invention, by embedding an enhancement module to acquire global and local joint feature information of infrared small targets in a serial manner, the local detail extraction capability of the codec and the long-term global information attention capability of the enhancement module are fully utilized. This promotes the fusion of global and local information attention, further enhances the network's ability to capture detailed features, and solves the problems of weak global information attention capability and poor joint global and local information in infrared small target detection methods.

[0063] In this embodiment of the invention, optionally, the step of extracting the deep feature map of the target infrared image through the enhancement module in the codec includes:

[0064] The enhancement module in the codec extracts a first feature map from the target infrared image and encodes the first feature map to obtain a first feature representation;

[0065] The enhancement module processes the first feature representation through a multi-head attention network to obtain a second feature representation;

[0066] The enhancement module uses an enhanced feedforward neural network to strengthen the second feature representation to obtain a third feature representation;

[0067] The enhancement module iteratively extracts long-distance information tokens from the third feature representation to obtain the deep feature map.

[0068] In the embodiments of this invention, please refer to Figure 2 First, the original input image is extracted using a CNN encoder. compact features That is, the first feature representation; then the image is viewed as a series of two-dimensional patch markers. Where N = (HW / P) 2 (P,P) represents the number of patches, and (P,P) represents the resolution of each patch. Then, positional embedding E is added. pos Obtain the embedded token token TokenE=E em +E pos ,in E = (E1, E2, ..., E n ), where n = H′W′ represents the number of markers.

[0069] To obtain the correlation among n tokens, the embedded token token E is processed using a multi-head self-attention (MSA) mechanism with m heads. The embedded token E is divided into m heads, represented as E = {E...} 1 E 2 ,…,E m},in They are then fed into the multi-head self-attention module MSA(·) to obtain interactive tags. E a =(E a-1 E a-2 ,…,E a-n ), where n = H′W′ represents the number of tags, i.e., the second feature representation. We define these processes as:

[0070] E a =MSA(LN(E))+E (1)

[0071] Where LN(·) represents layer normalization.

[0072] To prevent small targets from being lost during multiple global iterations of the Transformer, an Enhanced Feedforward Network (EFN) module is further utilized. Between two fully connected layers, two 3x3 convolutions and a BN layer are used for feature enhancement to obtain a third feature representation; the enhanced label after EFN is called... Repeat the iteration k times (k=12) to extract long-distance information tokens, and further restore the global long-term feature information map. That is, the enhancement module iteratively extracts long-distance information tokens from the third feature representation to obtain the deep feature map.

[0073] In this embodiment of the invention, optionally, the step of using an enhanced feedforward neural network to strengthen the second feature representation to obtain a third feature representation includes:

[0074] The second feature representation is restored to the first three-dimensional feature map through the first fully connected layer;

[0075] The first three-dimensional feature map is input into the first convolutional layer for convolution processing, then into the normalization layer for normalization processing, and then into the second convolutional layer for convolution processing to obtain the second three-dimensional feature map.

[0076] The second 3D feature map is fully connected through a second fully connected layer to obtain the third feature representation.

[0077] In this embodiment of the invention, in order to prevent small targets from being lost in multiple global iterations of the Transformer, the Enhanced Feedforward Network (EFN) module is further used to perform feature enhancement between two fully connected layers, through two 3*3 convolutions and one BN layer. That is, the first three-dimensional feature map is input into the first convolutional layer for convolution processing, then input into the normalization layer for normalization processing, and then input into the second convolutional layer for convolution processing to obtain the second three-dimensional feature map.

[0078] F feature3D '=Conv(BN(Conv(F feature3D (2)

[0079] Among them, F feature3D F is the first 3D feature map recovered after passing through the first fully connected layer. feature3D The feature map is enhanced by the convolutional neural network, then passes through a second fully connected layer and jumps to E. a Get reinforced marker The third feature representation is obtained.

[0080] In this embodiment of the invention, to address the problem of generally high false alarm rates in infrared small target detection, an enhanced Vision Transformer network combined with a convolutional neural network is introduced. By combining the local information extraction capability of the convolutional neural network with the long-term global information focus capability of the transformer, a detection network capable of accurately identifying infrared small targets is constructed. Furthermore, the design uses convolution between feedforward neural networks to preserve small target features as much as possible during transformer iteration, focusing on the long-term information of the background and small target, rather than just focusing on local information. This can also reduce the false alarm rate and improve detection accuracy.

[0081] In this embodiment of the invention, optionally, the step of fusing the shallow feature map and the deep feature map through the hybrid attention feature fusion module in the codec to obtain the target detection result includes:

[0082] The shallow feature maps of multiple levels of the codec are obtained, and the shallow feature maps of multiple levels are fused by pyramid fusion to obtain the fused shallow features.

[0083] The hybrid attention feature fusion module performs point-by-point and channel-by-channel attention interaction between the fused shallow features and the deep feature map to obtain the target detection result.

[0084] In this embodiment of the invention, since infrared small targets have very few pixels, there is a possibility of losing small targets as the network deepens. Deep networks, while possessing strong semantic representation capabilities, need to supplement rich shallow information. Therefore, this embodiment of the invention uses a multi-scale feature fusion strategy similar to UNet3+, designing a multi-layer pointwise channel attention fusion module to obtain shallow output features from multiple encoder layers. By applying the Multilayer Pointwise Channel Attention Fusion (MPCAF) module to the decoder, the shallow feature maps from the multi-layer encoder are fused using a pyramid method, and then interacted with the deep feature maps processed by Res Block*4 using pointwise and channel attention. During multi-scale output fusion, multi-layer pyramid features are used to obtain infrared small targets instead of just the last layer, resulting in multi-layer representations at different resolutions. This fusion process enhances the network's ability to detect small targets at different scales, improves the algorithm's robustness, and adapts to different target size variations. It can accurately and reliably generate infrared small target segmentation results in most interference scenarios with a low false alarm rate.

[0085] In this embodiment of the invention, optionally, the hybrid attention feature fusion module includes: a point-by-point attention module, a channel attention module, and a shallow-deep feature summing module. The point-by-point attention module focuses on the infrared small target detail bounding box information from shallow to deep features and from bottom to top, outputting a fused shallow output feature map with strong local detail awareness. The channel attention module focuses on the relationship between the infrared small target background and the target from deep to shallow features and from top to bottom, outputting a deep global output feature map of important weight channels. The shallow-deep feature summing module sums the shallow features and the deep features to balance network features.

[0086] In the embodiments of this invention, please refer to Figure 2 and Figure 3 Taking the use of the MPCAF module in the Up-Layer 3 layer as an example (feature maps and number of channels are only for illustrative purposes), the specific process is analyzed as follows:

[0087] like Figure 3 As shown, the shallow feature maps of the multi-layer encoder are first adjusted to a uniform size, and then fused through convolutional concatenation and convolutional fusion to obtain X. l The target detail features of the shallow network are further enhanced by a point-by-point attention module, and important channels are selected by the point-multiplication fusion channel attention module. By encoding smaller-scale details into deeper layers in a bottom-up manner, and simultaneously filtering effective channel information in a top-down manner, the fine details of small infrared targets are not drowned out by background noise, effectively improving detection accuracy and reducing false alarm rate. hRepresenting deep features, X l This represents the shallow features after fusion.

[0088] like Figure 4 As shown, the hybrid attention feature fusion module filters effective output features from the convolutional neural network and the Transformer module through pointwise attention and channel attention modules, respectively, and then multiplies and fuses the filtered features. The hybrid attention feature fusion module mainly utilizes the attention mechanism to filter effective features. To ensure no loss of model information during asymmetric fusion, information from before the attention processing is added when multiplying the extracted attention output feature maps. This allows for the enhancement of the feature maps while retaining more information.

[0089] like Figure 4 As shown, the model output mainly consists of three parts: a point-wise attention module, a channel attention module, and a shallow-deep feature summation module. The left side shows the point-wise attention (PA) block, and the right side shows the channel attention (CA) block. The point-wise attention module focuses on the shallow-to-deep, bottom-up details of infrared small targets, obtaining a fused shallow output feature map with strong local detail awareness. The channel attention module mainly focuses on the relationship between the infrared small target background and the target from deep to shallow, top-to-bottom, obtaining a deep global output feature map of important weight channels. The shallow-deep feature summation result is used to balance network features and prevent overfitting.

[0090] The above process can be explained using formula (3):

[0091]

[0092] Among them, X l X represents the result of fusing multiple shallow features. h Z represents the deep features, Z represents the output of the hybrid attention feature fusion module, PA(·) represents the point-by-point attention module processing, and CA(·) represents the channel attention processing. ⊙ and ⊙ represent element-wise multiplication and vector multiplication, respectively, and Res(·) represents the processing through 4 resblocks.

[0093] The point-by-point attention module is primarily used to capture detailed information about image targets, enhancing the network's spatial perception. In infrared small target detection, small targets are typically small in size and their locations are uncertain, placing high demands on the model's ability to perceive local details. In the shallow layers of deep learning convolutional neural networks, target detail information is richer, and the point-by-point attention module helps the model focus on important target pixels in the image, improving detection accuracy. By using the point-by-point attention module after fusing the outputs of multiple encoder layers, in conjunction with a hybrid attention feature fusion module, the detailed information output by the shallow convolutional neural network is enhanced.

[0094] Specifically, the pointwise attention module primarily applies pointwise convolution to aggregate multi-layer information and highlight the semantic features of small target details. First, the input features are processed through convolution operations to learn feature representations at different locations, helping the model capture local information and subtle features in the image. A schematic diagram of pointwise convolution is shown below. Figure 5 As shown, the learned feature maps are used to calculate the attention weights for each pixel. These weights can be used to weight the features according to the importance of each pixel, making the model pay more attention to pixels that contribute to the detection of small objects. The specific process is as follows:

[0095]

[0096] Among them, PA(X) l ) represents the output feature of the pointwise attention block; σ represents the Sigmoid function, which is used to normalize the attention weights to the interval [0,1]. PWConv1(·) represents the batch normalization operation, used to accelerate the training process of neural networks and improve the stability of the model; PWConv1(·) represents the first pointwise convolution operation, used to extract features; δ(·) represents the activation function, usually the ReLU function, used to introduce non-linearity; PWConv2(·) represents the second pointwise convolution operation, used to generate attention weights. This represents element-wise multiplication, X l This is a shallow feature.

[0097] Equation (4) describes the computation process of the pointwise attention block, where the input feature X is processed by two pointwise convolution operations PWConv1 and PWConv2. l Feature extraction and attention weight generation are performed; pointwise convolutions with a kernel size of 1*1 are performed, and two pointwise convolutions convert the input feature channels to C / 4 and then back to C. Subsequently, activation function δ and batch normalization are applied. After processing features and attention weights, the result is normalized using the Sigmoid function σ and then compared with X. l Multiplying them together yields the output feature PA(X). lThis output feature PA(X) l It will include input features X l Important detailed features.

[0098] As the number of neural network layers increases, the size of the output feature maps gradually decreases while the number of channels gradually increases. For infrared small target detection, important target semantic information is distributed in the channels containing effective information. Therefore, it is necessary to enhance the model's perceptual ability through channel attention modules, and to enhance the network's representational ability by dynamically adjusting the importance of different channels. By applying channel attention modules in the deep channels of the decoder, and combining them with a hybrid attention feature fusion module to calculate and adjust channel weights, the adjusted feature maps are finally fused with the original feature maps using weighted averages.

[0099] Specifically, in the channel attention module, global average pooling is first performed on the input features to compress the spatial size, focusing more on the information of each channel in the feature map. Feature learning is performed along the channel dimension, and channels containing richer information about small objects are given greater weight. The specific process is as follows:

[0100] Assume the input feature map is X H =Res(X h The feature map has dimensions H×W×C, where H represents the height of the feature map, W represents the width, and C represents the number of channels. Each element X in the feature map... ijc Let X represent the feature value of the i-th row, j-th column, and c-th channel. Global average pooling operation will convert the feature map X... H The eigenvalues ​​of each channel in the pooling layer are averaged to obtain the output feature Z1 of the global flat pooling layer, which has a dimension of 1×1×C. c Let c be the c-th number of Z1.

[0101]

[0102] Then, two fully connected layers are used to generate weights for each feature channel. After the first fully connected layer, the number of feature channels becomes C / 2, and after the second fully connected layer, the number of feature channels becomes C, resulting in the output feature Z2.

[0103] CA(Res(X h ))=Z2=A1(ReLU(A2Z1)) (6)

[0104] Where, A1∈R C×C / 2 and A2∈R C / 2×C These are the weights of the two fully connected layers; ReLU(·) is the ReLU function.

[0105] The first fully connected layer compresses the small target information data in the feature map and aggregates the channel information in the average pooled feature map, completing the interaction of small target information between channels and retaining more channels containing small target information. The second fully connected layer increases the number of channels in the feature map and predicts the importance of each channel, i.e., how much small target information it contains, which facilitates small target detection. After passing through a sigmoid activation function, a channel attention feature map is obtained, which is then multiplied element-wise with the original input features to complete the learning of channel attention. This process enables the model to automatically learn the importance of each channel and focus attention on the channels containing small target information, thereby improving the performance of small target detection. A multi-layer pointwise channel attention fusion module promotes the fusion of global and local information attention. Before information fusion at different levels, effective details and channel information are filtered through pointwise attention and channel attention, fusing important semantic information rather than directly splicing them together, further strengthening the network's ability to capture detailed features and focusing on weakly textured small targets.

[0106] Please refer to Figure 6 This invention provides a target detection device, comprising:

[0107] The first extraction module 61 is used to acquire a target infrared image and extract a shallow feature map of the target infrared image through a preset codec, wherein the shallow feature map represents local detail features in the target infrared image;

[0108] The second extraction module 62 is used to extract the deep feature map of the target infrared image through the enhancement module in the codec, wherein the deep feature map represents the deep relationship between small targets and the background in the target infrared image;

[0109] The first processing module 63 is used to fuse the shallow feature map and the deep feature map through the hybrid attention feature fusion module in the codec to obtain the target detection result.

[0110] In this embodiment of the invention, optionally, the preset codec is a U-shaped codec, and the enhancement module is a visual converter attention mechanism module with added enhanced feedforward neural network.

[0111] In this embodiment of the invention, optionally, the second extraction module includes:

[0112] The first processing submodule is used to extract the first feature map of the target infrared image from the enhancement module in the codec, and encode the first feature map to obtain the first feature representation;

[0113] The second processing submodule is used by the enhancement module to process the first feature representation through a multi-head attention network to obtain the second feature representation;

[0114] The third processing submodule is used by the enhancement module to strengthen the second feature representation using an enhanced feedforward neural network to obtain a third feature representation;

[0115] The fourth processing submodule is used by the enhancement module to iteratively extract long-distance information tokens from the third feature representation to obtain the deep feature map.

[0116] In this embodiment of the invention, optionally, the third processing submodule includes:

[0117] The fifth processing submodule is used to restore the second feature representation into the first three-dimensional feature map through the first fully connected layer;

[0118] The sixth processing submodule is used to input the first three-dimensional feature map into the first convolutional layer for convolution processing, then into the normalization layer for normalization processing, and then into the second convolutional layer for convolution processing to obtain the second three-dimensional feature map.

[0119] The seventh processing submodule is used to fully connect the second three-dimensional feature map through the second fully connected layer to obtain the third feature representation.

[0120] In this embodiment of the invention, optionally, the first processing module includes:

[0121] The eighth processing submodule is used to obtain shallow feature maps of multiple levels of the codec, and to perform pyramid fusion on the shallow feature maps of multiple levels to obtain fused shallow features.

[0122] The ninth processing submodule is used to perform point-by-point and channel attention interaction between the fused shallow features and the deep feature map through the hybrid attention feature fusion module to obtain the target detection result.

[0123] In this embodiment of the invention, optionally, the hybrid attention feature fusion module includes: a point-by-point attention module, a channel attention module, and a shallow-deep feature summing module. The point-by-point attention module focuses on the infrared small target detail bounding box information from shallow to deep features and from bottom to top, outputting a fused shallow output feature map with strong local detail awareness. The channel attention module focuses on the relationship between the infrared small target background and the target from deep to shallow features and from top to bottom, outputting a deep global output feature map of important weight channels. The shallow-deep feature summing module sums the shallow features and the deep features to balance network features.

[0124] The target detection device provided in this embodiment of the invention can achieve Figure 1 The various processes implemented in the method embodiments achieve the same technical effect, and will not be described again here to avoid repetition.

[0125] This invention provides an electronic device 70, see [link to relevant documentation]. Figure 7 As shown, Figure 7 This is a schematic diagram of an electronic device 70 according to an embodiment of the present invention, including a processor 71, a memory 72, and a program or instructions stored in the memory 72 and executable on the processor 71. When the program or instructions are executed by the processor, they implement the steps in any of the target detection methods of the present invention.

[0126] This invention provides a readable storage medium on which a program or instruction is stored. When the program or instruction is executed by a processor, it implements the various processes of the target detection method embodiment as described above and achieves the same technical effect. To avoid repetition, it will not be described again here.

[0127] This application also provides a computer program product, including computer instructions, which, when executed by a processor, implement the above-described... Figure 1 The various processes of the method embodiments shown can achieve the same technical effect, and will not be described again here to avoid repetition.

[0128] Computer-readable media include both permanent and non-permanent, removable and non-removable media, which can store information using any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include transient computer-readable media, such as modulated data signals and carrier waves.

[0129] It should be noted that the collection, gathering, updating, analysis, processing, use, transmission, and storage of user personal information involved in this disclosed technical solution all comply with relevant laws and regulations, are used for legitimate purposes, and do not violate public order and good morals. Necessary measures are taken to prevent unauthorized access to user personal information data and to safeguard user personal information security and network security.

[0130] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0131] The sequence numbers of the above embodiments of the present invention are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0132] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions to cause a service classification device (which may be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in the various embodiments of this application.

[0133] The above description is only a preferred embodiment of this application. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principle of this application, and these improvements and modifications should also be considered within the scope of protection of this application.

Claims

1. A target detection method, characterized in that, include: Acquire a target infrared image, and extract a shallow feature map of the target infrared image using a preset codec, wherein the shallow feature map represents local detail features in the target infrared image; The deep feature map of the target infrared image is extracted by the enhancement module in the codec, wherein the deep feature map represents the deep relationship between small targets and the background in the target infrared image; The shallow feature map and the deep feature map are fused by the hybrid attention feature fusion module in the codec to obtain the target detection result.

2. The target detection method according to claim 1, characterized in that, The preset codec is a U-shaped codec, and the enhancement module is a visual converter attention mechanism module with an enhanced feedforward neural network.

3. The target detection method according to claim 1, characterized in that, The step of extracting the deep feature map of the target infrared image through the enhancement module in the codec includes: The enhancement module in the codec extracts a first feature map from the target infrared image and encodes the first feature map to obtain a first feature representation; The enhancement module processes the first feature representation through a multi-head attention network to obtain a second feature representation; The enhancement module uses an enhanced feedforward neural network to strengthen the second feature representation to obtain a third feature representation; The enhancement module iteratively extracts long-distance information tokens from the third feature representation to obtain the deep feature map.

4. The target detection method according to claim 3, characterized in that, The step of strengthening the second feature representation using an enhanced feedforward neural network to obtain the third feature representation includes: The second feature representation is restored to the first three-dimensional feature map through the first fully connected layer; The first three-dimensional feature map is input into the first convolutional layer for convolution processing, then into the normalization layer for normalization processing, and then into the second convolutional layer for convolution processing to obtain the second three-dimensional feature map. The second 3D feature map is fully connected through a second fully connected layer to obtain the third feature representation.

5. The target detection method according to claim 1, characterized in that, The process of fusing the shallow feature map and the deep feature map through the hybrid attention feature fusion module in the encoder-decoder to obtain the target detection result includes: The shallow feature maps of multiple levels of the codec are obtained, and the shallow feature maps of multiple levels are fused by pyramid fusion to obtain the fused shallow features. The hybrid attention feature fusion module performs point-by-point and channel-by-channel attention interaction between the fused shallow features and the deep feature map to obtain the target detection result.

6. The target detection method according to claim 5, characterized in that, The hybrid attention feature fusion module includes: a point-by-point attention module, a channel attention module, and a shallow-deep feature summing module. The point-by-point attention module focuses on the bottom-up details of infrared small target bounding boxes, from shallow to deep features, and outputs a fused shallow output feature map with strong local detail awareness. The channel attention module focuses on the top-down relationship between the infrared small target background and the target, from deep to shallow features, and outputs a deep global output feature map with important weight channels. The shallow-deep feature summing module sums the shallow and deep features to balance the network features.

7. A target detection device, characterized in that, include: The first extraction module is used to acquire a target infrared image and extract a shallow feature map of the target infrared image through a preset codec, wherein the shallow feature map represents local detail features in the target infrared image; The second extraction module is used to extract the deep feature map of the target infrared image through the enhancement module in the codec, wherein the deep feature map represents the deep relationship between small targets and the background in the target infrared image; The first processing module is used to fuse the shallow feature map and the deep feature map through the hybrid attention feature fusion module in the codec to obtain the target detection result.

8. An electronic device, characterized in that, It includes a processor, a memory, and a program or instructions stored in the memory and executable on the processor, wherein the program or instructions, when executed by the processor, implement the steps of the target detection method as described in any one of claims 1 to 6.

9. A readable storage medium, characterized in that, The readable storage medium stores a program or instructions that, when executed by a processor, implement the steps of the target detection method as described in any one of claims 1 to 6.

10. A computer program product, characterized in that, It includes computer instructions that, when executed by a processor, implement the steps in the target detection method as described in any one of claims 1 to 6.

Images