Cross-band infrared target detection method, device, computer equipment and storage medium

Abstract

The present application relates to a cross-band infrared target detection method, device, computer equipment and storage medium. Among them, each lateral connection layer of the multi-scale decoupling module is directly connected to the different scale features of the original target detection network; each lateral connection layer calculates the consistency loss of the deep feature statistics of the input source domain image and the target domain image; in addition, the style consistency loss and the content consistency loss are calculated and the gradient is fed back to the decoder for gradient back propagation. In summary, the feature extraction network with shared parameters simultaneously accepts the gradient signals of the top network of the target detection network, the multi-scale decoupling module and the feature decoder for optimization. Even if the unlabeled target domain image is not directly input into the target detection network, the existence of the weight-shared feature encoder and the multi-scale decoupling module enables the target detection network to learn the features of the target domain image. The present invention greatly improves the detection and recognition accuracy of infrared targets in domain adaptation, especially weak infrared targets.

Description

Technical Field

[0001] The present application relates to the field of target detection technology, and in particular to a cross-band infrared target detection method, apparatus, computer equipment, and storage medium. Background Art

[0002] Existing object detection and recognition methods typically use datasets from a single detection source as benchmarks for research and evaluation. This means that the training and test sets are acquired using the same equipment and have relatively uniform feature distributions. However, due to factors such as weather and seasonal variations and changes in detector performance, image features acquired in real-world scenarios can deviate significantly from those in the training dataset. This can lead to a significant performance degradation of data-driven object detection algorithms, known as domain shift. This domain-to-domain shift can manifest itself in differences in global statistical features such as image mean and variance, as well as shifts in grayscale and texture in local image details. Even subtle feature shifts can significantly interfere with the inference results of data-driven algorithms. In particular, noise that is difficult to detect and quantify can cause detectors to output erroneous results. This is especially true for small, dim infrared targets such as small drones, where feature discrepancies significantly impact the performance of detection and recognition algorithms. While collecting more images in the target domain for training could fundamentally address this issue, the time and labor required to annotate data makes this approach inefficient and cumbersome in engineering applications. Therefore, how to make full use of a small amount of unlabeled target domain data for effective model training and reduce the degradation of algorithm performance caused by feature offset is the key to the practical application of infrared target detection and recognition algorithms.

[0003] While the domain shift problem has been extensively studied in visible light data, there is currently no mature and reliable benchmark method for addressing this issue in infrared target detection and recognition. In particular, in the infrared small target detection and recognition task, the small size and unclear texture features of the targets make direct transfer of existing domain-adaptive target detection methods difficult. Specifically, in this task, targets appear as clustered speckles with weak semantics. Therefore, rigid, direct feature alignment methods can destroy the inherent features of the small targets. Especially when limited data is available, there may be a significant semantic gap between the source and target domains. Consequently, rigid alignment can disrupt the inherent feature representation of small targets. Specifically, due to the dominance of high-level semantics, the low-level texture corresponding to the target's local details and speckle patterns risk being overwhelmed. This ultimately leads to the loss and erroneous alteration of the inherent features of the small targets, resulting in negative transfer. Therefore, how to enhance the model's local patterns while preserving the semantic information of the local details during feature alignment is crucial for domain transfer in the infrared small target detection and recognition task. That is, it is necessary to realize the migration of global statistical features such as mean and variance, and to keep the local detail information of the image as much as possible, especially to keep the characteristic pattern of the infrared weak target intact. Summary of the Invention

[0004] Based on this, it is necessary to provide a cross-band infrared target detection method, device, computer equipment and storage medium to address the above technical problems, so as to improve the detection and recognition accuracy of domain-adaptive infrared targets, especially weak infrared targets.

[0005] A cross-band infrared target detection method, comprising:

[0006] Obtain a pre-built target detection network; the target detection network includes a feature extraction network;

[0007] Obtain a pre-built convolutional autoencoder; the convolutional autoencoder includes a feature encoder, a multi-scale decoupling module, and a feature decoder; the feature encoder and the feature extraction network share network parameters; the multi-scale decoupling module includes multiple horizontal connection layers, and the network layers with the same feature map resolution output by the feature decoder and the feature encoder are connected to each other through each horizontal connection layer;

[0008] The feature encoder calculates multi-scale feature maps of the unlabeled target domain image and the labeled source domain image. The deep feature statistics consistency loss of the target domain image and the source domain image is calculated through the horizontal connection layer. The feature maps of the corresponding scales are decoupled to obtain the de-stylized feature vector. The de-stylized feature vector is passed to the corresponding network layer in the feature decoder, and the feature decoder calculates the transferred image.

[0009] Input the source domain image into the object detection network and calculate the recognition loss of the object detection network. Calculate the style consistency loss and content consistency loss of the target domain image and the source domain image based on the transferred image, source domain image, and target domain image. Construct a joint loss function based on the deep feature statistics consistency loss, recognition loss, style consistency loss, and content consistency loss.

[0010] The trained target detection network is obtained by optimizing the joint loss function, and the trained target detection network is used for infrared target detection.

[0011] A cross-band infrared target detection device, comprising:

[0012] A detection network acquisition module is used to acquire a pre-built target detection network; the target detection network includes a feature extraction network;

[0013] The autoencoder acquisition module is used to obtain a pre-built convolutional autoencoder; the convolutional autoencoder includes a feature encoder, a multi-scale decoupling module, and a feature decoder; the feature encoder and the feature extraction network share network parameters; the multi-scale decoupling module includes multiple horizontal connection layers, and the network layers with the same feature map resolution output by the feature decoder and the feature encoder are connected to each other through each horizontal connection layer;

[0014] The style transfer module is used to calculate multi-scale feature maps of the unlabeled target domain image and the labeled source domain image based on the feature encoder, calculate the consistency loss of the deep feature statistics of the target domain image and the source domain image through the horizontal connection layer, and perform feature decoupling on the feature maps of the corresponding scale to obtain the de-stylized feature vector. The de-stylized feature vector is passed to the corresponding network layer in the feature decoder, and the feature decoder is used to calculate the transferred image;

[0015] A loss function construction module is used to input the source domain image into the object detection network and calculate the recognition loss of the object detection network. The style consistency loss and content consistency loss of the target domain image and the source domain image are calculated based on the transferred image, the source domain image, and the target domain image. A joint loss function is constructed based on the deep feature statistics consistency loss, the recognition loss, the style consistency loss, and the content consistency loss.

[0016] The loss function optimization module is used to optimize the joint loss function to obtain a trained target detection network, and use the trained target detection network to perform infrared target detection.

[0017] A computer device includes a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps in the above method when executing the computer program.

[0018] A computer-readable storage medium stores a computer program, which implements the steps of the above method when executed by a processor.

[0019] In the above-mentioned cross-band infrared target detection method, device, computer equipment and storage medium, each lateral connection layer of the multi-scale decoupling module is directly connected to the different scale features of the original target detection network, which can be regarded as a bridge structure connecting the target detection network and the multi-scale decoupling module; in order to further involve the features of the target domain image in feedforward and guide training, each lateral connection layer calculates the consistency loss of the deep feature statistics of the input source domain image and the target domain image. By aligning the high-order statistics of the deep features, the representation differences of the same content between different domains can be eliminated; in addition, the style consistency loss and content consistency loss are calculated and the gradient is fed back to the decoder for gradient backpropagation. In summary, the feature extraction network with shared network parameters simultaneously receives the gradient signals of the top network of the target detection network, the multi-scale decoupling module and the feature decoder for optimization. The top network of the detector only receives the gradient of the target (the labeled source domain image input in the training phase) detection task, and the convolutional autoencoder is optimized only under the constraints of content consistency loss and style consistency loss. That is to say, although the unlabeled target domain image is not directly input into the target detection network, the existence of the weight-shared feature encoder and the multi-scale decoupling module enables the target detection network to learn the features of the unlabeled target domain image, which is equivalent to indirectly participating in the forward reasoning of the target detection network. In summary, the present invention guides the feature extraction network of the target detection network to learn the image features of the target domain without labels through the image style transfer task from the source domain to the target domain, which greatly improves the detection and recognition accuracy of infrared targets in domain adaptation, especially infrared weak targets. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] Figure 1 1 is a flow chart of a cross-band infrared target detection method according to an embodiment;

[0021] Figure 2 A schematic diagram of the network structure of a horizontal connection layer in one embodiment;

[0022] Figure 3 Schematic diagram of the structure of a convolutional autoencoder in one embodiment;

[0023] Figure 4 FIG1 is a schematic diagram of evaluating a migration image in one embodiment;

[0024] Figure 5 : This is a cross-domain infrared small target detection image in one embodiment; (a) is a mid-infrared image, and (b) is a long-wave infrared image;

[0025] Figure 6Schematic diagram of comparison of a migration image, a source domain image, and a target domain image in one embodiment;

[0026] Figure 7 is a visualization output of an image migration task in one embodiment;

[0027] Figure 8 A comparison diagram of feature differences between a source domain image and a target domain image in a feature extraction network in one embodiment;

[0028] Figure 9 The detection and recognition visualization results of two sets of methods in this paper and the original benchmark network Yolov5-s in Set-A are shown;

[0029] Figure 10 FIG. 1 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0030] In order to make the purpose, technical solutions and advantages of this application more clear, the following further describes this application in detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain this application and are not intended to limit this application.

[0031] In one embodiment, Figure 1 As shown, a cross-band infrared target detection method is provided, comprising the following steps:

[0032] Step 102: Obtain a pre-built object detection network.

[0033] The target detection network includes a feature extraction network. The target detection network is responsible for basic target detection tasks. Considering the real-time requirements of the infrared search and tracking system, the Yolov5s target detector can be used. The Yolov5s target detector mainly consists of three parts: a convolutional feature extraction backbone network, a neck network, and a detection head. The network is responsible for the input source domain image I S ∈D s Perform feature extraction and predict object locations, bounding boxes, and confidence scores in images.

[0034] Step 104: Obtain a pre-built convolutional autoencoder.

[0035] The convolutional autoencoder includes a feature encoder, a multi-scale decoupling module and a feature decoder. The multi-scale decoupling module includes multiple lateral connection layers. The network layers with the same feature map resolution output by the feature decoder and the feature encoder are connected in pairs through each lateral connection layer.

[0036] The feature encoder and the feature extraction network share network parameters, that is, the feature extraction backbone network in the target detection network and the feature encoder in the task guidance network share features. It can be regarded as the target detection network and the convolutional autoencoder sharing a feature extractor. However, combined with the subsequent steps, it can be seen that the input of the feature extraction network is the labeled source domain image, while the input of the feature encoder is the labeled source domain image and the unlabeled target domain image.

[0037] Step 106: Obtain multi-scale feature maps of the unlabeled target domain image and the labeled source domain image according to the feature encoder, calculate the consistency loss of the deep feature statistics of the target domain image and the source domain image through the lateral connection layer, perform feature decoupling on the feature maps of the corresponding scales to obtain a de-stylized feature vector, and pass the de-stylized feature vector to the corresponding network layer in the feature decoder, and obtain the transferred image through the feature decoder.

[0038] It can be understood that the target domain image I T ∈D T and the source domain image I S ∈D s Participate in the forward propagation of the convolutional autoencoder together.

[0039] By introducing the consistency loss of deep feature statistics, the difference in feature representation of source and target domain image data by the feature extractor is narrowed in joint training, and explicit feature alignment is achieved. That is, by aligning the high-order statistics of deep features, the difference in representation of the same content between different domains is eliminated.

[0040] Feature decoupling refers to separating task-related features from irrelevant features. In the style transfer task of the convolutional autoencoder, the lateral connection layer not only transfers features but also separates the content information and style information of the feature map of the corresponding scale, thereby obtaining a de-stylized feature vector.

[0041] Due to the existence of the lateral connection layer, in the feature decoder, except for the last network layer responsible for integrating the transferred image, the input of each network layer includes not only the output of the previous network layer, but also the de-stylized feature vector passed through the lateral connection layer. It can be seen that the introduction of the lateral connection layer realizes the communication between the low-dimensional features in the feature encoding layer and the high-dimensional features in the feature decoding layer, avoiding the risk of the detailed features of weak infrared targets being submerged by the background during feature aggregation during the style transfer process.

[0042] In step 108, the source domain image is input into the target detection network and the recognition loss of the target detection network is calculated. The style consistency loss and content consistency loss of the target domain image and the source domain image are calculated based on the migration image, the source domain image and the target domain image. A joint loss function is constructed based on the deep feature statistic consistency loss, the recognition loss, the style consistency loss and the content consistency loss.

[0043] Considering the style consistency loss and content consistency loss of the target domain image and the source domain image, it can ensure that the features of each level output by the feature encoder can maintain the detailed features of the weak target, and the output migration image also has the source domain image I S Content information and target domain image I T That is, while retaining the semantic content of the source domain image, the input de-stylized feature vector is reconstructed into a migration image that is close to the target domain image style.

[0044] At the same time, since the feature encoder and the feature extraction network share network parameters, although the unlabeled target domain image is not directly input into the target detection network, due to the existence of the feature encoder and the lateral connection layer, the features of the target domain image can participate in the feedforward of the target detection network and guide its training.

[0045] Step 110: Optimize the joint loss function to obtain a trained target detection network, and use the trained target detection network to perform infrared target detection.

[0046] A natural approach to content-preserving feature transfer is image style transfer, an approach used in some image-to-image translation work. However, these methods typically rely on additional image generation modules to achieve translation between the source and target domains. On the one hand, online translation-based methods impose additional computational burdens, reducing the algorithm's real-time performance. On the other hand, the target domain image typically serves only as input to the independent image generation module and does not participate in the detector's forward reasoning, meaning the detector itself has never actually seen the target domain image.

[0047] In the above-mentioned cross-band infrared target detection method, each lateral connection layer of the multi-scale decoupling module is directly connected to the different-scale features of the original target detection network, which is a bridge structure linking the target detection network and the multi-scale decoupling module. In order to further involve the features of the target domain image in feedforward and guide training, each lateral connection layer calculates the statistical consistency loss of the deep features of the input source domain image and the target domain image. By aligning the high-order statistics of the deep features, the representation differences of the same content between different domains can be eliminated. In addition, the style consistency loss and content consistency loss are calculated and the gradient is fed back to the decoder for gradient backpropagation. In summary, the feature extraction network, which shares network parameters, simultaneously receives gradient signals from the top network of the target detection network, the multi-scale decoupling module, and the feature decoder for optimization. The top network of the detector receives only the gradients of the target detection task (the labeled source domain image input during the training phase). The convolutional autoencoder is optimized only under the constraints of content consistency loss and style consistency loss. In other words, although the unlabeled target domain image is not directly input into the target detection network, the weight-shared feature encoder and the presence of the multi-scale decoupling module enable the target detection network to learn the features of the unlabeled target domain image, which is equivalent to indirectly participating in the forward reasoning of the target detection network. In summary, this method is equivalent to guiding the training of the detection network feature extractor by aligning the image style transfer task with statistical features, greatly improving the detection and recognition accuracy of infrared targets in domain adaptation, especially small infrared targets.

[0048] In one embodiment, each lateral connection layer includes a residual block portion, an identity layer bypass, and an instance normalization layer. De-stylized feature vectors are obtained by performing feature decoupling on feature maps of corresponding scales through the lateral connection layer, including:

[0049] Obtain the feature maps of the corresponding scale and input them into the residual blocks in the corresponding lateral connection layer and the identity layer bypass. Based on the residual blocks, calculate the feature statistics of each feature map at the corresponding scale. Feature statistics include feature mean and feature variance.

[0050] The feature statistics and the feature map in the identity layer bypass are used to obtain the input vector of the instance normalization layer. The input vector is normalized according to the instance normalization layer to obtain the de-stylized feature vector.

[0051] It can be understood that the process of feature decoupling is specifically implemented through instance normalization operation. Let the input feature be x∈(bs, c, w x , h x ), where bs is the number of input image batches, c is the number of channels, (w x , h x ) is the spatial resolution of the feature, then the operation process of the horizontal connection layer is as follows:

[0052] First, the input feature set with batch size bs is split into single image feature samples x k , k[1, bs], calculate the characteristic statistics of each sample respectively, record Res n (·) is the residual block in the nth horizontal connection layer, then:

[0053]

[0054]

[0055] in, and They are the feature mean and feature variance of the kth feature of the input nth horizontal connection layer, respectively. As a space-independent global statistic, the feature statistic can be used as the style representation of the kth feature of the input in the nth horizontal connection layer.

[0056] Furthermore, the instance features are normalized according to the statistic to obtain the de-stylized feature vector:

[0057]

[0058] Among them, CDL n (x k ) is the output of the nth horizontal connection layer to the k instances in its input batch, ε is the correction factor, and ε can be a very small scalar 1×10 -9 , to avoid the denominator being 0.

[0059] like Figure 2 As shown in Figure 1, a schematic diagram of the network structure of the lateral connection layer is provided. Since the convolutional autoencoder and the object detection network perform parameter optimization simultaneously during joint training, the unlabeled target domain image cannot participate in the forward pass relying solely on the convolutional autoencoder. This means that the feature extraction network with shared parameters does not directly utilize the target domain data for forward operations during training. The target domain image only participates in network optimization indirectly through the additional style transfer task. This implicit learning without direct constraints makes it difficult to maintain the stability of target domain feature learning. Therefore, this method considers adding a new mechanism, namely the CDB module, to the lateral connection layer of the convolutional autoencoder. This can obtain the feature information of the target domain image during the forward operation of the convolutional autoencoder, and achieve direct alignment of style features that are independent of the content of the target domain image, thereby constraining the optimization process of joint learning.

[0060] from Figure 2It can be seen that its main body is a residual structure with bottleneck convolution. An additional IN layer (instance regularization layer) and two output branches are introduced at the top layer of the residual block to output the content information and style information after feature decoupling respectively. The residual block part is divided into a bottleneck structure consisting of two 1×1 convolutions for channel integration and a 3×3 convolution layer for channel preservation, and an Identity layer (identity layer bypass). The two are added by corresponding elements and then output to the instance regularization layer. The instance regularization layer specifically uses the statistical information of a single input sample to perform feature normalization. Since the residual block part maintains the dimension of the input feature batch, the dimension of the normalized result x′ remains (bs, c, w x , h x In order to maintain the content details of the source domain image in the image style conversion task, the normalized output x′ is used as the content information of the source domain image and directly passed to the corresponding level of the decoder through the horizontal connection layer to achieve style transfer.

[0061] The statistical offset between the source domain image and the target domain image is output through another channel, and the loss calculation of minimizing the difference is performed. That is, the deep feature statistics of the source domain image and the target domain image are used as the optimization target. The specific form is:

[0062]

[0063]

[0064]

[0065] Among them, the binary statistics consistency loss Lssc(I S , I T ) is the mean of the binary statistic error values of a set of source domain images and target domain images in the corresponding layer, and Represent the source domain image I S and the target domain image I T After the feature extraction network (feature encoder) ED(·), the depth is n f The output features of the feature layer after passing through the residual layer in the corresponding horizontal connection layer, and Represents the mean square error of the binary statistics of the two features. Since the instance normalization operation is performed separately on each horizontal connection channel, the mean square error of the statistics of each channel is taken here:

[0066]

[0067] Among them, N fCorresponding to the access node of the multi-scale decoupling module and the feature extraction network, in order to achieve multi-scale feature alignment of the source domain and target domain images, we use the downsampling layer of the original feature extraction network to calculate the binary statistical error of the deep features at each level.

[0068] In one embodiment, the feature decoder is used to calculate the migration image, including:

[0069] In the feature decoder, the output vector of the current network layer is calculated based on the feature vector transmitted by the previous network layer and the de-stylized feature vector transmitted by the corresponding horizontal connection layer until the migration image is output.

[0070] like Figure 3 As shown in Figure 2, a schematic diagram of the convolutional self-encoder is provided. Figure 3 In the convolutional autoencoder, the feature encoder adopts the backbone network architecture of Yolov5-s, which mainly consists of five convolutional layers and four bottleneck layers. The layer achieves feature aggregation through 2x stride convolution, and finally The output of the layer is downsampled by 32 times, while the number of feature channels increases step by step. The input and output of each bottleneck layer keep the number of channels and feature resolution unchanged. In order to restore the initial resolution of the image and obtain the migrated image, this method uses the dual form of the feature encoder to construct the feature decoder. After the convolution layer of the feature decoder, feature upsampling can be achieved through bilinear interpolation. The feature resolution of the decoder convolution layer output is one-to-one corresponding to the corresponding encoder convolution layer. The layer restores the original resolution of the input image, and the last layer of the decoder It is an independent convolutional layer and is not connected to the feature encoder features through the horizontal connection layer. The layer is responsible for integrating and outputting the migration image I of the original resolution ST Convolutional Feature Encoder represents the convolutional feature encoder, Multi-scale Content Decoupling Block represents the multi-scale semantic decoupling module, Symmetrical Decoder represents the feature decoder with a symmetrical structure to the feature encoder, Second-order Statistical Consistency Loss represents the deep feature statistical consistency loss, Detection Loss represents the recognition loss of the target detection network, CDB represents the semantic decoupling module of the corresponding scale, which has been integrated in each horizontal connection layer, Data Forwarding represents the forward propagation of image data, Loss calculation represents the loss value calculation, and Bilinear interpolation represents bilinear interpolation.

[0071] Figure 3 The parameters of each network layer of the convolutional autoencoder in are shown in Table 1, where the 1st to 9th layers are the weight-shared feature extraction backbone network, the 10th to 15th layers are the feature decoders, Depth is the network depth, Layers is the network layer number, Type is the network layer type, Kernel size is the convolution kernel size, Strides is the convolution kernel step size, Width is the number of network layer channels, Output size is the network layer output feature resolution, Down-sampling is the downsampling multiple, and the "√" under the Detectionlayer column indicates that the network layer is connected to the detection head of the target detection network Yolov5-s.

[0072] In the calculation of the consistency loss of deep feature statistics shown in Table 1, N f =[1, 3, 5, 7, 9], i.e. the 1st, 3rd, 5th, 7th, and 9th layers, corresponding to Figure 3 of As shown in Table 1, The layer is the detection scale of the target detection network, and its features are connected to the neck of the target detector, so the guided optimized features will be directly used for forward reasoning of the top network of the detector.

[0073] Table 1 Detailed parameter settings of each layer of the convolutional autoencoder network

[0074]

[0075] In addition, the low-level convolutional features of the encoder are connected to the high-level features of the encoder of the corresponding resolution through the lateral connection layer, which can be expressed as:

[0076]

[0077] Among them, Cat represents the channel connection operation of the feature map, Conv represents the convolution operation with a convolution kernel size of 3×3 and a stride of 1, combined with the BN layer and the ReLU layer, and Upsample represents the 2x upsampling achieved by bilinear interpolation. It is an independent 5×5 convolution layer that is not connected to the encoder feature. This layer is responsible for integrating and outputting the migration image I of the original resolution. ST , the output migration image I ST Should have both source domain image I S Content information and target domain image I T Image style information.

[0078] In one embodiment, inputting a source domain image into an object detection network and calculating the recognition loss of the object detection network includes:

[0079] Input the source domain image into the target detection network and calculate the positioning loss of the target detection network:

[0080] L loc =1-GIoU

[0081]

[0082] Among them, L loc is the positioning loss, GIoU is the generalized intersection-over-union ratio, c s-g is the ground-truth bounding box s g and the predicted bounding box s t The smallest convex set between them, that is, the one that encloses s g 、s t The minimum bounding rectangle of

[0083] Calculate the confidence loss of the target detection network, and calculate the binary cross entropy of the sample pairs obtained by matching the positive samples:

[0084]

[0085] Among them, L obj is the confidence loss, Output the probability of the target box as the target for the target detection network, o c A Boolean value indicating whether the predicted value is a target. If it is a target, it is set to 1; if it is background, it is set to 0.

[0086] Calculate the classification loss of the object detection network using multi-class cross entropy loss:

[0087]

[0088] Where, L cls is the classification loss, is the predicted value of the i-th target output by the network, is a Boolean value indicating whether the target is of the i-th category, and M is the number of predicted target categories.

[0089] The recognition loss of the target detection network is obtained based on the positioning loss, confidence loss and classification loss:

[0090] L DET (I S )=γ l L loc +γ o L obj +γ c L cls

[0091] Among them, L DET (I S) is the recognition loss of the target detection network for the input source domain image, which is only caused by the source domain image I S and its label value, including positioning loss, confidence loss and classification loss. l is the weight parameter of the positioning loss, γ o is the weight parameter of confidence loss, γ c is the weight parameter of the classification loss, which can be set to γ l =0.05,γ o =1,γ c =0.5.

[0092] In one embodiment, calculating the style consistency loss and content consistency loss of the target domain image and the source domain image based on the migration image, the source domain image, and the target domain image includes:

[0093] Calculate the style consistency loss of the target domain image and the source domain image based on the migration image and the source domain image:

[0094]

[0095] Among them, VGG(·) represents the evaluation network, which does not update parameters during training. TGN(·) represents the convolutional autoencoder for task guidance. S ) is the migration image I generated based on the source domain image ST .

[0096] The content consistency loss of the target domain image and the source domain image is calculated based on the migration image and the target domain image. In order to maintain the content information of the weak target in the feature alignment process, the content consistency loss is used to constrain the joint training process. This part of the loss is only caused by the source domain image I S Participating calculations:

[0097]

[0098] In order to supervise the joint learning process of the Task Guidance Network style transfer task, an independent VGG-16 feature extraction network is used as the transfer image evaluation network and participates in the loss calculation of the joint training. After the Image-Net pre-training, the evaluation sub-network removes the last three fully connected classification layers and only retains the feature extraction function. It does not participate in the gradient back propagation during training. The specific evaluation process is as follows Figure 4 As shown, first, the source domain image I S , target and image I T , migration image I ST Input the feature extractor (Feature Extractor VGG-16) to obtain the feature matrix F of each levelS 、F T 、F ST , and then extract the deepest feature maps at each resolution, that is, the ReLU activation output feature maps of the 2nd, 4th, 7th, 10th, and 13th convolutional layers, respectively, to obtain feature matrices, which are used as the basis for measuring the content and style similarity between the transferred image and the source domain image and the target domain image:

[0099]

[0100] In order to ensure the consistency of migration and content preservation of features at each level in the guidance task, we use the 5 downsampling layers of the evaluation network VGG-16 as nodes for L Gram (I S , I T ) and L Content (I S ), that is, Nl = [2, 4, 7, 10, 13],

[0101] In terms of content similarity, an additional instance normalization layer is used to normalize the source domain features. and transfer image features Perform content feature separation to obtain normalized style-independent content information and

[0102]

[0103]

[0104]

[0105]

[0106] Among them (w n , h n ) represents the size of the feature map. Since the content information is position-dependent, the mean square error between the two is used as a measure of the consistency of the content information:

[0107]

[0108] In terms of style consistency, a feature space for capturing texture information is used to obtain the style representation of the input image. This method uses the Gram matrix to obtain the correlation representation between the filter responses of the input image in the convolution layer. Assume that the input image I is in the nth layer feature map F in the VGG network. n The scale is c×i×j, where c represents the number of channels and i×j is the spatial resolution. The Gram matrix calculation process is:

[0109]

[0110]

[0111] hf n =[F n11 ,F n21 ,F n31 ,...,F nij ] T

[0112] Among them, F nxy F n The element of length c at the spatial position (x, y), wf n With hf n They represent the spatial vectorization matrices of the input image in the horizontal and vertical directions of the n-th layer feature map in the VGG network. The Gram matrix of the n-th layer feature map is wf n With hf n The inner product, whose mathematical expectation value is calculated within the spatial range of the feature resolution of the layer, is a position-independent quantity. Therefore, by eliminating the Gram matrix differences between features, we can achieve the task of guiding the network to generate images closer to the target domain features. This is achieved by minimizing the mean square error. The difference at the nth layer can be expressed as:

[0113]

[0114] This feature space can be constructed on the response layer of any convolutional layer in the VGG network. Therefore, we set n to 2, 4, 7, 10, or 13, and extract the Gram matrix of features at different levels to obtain a multi-scale feature representation of the input image.

[0115] Easy to find, MSE Content (F S , F ST ) and MSE Gram (F T , F ST ) is differentiable for the convolutional activation layer at the top of the task-guided network, so these two parts of the loss participate in the gradient backpropagation during training, and the feature extraction part of the shared parameters is optimized simultaneously in the joint training with the detection network.

[0116] In summary, the convolutional autoencoder serves as the task-guiding branch that shares the feature extraction network parameters with the object detector, and is jointly trained end-to-end with the original object detector during the training phase. The feature extraction network, which shares parameters, is optimized simultaneously by the gradient signals from the detector's top network (i.e., the feature extraction network), the multi-scale decoupling module, and the decoder. The detector's top network only receives the gradient of the object detection task. The convolutional autoencoder is optimized only under the constraints of the content consistency loss and style consistency loss generated by the evaluation network. The evaluation network does not update parameters during training. Therefore, the total loss function for the joint optimization can be expressed as:

[0117] L total (I T ,I S )=ω D L DET (I S )+ω S L SSC (I T ,I S )+ω G L Gram (I T ,I S )+ω C L Content (I S )

[0118] In order to balance the influence of each task in the total loss of joint optimization, the weight index can be set as follows: D =1,ω S =5×10-4,ω G =1×10-4,ω C =5×10-5.

[0119] Here, a specific embodiment is used to experimentally analyze and verify this method.

[0120] 1. Implementation details and training methods

[0121] This experiment was implemented using Python 3.7 and the PyTorch framework. Training and testing were performed on a workstation equipped with a 4-GPU NVIDIA GeForce 2080s GPU. Stochastic gradient descent was used for model optimization. Training lasted approximately 500 iterations, with a batch size of 8 and an initial learning rate of 1 × 10⁻³. The learning rate decreased linearly with the number of iterations, decreasing by a factor of 100. For the other deep learning methods used in the comparative experiments, the same equipment and optimization methods were used for training with different network configurations.

[0122] 2. Cross-domain infrared small target detection dataset

[0123] In order to meet the training and testing requirements of the algorithm, based on the medium-wave infrared small target dataset MSUAV dataset, infrared low-altitude drone data was obtained in different cities through long-wave infrared detectors as an additional subset, thus forming an infrared small target detection cross-domain dataset. Specifically, the dataset contains two subsets, where subset Set-A is 3840 medium-wave infrared image data and subset Set-B is 1600 long-wave infrared image data. Each subset is divided into training set and test set in a uniform ratio of 7:3. The schematic diagrams of the two subsets are as follows: Figure 5 (a) with Figure 5 (b) shows that both subsets are normalized to 640×640, 8-bit infrared grayscale images. The dataset contains various drones approaching detection from far to near at different distances, angles, and flight attitudes. The target annotation information includes: category (Ph, Mav), center point coordinates (x, y), and bounding box width and height (w, h).

[0124] To quantitatively characterize the feature differences between the cross-domain subsets, image-level statistical analysis was performed on the two subsets, specifically including image average grayscale, average contrast, and average sharpness. Average sharpness is measured using the absolute value variance of the Laplace operator, with larger values representing clearer images and smaller values representing blurrier images. Detailed statistical information for the two cross-domain subsets is shown in Table 2. The statistical results show significant statistical differences in image-level features between the two datasets due to different detector architectures and dataset acquisition scenarios.

[0125] Table 2 Statistics of two cross-domain infrared dim target datasets

[0126]

[0127] In the ablation experiment, considering that Set-A is relatively large, the training set images and their label files of Set-A are used as the source domain training data. To simulate the situation where target domain data is insufficient in application scenarios, 10% of the images in the training set of Set-B are selected as the target domain training data (excluding the corresponding labels). The validation set of Set-B (with the true values corresponding to the labels) is used as the test set for the cross-domain detection task, that is, cross-domain detection from A to B.

[0128] In addition, a cross-validation experiment was set up in the comparative experiment. In addition to the above settings, a B→A cross-domain detection control group was set up. The training set images and label files of subset B were used as source domain training data, and 10% of the images in the training set images of subset A were selected as target domain training data. The test set of the B→A cross-domain detection task is the validation set of subset A.

[0129] In terms of data augmentation, in order to ensure the fairness of the comparative experiments, the same data augmentation method Mosaic as the original baseline algorithm Yolov5 was used in the training phase, and the augmented objects only included the labeled source domain training set.

[0130] 3. Ablation Experiment and Visualization Analysis

[0131] In order to evaluate the effectiveness of this method for cross-domain infrared dim target detection, we studied the optimal configuration of the network structure and parameters and conducted comprehensive ablation experiments, specifically targeting the following factors:

[0132] Q1: As an additional branch, can the convolutional autoencoder effectively improve the performance of the existing single-stage target detection baseline network in the cross-domain infrared weak target detection task?

[0133] Q2: As the main optimization target, the backbone network of the original detection network and the encoder of the convolutional autoencoder share parameters. Which specific layers should be selected for joint optimization to achieve the best performance?

[0134] Q3: In the convolutional autoencoder, the additional lateral connection components enable the feature extraction network to communicate directly with the decoder of the task guidance network. Can this improve the guiding role of the style transfer task for the object detection task?

[0135] Q4: Is the structure of the multi-scale feature decoupling layer effective as an additional component of the task-guided network?

[0136] Q5: Deep feature statistic alignment is a direct alignment method between source domain images and target domain images. Can the additional introduction of deep feature statistic consistency loss increase the performance of cross-domain detection tasks?

[0137] The quantitative evaluation indicator of the experiment is the average precision (AP), which is the area under the precision-recall curve (PR) of a certain class of detection. The specific expression is as follows:

[0138]

[0139] Where p is the detection accuracy, r is the recall rate, TP is the true positive, FP is the false positive, and FN is the false negative. The average AP of each target is the class average accuracy (mAP). The criterion for determining a true positive is that the IOU between the predicted box and the true target box is greater than the threshold value τ. When the threshold value τ = 0.5, mAP is obtained. 0.5 , taking 0.5-0.95 (interval 0.05) to get the class average accuracy mAP under each threshold 0.5:0.95,The larger the class average accuracy is, the better the detection performance of the algorithm.

[0140] 3.1 Effectiveness Analysis

[0141] To answer questions Q1 and Q2, verify the effectiveness of our method, and quantitatively evaluate the gains it generates over existing baseline networks, we conducted a control experiment against the original object detection network. The experimental group used our CDFA-Net method. Since the object detector portion of CDFA-Net is consistent with the Yolov5-s architecture, the control group used the original Yolov5-s object detection network. The loss function settings and weights of each sub-loss for the object detection task remained consistent between the experimental and control groups.

[0142] In deep networks, shallow features mainly represent low-dimensional details such as texture and edges of images. As the number of network layers increases, features are aggregated layer by layer, feature resolution decreases, and the information represented becomes more abstract, with stronger semantics than shallow features. In the cross-domain infrared weak target detection task, considering the small size and weak semantics of the target itself, whether the alignment of features at each level can positively improve the detection performance is a question that must be considered. Therefore, for question Q2, the experimental group CDFA was set up respectively. s2 -CDFA s5 , whose subscript represents the level of shared parameters, specifically corresponding to the N in the above content f Considering the module integrity of the original Yolov5-s backbone network, the minimum shared unit of the shared layer in the ablation experiment contains at least one convolutional layer and a bottleneck layer structure, namely the model CDFA in the table s2 , represents the parameter sharing up to the second downsampling layer, the shared layer is CDFA s3 -CDFA s5 And so on.

[0143] In the experiments, Yolov5-s and various CDFA-Net configurations were trained on the object detection task using the training set of Set-A and tested on the test set of Set-B. The experimental results are shown in Tables 3 and 4. Table 3 shows the average accuracy of CDFA-Net and the original Yolov5-s in cross-domain scenarios, with different numbers of shared backbone network layers. The experimental results show that compared with the original network, the CDFA-Net with the added task-guided semantically decoupled feature alignment method significantly improves the average accuracy of all categories in cross-domain scenarios. Compared with the original Yolov5-s, the CDFA method learns feature representations for unlabeled target domain data through an additional unsupervised low-level vision task and guides feature learning of the original network through weight sharing. Therefore, the physical structure only adds independent additional branches to the original network, without changing the basic structure of the original detection network. Therefore, the experimental results demonstrate that the addition of the CDFA method significantly improves the performance degradation of single-stage object detectors in cross-domain scenarios for infrared dim target detection and recognition. Moreover, as the number of shared layers increases, the improvement of network performance gradually increases, but the growth rate gradually decreases. s5 Under this configuration, the model's mAP 50 The value increased by 18.3%, mAP 95 The improvement reached 15.4%, and the number of shared layers is positively correlated with the performance improvement. This phenomenon shows that in each layer of the feature extraction network, guided learning from the target domain feature representation can bring gains to cross-domain target detection and recognition tasks. In the shared features, Since the features of these three levels are directly involved in the feature fusion of the neck network and the forward prediction of the detection head through the forward path, these features will directly affect the reasoning results of the network. s5 The shared layer contains the feature layers of all three connected detection heads, and the best results were achieved in the ablation experiment of the shared layer. The fact that the performance growth gradually slows down with the increase of the number of shared layers reflects the importance of low-level features in infrared dim target detection.

[0144] To further verify the above conclusions, Table 4 shows the comparison of the accuracy and recall of each network configuration when the confidence threshold is 0.5. The experimental results show that the detection and recognition indicators of each configuration for each category are higher than the original one, and when the CDFA is configured s5 Under this circumstance, the precision and recall rate indicators of each category achieved the best balance.

[0145] Table 3 Comparison of cross-domain detection and recognition performance between CDFA-Net and Yolov5-s under different numbers of shared layers

[0146]

[0147] Table 4 Comparison of precision and recall between CDFA-Net and Yolov5-s when the confidence threshold is 0.5

[0148]

[0149] In order to further observe the traction effect of the CDFA method in cross-domain tasks, the image migration task of the task guidance module is visualized. Figure 6 As shown, the task-guided network transforms the source image as input, generating a transferred image that is visually closer to the target image in style. The morphological features and detail representation of the infrared drone target are largely preserved. To quantitatively analyze the effectiveness of image transfer, statistical features of the source, target, and transferred images were analyzed. Table 5 shows that the grayscale mean, contrast, and image sharpness of the transferred image are smaller than those of the target image. This means that even when there are significant statistical differences between the source and target images, the task-guided network ensures that the global feature representation of the transferred source image is closer to that of the target domain after transfer. While the transferred image retains the content information of the source image, the semantic differences still lead to a certain gap between the statistical characteristics of the transferred image and the target domain. This demonstrates that the trained network can effectively learn the feature representation of the source image while fully preserving its content information.

[0150] Table 5 Comparison of statistical features of source domain images, target domain images, and transferred images

[0151]

[0152] 3.2 Effectiveness Analysis of Semantically Decoupled Feature Alignment Module

[0153] The multi-scale semantic decoupling module is a key component of the task-guided sub-network. This component realizes direct communication between the encoder and decoder of the task-guided network through the lateral connection of the content feature decoupling layer CDL on the multi-scale feature layer. This structure can realize the bypass forward transmission of multi-scale features, which enables the reverse gradient of the task-guided sub-network to be directly transferred from the decoder to the corresponding scale of the feature sharing layer, thereby strengthening the feature guidance role in joint learning.

[0154] Table 6 CDFA-Net ablation experiment model structure settings for each element

[0155]

[0156]

[0157] In order to answer questions Q3-Q4 and verify the gain effect of this module on the overall network, ablation experiments were conducted on the core elements of this part, mainly including the horizontal connection structure, the content feature decoupling layer CDL module and its multi-scale structure. s5 , set up a CDFA without horizontal connection group noncat s2 -CDFA noncat s5 , and CDFA without CDL module group nonCDL s2 -CDFA nonCDL s5 The subscripts s2-s5 represent the number of shared feature layers, and the setting method is the same as that in Section 5.3.2. For Q5, an experimental group was designed for the consistency loss of deep feature statistics derived from the CDL module. The model of this experimental group is labeled CDFA. nonsoc s2 -CDFA nonsoc s5 , its network structure is the same as that of the full-factor group, but the deep feature statistic consistency loss is not included in the calculation during training, and the other loss weights remain unchanged. The specific experimental settings are shown in Table 6.

[0158] A. Horizontal connection structure and content feature decoupling layer

[0159] The content-feature decoupling layer exists in the lateral connection branch of the task-guided network, so these two parts are evaluated separately. The ablation experiment results are shown in Tables 7 and 8.

[0160] Table 7 Comparison of ablation experimental results of lateral connection structure and content feature decoupling layer at different sharing levels

[0161]

[0162]

[0163] The experimental results show that, firstly, the model CDFA noncat s2 It represents the addition of the basic structure of the task-guided network to the original detection model. Compared with the original detection network, the various indicators of the model have been improved to varying degrees, which proves the effectiveness of the task-guided unsupervised transfer method for infrared dim target detection tasks. Secondly, the longitudinal comparison of the horizontal connection group CDFA noncat s2 -CDFA noncat s5 and CDL module group CDFA nonCDL s2 -CDFAnonCDL s5 From the various performance indicators, it can be found that the more levels of shared parameters, the greater the improvement in performance indicators. Among them, the performance of s4 and s5 is significantly better than that of s2 and s3. This is the same as the conclusion of Section 5.3.1. The reason may be that the former contains more detection layer features. In Yolov5-s, the three spatial resolutions of the detection layer are 80×80, 40×40, and 20×20. The 20×20 feature map of the deepest level has been downsampled 32 times. Therefore, compared with other layers, its spatial accuracy is significantly lower, and its contribution to the reasoning of small target detection is weaker. The above reasons may have led to the performance indicators of the s4-s5 models being relatively close. Third, horizontal comparison of the CDFA of the horizontal connection group noncat CDFA with CDL module group nonCDL The experimental results show that, under the same shared layers, the performance indicators of the model containing the CDL module are generally higher than those of the experimental group with only horizontal connections, which proves that the addition of the CDL module has brought significant gains to the overall performance of the network.

[0164] Table 8 Comparison of precision and recall under different settings when the confidence threshold is 0.5

[0165]

[0166]

[0167] In this method, the multi-scale CDL module provides an additional content transmission path for the autoencoder structure. This module transfers the multi-scale content information in the feature extraction network to the decoder structure, maintaining the consistency of the image content information in the task-guided sub-network in the encoder and decoder, especially ensuring that the underlying features of infrared weak targets will not be lost due to the reduction of feature resolution caused by convolution downsampling. It is found that the introduction of the CDL method greatly improves the convergence speed of the network, especially the convergence speed of the task-guided network. Figure 7 The image migration task of the task guidance module in the training phase is visualized as shown in the figure, where CDFA noncat s5 Does not contain MCD structure, CDFA s5It is a complete structure containing five layers of MCD. In the figure, in each pair of images, the image on the left side is the migration image generated by the task-guided network under different numbers of iterations, and the image on the right side is the corresponding original training image, that is, the network input image after Mosaic enhancement during the training phase. Through observation, it is found that when there is no CDL module, the shape of the generated image is quite different from the original image under the same number of iterations, the background and the target have certain distortion in details, and the quality of the migration image improves slowly with the increase of the number of iterations; when the CDL method is introduced, the additional image migration sub-network achieves a better migration effect with fewer iterations, the image content maintains a high degree of consistency with the original image, and the details of the infrared weak target remain more complete. This means that the network parameter convergence path is more reasonable, and the feature representation of the target domain can be learned in a shorter optimization time.

[0168] B. Deep Feature Statistics Consistency Loss

[0169] Another important function of the CDL module is to extract multi-level style features from source domain images and unlabeled target domain images. These features represent the high-order characteristics of content-independent deep features. By eliminating the difference in feature quantities between the two domains, content-preserving feature domain alignment can be achieved. This alignment process is specifically tested by calculating the consistency loss of deep feature statistics separated by the CDL module. In order to answer question Q5, quantitative evaluation and verification of the impact of the consistency loss of deep feature statistics on the network are performed, and an ablation experiment of the consistency loss of deep feature statistics at different shared levels is set up. The models of this experimental group are labeled CDFAnonsocs2-CDFAnonsocs5. As shown in Table 6, this experimental group only loses L in the training stage compared with the complete CDFA-Net. SSC The weight ω S The rest of the experimental settings remained the same.

[0170] The experimental results, shown in Tables 9 and 10, show that the full CDFA module, with the deep feature statistics consistency loss added during training, shows improvements across all performance metrics at all shared layers compared to the model without it, with a relatively uniform improvement across all layers. Furthermore, it can be observed that adding this loss during training achieves a better balance between recall and precision. Essentially, the content feature decoupling layer decouples the semantically irrelevant features of the input features through instance normalization, enabling feature extraction and cross-domain feature variance measurement from unlabeled target domain data, thereby enabling unsupervised feature learning and alignment. Ulyanov et al. have demonstrated that instance normalization can effectively eliminate the variance in style features between input instances, but it also weakens the inherent characteristics of the instances. Therefore, while this method works well for low-level visual tasks such as style transfer, it performs poorly for image segmentation and object detection and recognition tasks. In this method, instance normalization is used as a means to separate content and style information. A deep feature statistical consistency loss achieves unsupervised learning of target domain features solely through alignment of style feature parameters, without altering the information flow in the forward detection pathway, thereby preserving the inherent characteristics of the target. Furthermore, by integrating the method into an additional subnetwork branch, the detector structure remains unchanged, thereby maximizing the robustness of existing mature models for detection tasks. A small number of available unlabeled target domain images are allowed to participate in the forward propagation and loss calculation, achieving robust feature alignment through task guidance. The algorithm only participates in the optimization as a side-channel guidance signal, constraining the convergence path of the joint learning.

[0171] Table 9 Comparison of network performance without loss of consistency of deep feature statistics under different sharing levels

[0172]

[0173] Table 10 Comparison of accuracy and recall with and without loss of consistency of deep feature statistics when the confidence threshold is 0.5

[0174]

[0175] In order to further visualize the degree to which this method learns the features of the target domain image, the feature differences between the source domain image and the target domain image in the feature extraction network are extracted and calculated. In order to exclude the influence of semantic differences, the Gram distance is used to measure the stylization difference between the two domains. Figure 8 As shown in the figure, the left bar graph represents the L-1 distance between the average value of the Gram matrix of each feature layer obtained by inputting the source domain test set image and the target domain test set image in the original detection network, and the right bar graph represents the Gram difference of cross-domain images in CDFA-Net. It is worth noting that Figure 8 in The layer is the input feature layer of the detector's top-level network. Its features directly contribute to the prediction of the target's category and location, directly influencing the network's detection and recognition results. Statistical results show that this method significantly reduces the differences in stylistic features across cross-domain data, demonstrating that it mitigates the negative impact of content-independent stylistic differences on the detection and recognition network's discrimination.

[0176] 4. Comparative Experiment

[0177] To demonstrate the superior performance of our method, this section compares it with other state-of-the-art methods using cross-validation on source and target domain datasets. This involves alternating between Set-A and Set-B as the source and target domains, testing the cross-domain detection and recognition performance of each model. Due to the limited public research on domain-adapted detection and recognition of infrared small targets, a general domain-adapted target detection and recognition method was used for comparison. Considering the real-time requirements of this task and the consistency of the underlying framework, we compared the leading single-stage domain-adapted target detection and recognition methods, including IA-Yolo, I3Net, SCANet, and IBN-Net. Furthermore, to demonstrate the performance of each domain-adapted method for infrared target detection and recognition, we also tested their underlying detection networks using the same data partitioning and training model. These included the underlying networks of our method and IA-Yolo, Yolo-v5; I3Net, SSD; and SCANet, FCOS.

[0178] 4.1 From Set-A to Set-B

[0179] First, we use the same experimental setup as the ablation experiment, with Set-A as the source domain and Set-B as the target domain. The training set for this experiment consists of 10% of the unlabeled images from the training set of Set-A and the training set of Set-B. The test set for this experiment is the test set of Set-B.

[0180] Quantitative experimental results, shown in Table 11, show that two common single-stage domain adaptation object detection methods, IA-Yolo and I3Net, show little significant performance improvement compared to their baseline networks, Yolo-v5 and SSD, and even exhibit negative transfer in some metrics. Thanks to its graph-based semantic transfer mechanism, the SCAN method achieves small positive improvements in all metrics compared to its baseline network, FCOS. Furthermore, IBN-Net, which also utilizes instance normalization to solve the domain adaptation image segmentation problem, was migrated to an object detection and recognition network based on the Yolo-v5 network. The specific embedding method involves adding an IBN module between the backbone network and the detector neck network. Because the Yolo-v5 detector has three detection scales, three additional IBN modules were added. Experimental results show that the model with the IBN module achieves a slight improvement in class average accuracy compared to the original network. Furthermore, this method achieves a greater improvement than other common domain adaptation detection methods, demonstrating the effectiveness of instance normalization for infrared dim small object detection and recognition.

[0181] The proposed CDFA-Net demonstrated the best performance among all models, significantly improving all metrics for cross-domain infrared dim target detection and recognition compared to the baseline network, Yolo-v5. Compared to other single-stage domain adaptation methods, such as IBN-Net, this method not only directly performs multi-level feature alignment within the feature network (using a loss of deep feature statistics consistency), but also incorporates additional task-guided branches to address the problem of feature preservation for infrared dim targets and the semantic gap in small sample sizes, factors that have been neglected in previous approaches.

[0182] Figure 9 Two sets of visualizations are presented for detection and recognition in Set-A using our proposed method and the original baseline network, Yolo-V5s. Qualitative experimental results show that due to the more complex background and more severe ground interference in Set-A, the baseline method, Yolo-V5s, produces more false alarms. However, under the same output threshold setting, the model trained using our proposed method produces fewer false alarms and demonstrates better stability in cross-domain infrared dim small target detection.

[0183] Compared to general feature alignment methods that address domain confusion, this method considers the impact of semantic differences on feature shift, making it a soft feature alignment method. Specifically, hard alignment methods that directly eliminate feature differences directly align high-level semantic features. These features are inherent properties of real-world objects and are crucial for target detection and recognition tasks. Therefore, altering the inherent features of infrared images, especially the semantic features within deep layers, is fatal for target detection and recognition tasks, leading to the risk of negative network transfer. Furthermore, since target domain images are typically extremely small, the richness of the object semantic information they contain is very limited compared to the vast amount of source domain data. Therefore, this semantic gap is ubiquitous in cross-domain target detection and recognition problems. Furthermore, due to the small size and weak features of weak infrared targets, they are more susceptible to interference, which has a more severe negative impact on network performance.

[0184] With this in mind, when there is a semantic gap between the source and target domains (e.g., the target domain does not have the same ground object background as the source domain), this method only aligns the high-order statistics of the features at the detection scale, avoiding the mis-transfer of content features in the case of hard alignment. In particular, for high-abstraction deep features, this method uses an additional pixel-to-pixel low-level image translation task to maintain the details and semantic consistency of small infrared targets during the feature alignment process, thereby reducing the risk of semantic confusion in the network's object detection tasks in new scenes in the target domain.

[0185] Furthermore, IA-Yolo and the proposed method use the same baseline network and employ image translation as their domain adaptation strategy. However, the IA-Yolo method uses the image translation module as a serial branch of the detector. Therefore, both during the training and application phases, the image must first be fed into the image translation branch for computation, and then the image translation results must be fed into the detector. This increases the algorithm's overall computational burden and runtime. In contrast, in the proposed method, the additional branch of the task-guided subnetwork does not participate in forward reasoning during the testing phase, and the original detector's structure and forward path remain unchanged. Therefore, the algorithm does not incur additional computational burden, maintaining the same runtime speed as the baseline single-stage algorithm, Yolov5s, without incurring additional computational burden or model size, thus ensuring the algorithm's real-time performance and edge deployment capabilities.

[0186] In one embodiment, a cross-band infrared target detection device is provided, comprising:

[0187] A detection network acquisition module is used to acquire a pre-built target detection network; the target detection network includes a feature extraction network;

[0188] The autoencoder acquisition module is used to obtain a pre-built convolutional autoencoder; the convolutional autoencoder includes a feature encoder, a multi-scale decoupling module, and a feature decoder; the feature encoder and the feature extraction network share network parameters; the multi-scale decoupling module includes multiple horizontal connection layers, and the network layers with the same feature map resolution output by the feature decoder and the feature encoder are connected to each other through each horizontal connection layer;

[0189] The style transfer module is used to calculate multi-scale feature maps of the unlabeled target domain image and the labeled source domain image based on the feature encoder, calculate the consistency loss of the deep feature statistics of the target domain image and the source domain image through the horizontal connection layer, and perform feature decoupling on the feature maps of the corresponding scale to obtain the de-stylized feature vector. The de-stylized feature vector is passed to the corresponding network layer in the feature decoder, and the feature decoder is used to calculate the transferred image;

[0190] The loss function construction module is used to input the source domain image into the target detection network and calculate the recognition loss of the target detection network, calculate the style consistency loss and content consistency loss of the target domain image and the source domain image based on the migration image, source domain image and target domain image, and construct a joint loss function based on the deep feature statistics consistency loss, recognition loss, style consistency loss and content consistency loss.

[0191] The loss function optimization module is used to optimize the joint loss function to obtain a trained target detection network, and use the trained target detection network to perform infrared target detection.

[0192] For the specific definition of the cross-band infrared target detection device, please refer to the definition of the cross-band infrared target detection method above, and will not be repeated here. The various modules in the above-mentioned cross-band infrared target detection device can be implemented in whole or in part by software, hardware, or a combination thereof. The above-mentioned modules can be embedded in or independent of the processor in the computer device in hardware form, or can be stored in the memory of the computer device in software form, so that the processor can call and execute the operations corresponding to the above modules.

[0193] In one embodiment, a computer device is provided. The computer device may be a terminal, and its internal structure diagram may be as follows: Figure 10As shown. The computer device includes a processor, memory, network interface, display screen and input device connected via a system bus. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and computer program in the non-volatile storage medium. The network interface of the computer device is used to communicate with an external terminal via a network connection. When the computer program is executed by the processor, a cross-band infrared target detection method is implemented. The display screen of the computer device can be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer device can be a touch layer covering the display screen, or a key, trackball or touchpad provided on the computer device housing, or an external keyboard, touchpad or mouse, etc.

[0194] Those skilled in the art will understand that Figure 10 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than shown in the figure, or combine certain components, or have a different component arrangement.

[0195] In one embodiment, a computer device is provided, including a memory and a processor. The memory stores a computer program, and the processor implements the steps of the method in the above embodiment when executing the computer program.

[0196] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps of the method in the above embodiment are implemented.

[0197] The technical features of the above embodiments can be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0198] The above-described embodiments merely represent several implementation methods of the present application. While the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the present invention. It should be noted that a person skilled in the art could make various modifications and improvements without departing from the spirit of the present application, all of which fall within the scope of protection of the present application. Therefore, the scope of protection of the present patent application shall be determined by the appended claims.

Claims

1. A cross-band infrared target detection method, characterized in that: The method comprises: Obtaining a pre-built target detection network; the target detection network includes a feature extraction network; Obtain a pre-built convolutional autoencoder; the convolutional autoencoder includes a feature encoder, a multi-scale decoupling module, and a feature decoder; the feature encoder and the feature extraction network share network parameters; the multi-scale decoupling module includes multiple horizontal connection layers, and the network layer with the same feature map resolution output by the feature decoder and the feature encoder is connected to each other through each of the horizontal connection layers; Obtaining multi-scale feature maps of an unlabeled target domain image and a labeled source domain image according to the feature encoder, calculating a statistical consistency loss of deep features of the target domain image and the source domain image through the lateral connection layer, performing feature decoupling on the feature maps of corresponding scales to obtain a de-stylized feature vector, and passing the de-stylized feature vector to a corresponding network layer in the feature decoder, and obtaining a transferred image through the feature decoder; Inputting the source domain image into the object detection network and calculating the recognition loss of the object detection network, calculating the style consistency loss and content consistency loss of the target domain image and the source domain image based on the migration image, the source domain image, and the target domain image, and constructing a joint loss function based on the deep feature statistic consistency loss, the recognition loss, the style consistency loss, and the content consistency loss; The joint loss function is optimized to obtain a trained target detection network, and the trained target detection network is used to perform infrared target detection.

2. The method according to claim 1, characterized in that Each of the lateral connection layers includes a residual block portion, an identity layer bypass, and an instance regularization layer; The de-stylized feature vector is obtained by performing feature decoupling on the feature map of the corresponding scale through the lateral connection layer, including: Obtain the feature map of the corresponding scale and input it into the residual block part and identity layer bypass in the corresponding lateral connection layer respectively; Calculate the feature statistics of each feature map at the corresponding scale according to the residual block part; the feature statistics include feature mean and feature variance; Obtaining an input vector for the instance normalization layer from the feature statistics and the feature map in the identity layer bypass; The input vector is normalized according to the instance normalization layer to obtain a de-stylized feature vector.

3. The method according to claim 1, characterized in that The feature decoder is used to calculate a migration image, including: In the feature decoder, the output vector of the current network layer is calculated based on the feature vector transmitted by the previous network layer and the de-stylized feature vector transmitted by the corresponding horizontal connection layer until the migration image is output.

4. The method according to claim 1, wherein Inputting the source domain image into the object detection network and calculating the recognition loss of the object detection network, including: Inputting the source domain image into the object detection network; Calculate the localization loss of the object detection network: L loc =1-GIoU Among them, L loc is the positioning loss, GIoU is the generalized intersection-over-union ratio, c s-g is the ground-truth bounding box s g and the predicted bounding box s t The smallest convex set between them, that is, the one that encloses s g 、s t The minimum bounding rectangle of Calculate the confidence loss of the object detection network: Among them, L obj is the confidence loss, Output the probability of the target box as the target for the target detection network, o c A Boolean value indicating whether the predicted value is a target. If it is a target, it is set to 1; if it is a background, it is set to 0. Calculate the classification loss of the object detection network: Where, L cls is the classification loss, is the predicted value of the i-th target output by the network, is a Boolean value indicating whether the target belongs to the i-th category, and M is the number of predicted target categories; The recognition loss of the target detection network is obtained according to the positioning loss, the confidence loss and the classification loss: L DET (I S )=γ l L loc +g o L obj +g c L cls Among them, L DET (I S ) is the recognition loss of the target detection network for the input source domain image, γ l is the weight parameter of the positioning loss, γ o is the weight parameter of confidence loss, γ c is the weight parameter of classification loss.

5. The method according to claim 1, wherein Calculating a style consistency loss and a content consistency loss of the target domain image and the source domain image according to the migration image, the source domain image, and the target domain image, comprising: Calculating a style consistency loss between the target domain image and the source domain image based on the migration image and the source domain image; A content consistency loss between the target domain image and the source domain image is calculated according to the migration image and the target domain image.

6. A cross-band infrared target detection device, characterized in that: The device comprises: A detection network acquisition module is used to acquire a pre-built target detection network; the target detection network includes a feature extraction network; An autoencoder acquisition module is used to obtain a pre-built convolutional autoencoder; the convolutional autoencoder includes a feature encoder, a multi-scale decoupling module, and a feature decoder; the feature encoder and the feature extraction network share network parameters; the multi-scale decoupling module includes multiple horizontal connection layers, and the network layer with the same feature map resolution output by the feature decoder and the feature encoder is connected to each other through each of the horizontal connection layers; a style transfer module, configured to calculate, based on the feature encoder, multi-scale feature maps of an unlabeled target domain image and a labeled source domain image, calculate, through the lateral connection layer, a consistency loss of deep feature statistics of the target domain image and the source domain image, perform feature decoupling on the feature maps of corresponding scales to obtain a de-stylized feature vector, and pass the de-stylized feature vector to a corresponding network layer in the feature decoder, thereby calculating a transferred image through the feature decoder; a loss function construction module, configured to input the source domain image into the object detection network and calculate the recognition loss of the object detection network, calculate the style consistency loss and content consistency loss of the target domain image and the source domain image based on the migration image, the source domain image, and the target domain image, and construct a joint loss function based on the deep feature statistic consistency loss, the recognition loss, the style consistency loss, and the content consistency loss; The loss function optimization module is used to optimize the joint loss function to obtain a trained target detection network, and use the trained target detection network to perform infrared target detection.

7. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 5 are implemented.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 5 are implemented.

Images