Infrared small target detection optimization method and system based on data and characteristics

Abstract

The invention relates to an infrared small target detection optimization method and system based on data and features, and belongs to the technical field of computer vision. The method comprises the following steps: preprocessing training data in an infrared small target data set, classifying the training data, and randomly cutting and expanding the data volume; and inputting the classified data and the labels corresponding to the classified data into a diffusion model for training. Then, trained diffusion model parameters are loaded, random noise, random categories and random labels are input to generate infrared data, original training data are fused, and a new synthetic training data set is obtained; in the neural network, a memory-based detection head is applied, and the synthesized data set is used for training. And finally, loading the trained network parameters, and inputting an infrared image needing to be detected to obtain a detection result. The diffusion model is introduced to generate infrared data, the diversity of training data is increased, and the detection precision and stability are improved.

Description

Technical Field

[0001] The invention relates to a data- and feature-based infrared small target detection optimization method and system, belonging to the technical field of computer vision. Background Art

[0002] Infrared small target detection aims to detect small-sized targets in infrared images, which usually occupy a small part of the image. Compared with visible light images, infrared images can adapt to low-light environments, penetrate obstacles and distinguish targets based on their thermal characteristics. Therefore, infrared small target detection has wide applications in traffic management, medical diagnosis and maritime surveillance. However, it is still a challenge to effectively detect small targets from infrared images due to the interference of noise and clutter.

[0003] Early traditional infrared small target detection methods rely on hand-crafted features and prior knowledge, making them susceptible to noise and parameter sensitivity. Therefore, it is challenging to achieve the desired detection performance. Recently, deep learning has made significant progress in infrared small target detection. Unlike traditional methods, deep learning-based methods can automatically learn end-to-end in a data-driven manner, thereby improving detection accuracy and versatility for various scenarios.

[0004] However, methods based on deep learning do not always achieve the expected good results. Experiments have found that when the target is in different backgrounds, the detection results are very different, although these detection cases appear relatively simple to the naked eye. As the size of the target decreases, the detection performance of the target in different backgrounds tends to be stable. The present invention refers to this phenomenon as the phenomenon of relative degradation of detection. The present invention analyzes that there are two main reasons for the phenomenon of relative degradation of detection: unreasonable data distribution in the data set and insufficient learning of local target features. The method based on deep learning is data-driven. The existing infrared small target data set is generally small in size, and it is difficult to cover various situations of different types of targets in different backgrounds, resulting in poor generalization performance of the network. When the target size is large, the network relies more on the contrast relationship between a certain background and foreground for detection. Therefore, even in some simple backgrounds, the features of large targets may be mistakenly identified as background information by the network. In addition, the network will learn various combinations of scenes and targets during the training process, but may not be able to fully utilize the learned information in one detection. Storing this information and using it to enhance features can effectively improve detection performance and alleviate the phenomenon of relative degradation. Summary of the invention

[0005] The purpose of the present invention is to creatively propose a data- and feature-based infrared small target detection optimization method and system based on the needs of infrared small target detection, aiming at the relative detection degradation phenomenon existing in the existing deep learning technology, that is, the larger the size of the target, the more seriously its detection effect is affected by the background.

[0006] The innovations of the present invention include: introducing a diffusion model into infrared small target detection for data generation, generating diversified infrared data to make up for the deficiencies of existing data sets in data distribution and quantity. At the same time, a memory-based detection head is designed to enhance local features, and the accuracy of the final detection result is improved by combining the original features with the memory.

[0007] The present invention is implemented by adopting the following technical solutions.

[0008] A data and feature-based infrared small target detection optimization method comprises the following steps:

[0009] Step 1: Preprocess the training data in the infrared small target dataset, classify the training data, and randomly crop and expand the data volume.

[0010] Step 2: Input the classified data and its corresponding labels into the diffusion model for training.

[0011] Step 3: Load the trained diffusion model parameters, input random noise, random categories and random labels to generate infrared data, integrate them into the original training data, and obtain a new synthetic training data set.

[0012] Step 4: Apply a memory-based detection head in the neural network and train it using the synthetic dataset.

[0013] Step 5: Load the trained network parameters, input the infrared image to be detected, and obtain the detection results.

[0014] A data and feature-based infrared small target detection optimization system comprises a data generation module, a feature extraction module and a feature enhancement module.

[0015] Beneficial Effects

[0016] Compared with the prior art, the present invention has the following advantages:

[0017] 1. The present invention introduces a diffusion model to generate infrared data, and a large amount of labeled infrared data can be obtained without additional processing, thereby increasing the diversity of training data.

[0018] 2. The present invention designs a memory-based detection head to enhance local features, thereby improving detection accuracy and stability. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Figure 1 It is a schematic diagram of detecting relative degradation phenomenon proposed by the present invention.

[0020] Figure 2 is a flow chart of the method of the present invention.

[0021] Figure 3 It is a schematic diagram of the data generation method of the present invention.

[0022] Figure 4 It is an effect diagram of infrared data generated by the data generating method of the present invention.

[0023] Figure 5 It is a schematic diagram of the feature enhancement method of the present invention.

[0024] Figure 6 It is a schematic diagram of the system of the present invention. DETAILED DESCRIPTION

[0025] In order to better illustrate the purpose and advantages of the present invention, the present invention is further described below in conjunction with the accompanying drawings.

[0026] like Figure 1 As shown in Figure 1, some existing deep learning-based infrared small target detection methods have large fluctuations in detection results when the background changes. As the target size decreases, this fluctuation gradually stabilizes. This phenomenon is called relative detection degradation.

[0027] like Figure 2 As shown, a data and feature-based infrared small target detection optimization method includes the following steps:

[0028] Step 1: Preprocess the training data in the infrared small target dataset. Classify the training data and perform random cropping to expand the data volume.

[0029] The present invention generates data based on the training set of the infrared small target detection data set, and classifies the training set data in order to improve the training effect of the generated model. At the same time, the existing infrared small target data set is generally small in scale. In order to expand the amount of training data, the original data is randomly cropped and enlarged back to the original size, thereby achieving the purpose of expanding the amount of data.

[0030] Step 2: Input the classified data and its corresponding labels into the diffusion model for training.

[0031] like Figure 3 As shown, the present invention introduces a diffusion model to generate data. In the process of adding noise, the real data x0 gradually transforms into pure noise x t , which is achieved by gradually adding noise over a series of time steps:

[0032]

[0033] Among them, q(x t |x t-1 ) is the conditional distribution, representing the data transition from time step t-1 to time step t; represents Gaussian distribution; I represents the identity matrix; β t is the time step parameter that controls the amount of noise added.

[0034] Furthermore, a linear scheduling method can be used to calculate β t :

[0035]

[0036] Among them, β start and β end denotes the predefined start and end time step parameters respectively. T is the total number of time steps; t denotes the time step.

[0037] The entire forward process is expressed as:

[0038]

[0039] Among them, x0 represents the initial image, x 1:T Represents the image after T.

[0040] Infer the noisy image x at any time step t from x0 t :

[0041]

[0042] in, represents the overall retention ratio of the data in the first t steps, α t Indicates the proportion of the data retained in the previous step.

[0043] In the reverse denoising process, the model learns to gradually remove the noise and restore the original data:

[0044]

[0045] Among them, p θ (x t-1 |x t ) is the conditional probability distribution, which means that the data of the previous time step t-1 is generated at time step t; ∑ θ (x t ,t) is the covariance of the model prediction, indicating the uncertainty of the prediction; μ θ (x t ,t) is the mean predicted by the model, which represents the expected value from t to t-1:

[0046]

[0047] Among them, ∈ θ is the noise predicted by the noise model.

[0048] To generate high-quality data, the present invention adds two additional conditional inputs to the noise model: target label and scene category.

[0049] First, since the training data contains small targets, in order to prevent uncontrollable targets from appearing in the generated data, the target label matching the infrared image is encoded as a condition. At the same time, the category number of the infrared image is mapped to the category code. Finally, the label encoding and category encoding are integrated into the features together with the time step encoding for model training.

[0050] Step 3: Load the trained diffusion model parameters, input random noise, random categories and random labels to generate infrared data, integrate them into the original training data, and obtain a new synthetic training data set.

[0051] Specifically, after the diffusion network training is completed, the optimal parameters are loaded to obtain the optimal network model, and then random noise, random categories and random labels are input into the network model, and the model will generate the corresponding infrared image according to the given input. Figure 4 As shown in Figure 1, the model generates multiple types of labeled infrared image data. These generated infrared data are integrated into the original training set to obtain a larger and more diverse synthetic training set.

[0052] Step 4: Apply the memory-based detection head to the existing neural network and train it using the synthetic dataset.

[0053] The present invention designs and implements a detection head module based on memory that can be plugged and used.

[0054] First, the feature map is divided into several blocks of the same size, and a memory bank is established for each block. After the feature map is input, the similarity between each block feature and each memory bank is calculated to obtain a similarity matrix.

[0055] Then, the Hungarian algorithm is used on the similarity matrix to match the best memory for each block. The best matching feature and memory are input into the evaluation module to obtain a score that indicates how well the feature matches each memory in the library. The evaluation module e can be expressed as:

[0056] e=Sig(Sparse(Relu(BN(Conv(x)))))

[0057] Among them, BN() and Sparse() represent batch normalization layer and sparse convolution layer respectively, Relu() and Sig() are ReLU and sigmoid activation functions respectively, and x represents the new feature that combines memory and original features.

[0058] Based on the evaluation scores, the top k most relevant memories are selected to be concatenated with the feature. They are then concatenated back to the complete feature.

[0059] The features integrated with the memory are passed through the final convolutional layer to obtain the final prediction result.

[0060] The detection head is applied to any infrared small target detection network and trained using a synthetic infrared dataset.

[0061] Step 5: Load the trained network parameters and input the infrared image to be detected to obtain the detection results.

[0062] By training on a larger and more diverse synthetic dataset and enhancing local features, the neural network is more robust to background changes and the detection performance is enhanced, thereby optimizing infrared small target detection. Specifically, by loading the optimal network parameters and inputting the infrared image to be detected, a more complete and stable infrared small target detection result is obtained.

[0063] On the other hand, in order to achieve the purpose of the present invention, the present invention further proposes a data and feature-based infrared small target detection optimization system, including a data generation module 10, a feature extraction module 20 and a feature enhancement module 30. Figure 5 shown.

[0064] The data generation module 10 is used to obtain a larger-scale and more diverse synthetic data set based on the training data of the existing data set.

[0065] The feature extraction module 20 is used to configure any existing infrared small target detection neural network to extract features of infrared small targets.

[0066] The feature enhancement module 30 enhances the infrared small target features extracted by the feature extraction module 20 based on memory, thereby optimizing the robustness of the neural network to background changes and enhancing the detection performance.

[0067] The output ends of the data generation module 10 and the feature enhancement module 30 are connected to the input end of the feature extraction module 20 respectively.

Claims

1. A data and feature based infrared small target detection optimization method, It is characterized in that The following steps are involved: Step 1: Preprocess the training data in the infrared small target dataset, classify the training data, and randomly crop and expand the data volume; Step 2: Input the classified data and its corresponding labels into the diffusion model for training; In the process of adding noise, the real data x 0 Gradually transforms into pure noise x t , which is achieved by gradually adding noise over a series of time steps: Among them, q(x t |x t-1 ) is the conditional distribution, representing the data transition from time step t-1 to time step t; represents Gaussian distribution; I represents the identity matrix; β t is the time step parameter that controls the amount of noise added; The entire forward process is expressed as: Among them, x 0 represents the initial image, T is the total number of time steps; t represents the time step, x 1:T represents the image after T; From x 0 Infer the noisy image x at any time step t t : in, represents the overall retention ratio of the data in the first t steps, α t Indicates the proportion of the data retained in the previous step; In the reverse denoising process, the model learns to gradually remove the noise and restore the original data: Among them, p θ (x t-1 |x t ) is the conditional probability distribution, which means that the data of the previous time step t-1 is generated at time step t; ∑ θ (x t ,t) is the covariance of the model prediction, indicating the uncertainty of the prediction; μ θ (x t ,t) is the mean predicted by the model, which represents the expected value from t to t-1: Among them, ∈ θ is the noise predicted by the noise model; Step 3: Load the trained diffusion model parameters, input random noise, random categories and random labels to generate infrared data, integrate them into the original training data, and obtain a new synthetic training data set; Step 4: Apply a memory-based detection head in the neural network and train it using the synthetic dataset. Step 5: Load the trained network parameters, input the infrared image to be detected, and obtain the detection results.

2. The infrared small target detection optimization method based on data and features as claimed in claim 1, It is characterized in that Calculate β using the linear scheduling method t : Among them, β start and β end denote the predefined start and end time step parameters respectively; T is the total number of time steps; and t denotes the time step.

3. The infrared small target detection optimization method based on data and features as claimed in claim 1, It is characterized in that Add two conditional inputs to the noise model: target label and scene category; Firstly, the target label matched with the infrared image is encoded as a condition. Meanwhile, the category number of the infrared image is mapped to the category code. Finally, the label code and category code are integrated into the features together with the time step code for model training.

4. The infrared small target detection optimization method based on data and features as claimed in claim 1, It is characterized in that In step 3, after the diffusion network training is completed, the optimal parameters are loaded to obtain the optimal network model, and then random noise, random categories and random labels are input into the network model, and the model will generate the corresponding infrared image according to the given input.

5. The infrared small target detection optimization method based on data and features as claimed in claim 1, It is characterized in that In step 4, a plug-and-play memory-based detection head module is used; First, the feature map is divided into several blocks of the same size, and a memory bank is established for each block. After the feature map is input, the similarity between each block feature and each memory bank is calculated to obtain a similarity matrix. Then, the Hungarian algorithm is used on the similarity matrix to match the best memory for each block; the best matching feature and memory are input into the evaluation module to obtain a score that indicates how well the feature matches each memory in the library; Based on the evaluation scores, select the top k most relevant memories and features to concatenate, and then concatenate them back to the complete feature; The features integrated with the memory are passed through the final convolutional layer to obtain the final prediction result; The detection head is applied to any infrared small target detection network and trained using a synthetic infrared dataset.

6. The infrared small target detection optimization method based on data and features as claimed in claim 5, It is characterized in that The evaluation module e is expressed as: e=Sig(Sparse(Relu(BN(Cone(x)))) Among them, BN() and Sparse() represent batch normalization layer and sparse convolution layer respectively, Relu() and Sig() are ReLU and sigmoid activation functions respectively, and x represents the new feature that combines memory and original features.

7. A data and feature based infrared small target detection optimization system for implementing the method of claim 1, It is characterized in that It includes a data generation module 10, a feature extraction module 20 and a feature enhancement module 30; Wherein, the data generation module 10 is used for training data based on an existing data set; A feature extraction module 20, used to configure any existing infrared small target detection neural network to extract features of infrared small targets; A feature enhancement module 30, which enhances the infrared small target features extracted by the feature extraction module 20 based on memory; The output ends of the data generation module 10 and the feature enhancement module 30 are connected to the input end of the feature extraction module 20 respectively.

Images