Infrared image data enhancement method and system based on quantization and diffusion model

Abstract

The invention discloses an infrared image data enhancement method based on a quantization and diffusion model, and relates to the technical field of data enhancement, and the method comprises the steps: carrying out the non-uniform quantization of infrared image data through employing a first quantization parameter, and obtaining first quantized image data; establishing a data enhancement model comprising a pixel coarse reconstruction module and a generation module based on a diffusion model; training by using the first quantized image data to obtain a trained data enhancement model; performing non-uniform quantization operation on the infrared image data by using a second quantization parameter to obtain second quantization image data; and inputting the second quantized image data into the trained data enhancement model for reasoning, and generating a new infrared image data sample subjected to data enhancement. According to the invention, a new sample is created through a cross-sampling non-uniform quantization technology and a data enhancement model, abundant training data with a complete structure are provided, and the training effect of an infrared small target detection model is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of infrared image data enhancement, and more specifically, to an infrared image data enhancement method and system based on quantization and diffusion model. Background Art

[0002] In recent years, with the rapid development of computer vision technology, high-quality image data has become the basis for the success of many visual tasks. However, in the field of infrared images, especially infrared small target detection, the limitation of data quality has always been the key bottleneck for improving model performance. Infrared small target images have blurred target edges and low contrast due to factors such as weak signals and complex background noise, which brings great challenges to detection tasks. Especially under low signal-to-noise ratio conditions, traditional methods are difficult to effectively distinguish small targets from the background, which limits their practical applications in remote sensing monitoring, military detection, and automatic early warning.

[0003] Objects in infrared images usually appear as small targets due to the long shooting distance. In addition, they are affected by various factors, such as environmental factors (drones may be affected by fog or clouds) and equipment factors (sensors inevitably introduce noise), which makes small infrared targets submerged in various complex backgrounds and difficult to detect.

[0004] With the rapid development of deep learning technology, data-driven infrared small target detection methods have become a hot research area. These methods mainly rely on convolutional neural networks (CNNs) to extract image features and achieve accurate positioning and segmentation of targets through iterative training. However, to achieve excellent detection results through these deep learning methods, a key prerequisite is to have sufficient training data sets. However, due to the limitations of technology and equipment, the complexity of environmental conditions, privacy and security issues, and the limitations of data sharing, it has become very difficult to obtain infrared image data. In addition, collecting a large amount of data and manually annotating it is a time-consuming and labor-intensive task that requires a lot of human and material resources. Therefore, how to use limited infrared image data to generate a large amount of infrared image data has become an urgent problem to be solved in current research.

[0005] The existing infrared small target data enhancement methods easily destroy the structural features and pixels of small targets through the generated images, which has an adverse effect on the precise detection task. At the same time, they rarely consider the data distribution information of infrared images, which affects the diversity and authenticity of infrared image data. Summary of the invention

[0006] The purpose of the present invention is to provide an infrared image data enhancement method and system based on quantization and diffusion models, so as to overcome the problems that the infrared small target data enhancement method in the prior art easily destroys the structural features and pixels of the small target through the generated images, which has an adverse effect on the precise detection task, and at the same time, less consideration is given to the data distribution information of the infrared image, which affects the diversity and authenticity of the infrared image data.

[0007] To achieve the above object, the technical solution of the present invention is as follows:

[0008] A method for infrared image data enhancement based on quantization and diffusion model comprises the following steps:

[0009] Acquire original infrared image data and perform a non-uniform quantization operation using a first quantization parameter to obtain first quantized image data;

[0010] Establishing a data enhancement model including a pixel coarse reconstruction module and a generation module based on a diffusion model;

[0011] Using the first quantized image data to train the pixel coarse reconstruction module to obtain first reconstructed image data, and using the first reconstructed image data to train the diffusion model-based generation module to obtain the trained data enhancement model;

[0012] Performing a non-uniform quantization operation on the original infrared image data using a second quantization parameter to obtain second quantized image data;

[0013] The second quantized image data is input into the trained data enhancement model for inference, and the second reconstructed image data is obtained through the pixel coarse reconstruction module and input into the diffusion model-based generation module to generate new infrared image data samples after data enhancement.

[0014] Preferably, the first quantization parameter and the second quantization parameter are obtained by cross sampling the same normal distribution.

[0015] Preferably, the cross sampling to obtain the first quantization parameter and the second quantization parameter is specifically:

[0016] The mean is n and the variance is m 2 The first quantization parameter and the second quantization parameter are obtained by cross sampling according to the normal distribution of X1, X3, ..., X k , the second quantization parameter is X2, X4..., X k-1 , or, the first quantization parameter is X2, X4..., X k-1 , the second quantization parameter is X1, X3, ..., X k, where X k -X k-1 = m, and X k , n, m, and k are all positive integers.

[0017] Preferably, the performing of the non-uniform quantization operation specifically includes:

[0018] A non-uniform quantizer is set to have a random interval and value, and the non-uniform quantizer includes a non-overlapping interval set S i ={[a i ,a i+1 )}, a set of replicated values ​​y i and a value Num representing the number of intervals, and Num is the first quantization parameter or the second quantization parameter, a i It is obtained by the following formula:

[0019] a0,a1,...,a n-1 =sort(a ′ 0,a ′ 1,...,a ′ n-1 )

[0020] a′ i =U(min(x),max(x)),i=0,1,...,Num-1

[0021] Among them, sort is a sorting algorithm, U represents random sampling in the interval, min(x) and max(x) represent the minimum and maximum values ​​of the pixels in each channel of the image, respectively, and the image pixel values ​​falling in the interval are copied by the value y. i replace:

[0022] y i =U(a i ,a i+1 ),i=0,1,...,Num-1

[0023] Different quantization interval sizes are selected, and the non-uniform quantizer is used to discretize continuous pixel values, and the image is compressed to different degrees to reduce data complexity while retaining the structural features of the image at different scales.

[0024] Preferably, the pixel coarse reconstruction module includes three parts: shallow feature extraction, deep feature extraction and image reconstruction;

[0025] The shallow feature extraction part extracts the shallow features of the quantized image data through a 3×3 convolution layer and divides it into multiple non-overlapping blocks for embedding;

[0026] The deep feature extraction part extracts deep features from the block embedding through several normalization layers and attention layers;

[0027] After fusing the shallow features with the deep features, the image reconstruction part performs three interpolation upsampling to obtain reconstructed image data with the same resolution as the original infrared image data, wherein each interpolation is followed by a convolution layer and a Leaky ReLU activation layer.

[0028] Preferably, the pixel coarse reconstruction module is trained and optimized with the goal of minimizing the pixel loss between the first reconstructed image data and the input first quantized image data, and the loss function of the pixel loss is as follows:

[0029]

[0030] Among them, I restore represents a pixel of the first reconstructed image data, I input Represents a pixel of the first quantized image data as input.

[0031] Preferably, the diffusion model-based generation module includes a potential diffusion model based on a U-Net network, including an encoder ε and a decoder

[0032] Preferably, when the first reconstructed image data is used to train the generation module based on the diffusion model, the first reconstructed image data is used as the input image I in , which is encoded into latent space variables by the encoder ε during training Then gradually add Gaussian noise to get a potential image close to Gaussian distribution Generate denoised image by t-step resampling at last, Through the decoder Generate a resampled image by minimizing the loss function To optimize the generation module based on the diffusion model, so that the generated image is closer to the real image, the loss function as follows:

[0033]

[0034] By minimizing the deviation between the noise of the real image and the noise of the generated image, the generation module based on the diffusion model can predict the data distribution of the infrared image and generate infrared image data that is closer to the reality.

[0035] Preferably, the specific steps of using the diffusion model-based generation module to generate infrared image data include:

[0036] The input reconstructed image data I is encoded into a potential representation by the encoder ε Then at each time step t, a hyperparameter β is added t ∈(0,1) Gaussian noise, generating the latent image

[0037]

[0038] in α t =1-β t , When the value of t is large enough, z t Approximate to the standard Gaussian distribution, the noise prediction network ∈ θ Trained as a denoising model and passed The loss function is optimized:

[0039]

[0040] in, Express expectations, represents a random variable in the latent space, c is a conditional variable, t is the time step, represents a step in the diffusion process, and ∈ is the Gaussian noise sampled from the standard Gaussian noise;

[0041] Use by The denoising model after the loss function is optimized performs t-step resampling to remove the noise in the potential image and generate a denoised image The denoised image Through the decoder Generate a resampled image, i.e., infrared image data.

[0042] To achieve the purpose of the invention, the present invention further provides an infrared image data enhancement system based on quantization and diffusion model, and an infrared image data enhancement method based on quantization and diffusion model using the above technical solution, the system comprising:

[0043] A cross quantization unit, configured to perform a non-uniform quantization operation on the original infrared image data using a first quantization parameter to obtain first quantized image data or to perform a non-uniform quantization operation on the original infrared image data using a second quantization parameter to obtain second quantized image data;

[0044] A modeling unit, used for establishing a data enhancement model including a pixel coarse reconstruction module and a generation module based on a diffusion model;

[0045] A training unit, configured to train the pixel coarse reconstruction module using the first quantized image data to obtain first reconstructed image data, and to train the diffusion model-based generation module using the first reconstructed image data to obtain the trained data enhancement model;

[0046] The infrared data image generation unit is used to input the second quantized image data into the trained data enhancement model for inference, obtain the second reconstructed image data through the pixel coarse reconstruction module and input it into the diffusion model-based generation module to generate data-enhanced infrared image data samples.

[0047] Compared with the prior art, the present invention has the following beneficial effects:

[0048] The present invention creates new samples by performing multi-scale feature compression on the infrared small target data set through non-uniform quantization operations, which will not damage the small target pixels and well preserves the structural characteristics of the small target. Through the generation module based on the diffusion model, the integrity of the target is preserved in the process of generating infrared image data, and the Gaussian distribution is used to control the sample diversity. Even in the case of few samples, rich and structurally complete training data can be generated, which significantly improves the training effect of the infrared small target detection model in the few-sample scenario. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] Figure 1 A schematic diagram of the steps of an infrared image data enhancement method based on quantization and diffusion model of the present application;

[0050] Figure 2 This is a schematic diagram of the cross sampling process of Example 1 of the present application;

[0051] Figure 3 This is a schematic diagram of the process of non-uniform quantization of Example 1 of the present application;

[0052] Figure 4 A result diagram of a quantized image and an output image of the system of Example 2 of the present application;

[0053] Figure 5 A schematic diagram of a process of generating and outputting infrared image data in the inference phase using an infrared image data enhancement system based on a quantization and diffusion model that has been trained using a loss function according to Embodiment 2 of the present application;

[0054] Figure 6 A schematic diagram of the detection results of an infrared small target detection model using the method of the present invention to generate an infrared image training data set;

[0055] Figure 7 The present invention is a schematic diagram of the detection results of an infrared small target detection model that uses the method of the present invention to generate an infrared image training data set. DETAILED DESCRIPTION

[0056] In order to more clearly understand the above-mentioned purpose, features and advantages of the present invention, the present invention is further described in detail below in conjunction with the accompanying drawings and specific embodiments. It should be noted that the embodiments of the present application and the features in the embodiments can be combined with each other without conflict.

[0057] In the following description, many specific details are set forth to facilitate a full understanding of the present invention. However, the present invention may also be implemented in other ways different from those described herein. Therefore, the protection scope of the present invention is not limited to the specific embodiments disclosed below.

[0058] Example 1

[0059] See also Figure 1 Embodiment 1 of the present application provides an infrared image data enhancement method based on quantization and diffusion model, comprising the following steps:

[0060] S1: Acquire original infrared image data and perform a non-uniform quantization operation using a first quantization parameter to obtain first quantized image data.

[0061] Cross Sampling: See Figure 2 , Figure 2 The present invention is a flow chart of cross sampling of Example 1 of the present application. The present invention samples different quantization parameters in the same normal distribution, and obtains a first quantization parameter and a second quantization parameter, which are used in the training stage and the inference stage to quantize the image, so as to provide visual diversity. In the training stage, the first quantization parameter is used as the quantization parameter, and in the inference stage, the second quantization parameter is used as the quantization parameter. The cross sampling is as follows: for a mean of n and a variance of m 2 The first quantization parameter and the second quantization parameter are obtained by cross sampling according to the normal distribution of X1, X3, ..., X k , the second quantization parameter is X2, X4..., X k-1 , or, the first quantization parameter is X2, X4..., X k-1 , the second quantization parameter is X1, X3, ..., X k , where X k -X k-1 = m, and X k , n, m, and k are all positive integers.

[0062] Unlike traditional data enhancement methods, it does not destroy structural features when generating new samples. Different sets of quantization parameters are sampled in training and inference respectively, and the generated images have richer textures and accurate semantic distribution. The sample diversity is controlled by normal distribution, which can provide rich training data.

[0063] Non-uniform quantization: See Figure 3 , Figure 3 Schematic diagram of the non-uniform quantization process of Example 1 of the present application. The present invention generates a large number of training data samples by quantizing and compressing infrared small target images, thereby improving the performance of the infrared small target detection model. Specifically, a non-uniform quantizer with random interval intervals and values ​​is set, and the non-uniform quantizer includes a non-overlapping interval set S i ={[a i ,a i+1 )}, a set of replicated values ​​y i , a value Num representing the number of intervals, and Num is the first quantization parameter or the second quantization parameter. i It is obtained by the following formula:

[0064] a0,a1,...,a n-1 =sort(a′0,a′1,...,a′ n-1 )

[0065] a′ i =U(min(x),max(x)),i=0,1,...,Num-1

[0066] Among them, sort is a sorting algorithm, U represents random sampling within the interval, min(x) and max(x) represent the minimum and maximum pixel values ​​of each channel of the image respectively. The pixel values ​​of the image that fall within the interval are copied by the value y i replace:

[0067] y i =U(a i ,a i+1 ),i=0,1,...,Num-1

[0068] By selecting different quantization interval sizes and discretizing continuous pixel values ​​through non-uniform quantizers, the image is compressed to different degrees to reduce data complexity and achieve images with multiple details compressed and modified. This quantization method combines the diffusion model to generate more diverse enhanced samples, effectively broadening the number of data sets and generating more diverse and challenging samples for the detection model.

[0069] S2: Establish a data enhancement model including a pixel coarse reconstruction module and a generation module based on a diffusion model.

[0070] The pixel coarse reconstruction module aims to learn the mapping representation from quantized compressed images to complex images. By inputting images of different quantization levels into the trained model, target images with different details are generated. The module is mainly divided into three parts, namely the shallow feature extraction module, the deep feature extraction module and the high-quality image reconstruction module. Specifically, the quantization interval and replication value of each image are randomly selected, the quantized image is subjected to shallow feature extraction through 3x3 convolution, and is divided into multiple non-overlapping blocks for embedding, and then the deep features are extracted through multiple normalization layers and attention layers. After the shallow and deep features are fused, they are upsampled back to the original resolution through cubic interpolation, and each interpolation is followed by a convolution layer and a Leaky ReLU activation layer.

[0071] The generation module based on the diffusion model selects the latent diffusion model to generate prior information. The latent diffusion model reduces the computational complexity by gradually "denoising" the data in the latent representation space, decoding it into a complete image, and compressing the image into the latent space. The latent diffusion model is based on the U-Net network structure, which includes an encoder ε and a decoder The prior mechanism of the diffusion model significantly enhances the realism of the generated data, provides more typical samples for unsupervised learning, and improves the learning and generalization capabilities of the model. The images generated and processed by the diffusion model help the detection model learn more features, enhancing robustness and detection accuracy.

[0072] S3: Use the first quantized image data to train the pixel coarse reconstruction module to obtain first reconstructed image data, and use the first reconstructed image data to train the diffusion model-based generation module to obtain the trained data enhancement model.

[0073] The pixel coarse reconstruction module is trained and optimized with the goal of minimizing the pixel loss between the first reconstructed image data and the input first quantized image data. The loss function of the pixel loss is as follows:

[0074]

[0075] Among them, I restore represents a pixel of the first reconstructed image data, I input Represents a pixel of the first quantized image data as input.

[0076] When the first reconstructed image data is used to train the generation module based on the diffusion model, the first reconstructed image data is used as the input image I in , which is encoded into latent space variables by the encoder ε during training Then gradually add Gaussian noise to get a potential image close to Gaussian distribution Noise prediction network ε θTrained as a denoising model and passed The loss function is used for training optimization:

[0077]

[0078] in, Express expectations, represents a random variable in the latent space, c is the conditional variable, t is the time step, represents a step in the diffusion process, and ε is the Gaussian noise sampled from standard Gaussian noise.

[0079] The denoised image is generated by performing t-step resampling through the denoising model at last, Through the decoder Generate a resampled image by minimizing the loss function To optimize the generation module based on the diffusion model, so that the generated image is closer to the real image, the loss function as follows:

[0080]

[0081] By minimizing the deviation between the noise of the real image and the noise of the generated image, the generation module based on the diffusion model can predict the data distribution of the infrared image and generate infrared image data that is closer to the reality.

[0082] The trained latent diffusion model can learn the distribution of images, capture the shape and texture of objects, and thus generate images that conform to the real distribution and reasonable semantics. In order for the diffusion model to generate high-quality infrared image data in the target domain, it is fine-tuned using the first reconstructed image data generated by the pixel coarse reconstruction module after quantization using the infrared small target dataset. By learning a large number of infrared small target images, the model can understand the background semantics of the image, the shape and texture features of the small target, and thus generate realistic and high-quality infrared images. The diffusion model introduces real-world knowledge by learning image distribution. It captures potential data features without the need for additional labels, generating more realistic and diverse samples for dataset reconstruction.

[0083] S4: performing a non-uniform quantization operation on the original infrared image data using a second quantization parameter to obtain second quantized image data.

[0084] S5: Input the second quantized image data into the trained data enhancement model for inference, obtain the second reconstructed image data through the pixel coarse reconstruction module and input it into the generation module based on the diffusion model to generate a new infrared image data sample after data enhancement.

[0085] The second reconstructed image data output by the pixel coarse reconstruction module is used as the input image I and is compressed into a potential representation by the encoder ε Then at each time step t, a hyperparameter β is added t ∈(0,1) Gaussian noise, generating the latent image

[0086]

[0087] in α t =1-β t and When the value of t is large enough, Approximates a standard Gaussian distribution. Use The denoising model after the loss function is optimized performs t-step resampling to remove the noise in the potential image and generate a denoised image The denoised image Through the decoder Generate a resampled image, i.e., infrared image data, and complete the data enhancement of the infrared image.

[0088] Example 2

[0089] This embodiment 2 is based on embodiment 1, and applies an infrared image data enhancement method based on quantization and diffusion model in embodiment 1 to provide an infrared image data enhancement system based on quantization and diffusion model. The system is specifically as follows:

[0090] A cross quantization unit, configured to perform a non-uniform quantization operation on the original infrared image data using a first quantization parameter to obtain first quantized image data or to perform a non-uniform quantization operation on the original infrared image data using a second quantization parameter to obtain second quantized image data;

[0091] A modeling unit, used for establishing a data enhancement model including a pixel coarse reconstruction module and a generation module based on a diffusion model;

[0092] A training unit, configured to train the pixel coarse reconstruction module using the first quantized image data to obtain first reconstructed image data, and to train the diffusion model-based generation module using the first reconstructed image data to obtain the trained data enhancement model;

[0093] The infrared data image generation unit is used to input the second quantized image data into the trained data enhancement model for inference, obtain the second reconstructed image data through the pixel coarse reconstruction module and input it into the diffusion model-based generation module to generate data-enhanced infrared image data samples.

[0094] See also Figure 4 , Figure 4 This is a result diagram of the quantized image and the output image of the system of Example 2 of the present application.

[0095] See also Figure 5 , Figure 5 A schematic diagram of a process for generating and outputting infrared image data in the inference stage using an infrared image data enhancement system based on a quantization and diffusion model that has been trained using a loss function in Example 2 of the present application. An infrared small target image in an infrared data set is input as an input image into a cross quantization unit for non-uniform quantization to obtain a quantized image, which is then input into an infrared data image generation unit. A second reconstructed image data is generated through a pixel coarse reconstruction module and input into a generation module based on a diffusion model to generate an output image for expanding the infrared small target image data set.

[0096] The other steps of this embodiment 2 are the same as those of embodiment 1 and will not be repeated here.

[0097] Example 3

[0098] This embodiment 3 is based on embodiment 1, and uses an infrared image data enhancement method based on quantization and diffusion model in the above embodiment 1 to conduct experiments to verify the effectiveness of the method of the present invention, as follows:

[0099] The NUDT-SIRST dataset and the SIRST dataset were selected for experiments. The SIRST dataset is a dataset specially built for single-frame infrared small target detection. It selects representative pictures from infrared sequences of hundreds of different scenes. The dataset contains 427 infrared images and 480 targets. The targets in the images are confirmed by observing the moving sequences to ensure that they are real targets rather than pixel-level impulse noise. NUDT-SIRST dataset This dataset is an artificially synthesized dataset of five main background scenes: city, field, highlight, ocean, and cloud. It contains a variety of small targets such as points, airplanes, drones, and ships. Each image is synthesized from a real background of different targets (e.g., points, airplanes, and extended targets) with different SCRs and rich postures. The total amount of data is 1327.

[0100] 1. Pixel coarse reconstruction module training phase: In order to generate diversified remote sensing images and expand the scale of infrared small target data sets, the pixel coarse reconstruction module of the data enhancement model is trained. In the training phase, we sample from a normal distribution with a mean of 17 and a variance of 4 to obtain the first quantization parameter [7,9,11,...,23,25,27]. The first quantization parameter is used to perform non-uniform quantization operations on the original small target remote sensing image to compress the image information, and the quantized image is used as a training sample for the pixel coarse reconstruction module. The training batch size of the pixel coarse reconstruction module is set to 3, the number of training steps is set to 25000, and the learning rate is set to 0.001. After the training is completed, the first reconstructed image data is output as the training sample for the subsequent modules.

[0101] 2. Training phase of the generation module based on the diffusion model: During the training process, the diffusion model based on the U-Net network structure pre-trained on ImageNet is loaded as the pre-training model of the generation module. The first reconstructed image data quantized and restored by the pixel coarse reconstruction module is used as the training data. The batch size, training steps, and learning rate are set to 1, 20, and 0.01 respectively for training fine-tuning.

[0102] 3. Generate infrared images for data augmentation inference stage: After both modules in the data augmentation model are trained, for the inference stage, another set of quantization values ​​is sampled from the same normal distribution, i.e., the second quantization parameter is [8, 10, 12, .., 22, 24, 26], and samples are drawn on NUDT-SIRST and SIRST for non-uniform quantization. The generation module based on the diffusion model uses DDPM sampling with time interval jumps for resampling, and the time step is set to 50. Twice the image data is generated on the SIRST and NUDT-SIRST training sets.

[0103] 4. Testing and evaluation: In order to evaluate the performance of the infrared image dataset generated by the method of the present invention, the infrared small target detection model is trained with the generated data and tested with the test set. The intersection-over-union ratio, detection probability and false alarm rate are used as evaluation indicators.

[0104]

[0105]

[0106] Table 1 Experimental results

[0107] See also Figure 6 and Figure 7 , Figure 6 and Figure 7The following is a schematic diagram of the detection results of different infrared small targets of the infrared small target detection model using the method of the present invention to generate an infrared image training data set. Please refer to Table 1. The experimental results show that the data enhancement method of the present invention has greatly improved the performance of the detection model, and the generated image quality is higher and the visual effect is better. In particular, it shows strong adaptability and robustness in the infrared small target detection task. Through this infrared small target data enhancement method based on quantization operation and diffusion model, combined with the internal and external collaborative data enhancement strategy, the quality of infrared images and the accuracy of detection are improved, providing an efficient and reliable data enhancement solution for infrared small target detection in complex scenes.

[0108] The other steps of this embodiment 3 are the same as those of embodiment 1 and will not be repeated here.

[0109] In summary, the present invention proposes a method for infrared small target data enhancement based on quantization operation and diffusion model, which improves the performance of the model in complex scenes through a data enhancement strategy of quantization and pixel reconstruction. In terms of quantization operation, cross sampling is used to obtain different quantization parameters for non-uniform quantization. Different quantization parameters are used in the training stage and the inference stage to perform different degrees of quantization compression on the image. In the pixel coarse reconstruction module, shallow and deep features are extracted from the quantized image and fused, and the compressed quantized pixels are coarsely reconstructed. In the generation module based on the diffusion model, the distribution of the data set is learned through the diffusion model, and real-world knowledge is introduced into the image through the resampling strategy, so that the generated infrared image is more diverse and realistic.

[0110] Obviously, the above embodiments of the present invention are merely examples for clearly illustrating the present invention, and are not intended to limit the embodiments of the present invention. For those skilled in the art, other different forms of changes or modifications can be made based on the above description. It is not necessary and impossible to list all the embodiments here. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the protection scope of the claims of the present invention.

Claims

1. A method for infrared image data enhancement based on quantization and diffusion model, characterized in that: The following steps are involved: Acquire original infrared image data and perform a non-uniform quantization operation using a first quantization parameter to obtain first quantized image data; Establishing a data enhancement model including a pixel coarse reconstruction module and a generation module based on a diffusion model; Using the first quantized image data to train the pixel coarse reconstruction module to obtain first reconstructed image data, and using the first reconstructed image data to train the diffusion model-based generation module to obtain the trained data enhancement model; Performing a non-uniform quantization operation on the original infrared image data using a second quantization parameter to obtain second quantized image data; The second quantized image data is input into the trained data enhancement model for inference, and the second reconstructed image data is obtained through the pixel coarse reconstruction module and input into the diffusion model-based generation module to generate new infrared image data samples after data enhancement.

2. The infrared image data enhancement method based on quantization and diffusion model according to claim 1, characterized in that: The first quantization parameter and the second quantization parameter are obtained by cross sampling the same normal distribution.

3. The infrared image data enhancement method based on quantization and diffusion model according to claim 2 is characterized in that: The cross sampling to obtain the first quantization parameter and the second quantization parameter is specifically: The mean is n and the variance is m 2 The first quantization parameter and the second quantization parameter are obtained by cross sampling according to the normal distribution of X1, X3, ..., X k , the second quantization parameter is X2, X4..., X k-1 , or, the first quantization parameter is X2, X4..., X k-1 , the second quantization parameter is X1, X3, ..., X k , where X k -X k-1 = m, and X k , n, m, and k are all positive integers.

4. The infrared image data enhancement method based on quantization and diffusion model according to any one of claims 1 to 3, characterized in that: The performing of the non-uniform quantization operation specifically includes: A non-uniform quantizer is set to have a random interval and value, and the non-uniform quantizer includes a non-overlapping interval set S i ={[a i ,a i+1 )}, a set of replicated values ​​y i and a value Num representing the number of intervals, and Num is the first quantization parameter or the second quantization parameter, a i It is obtained by the following formula: a0,a1,...,a n-1 =sort(a ′ 0,a ′ 1,...,a ′ n-1 ) a′ i =U(min(x),max(x)),i=0,1,...,Num-1 Among them, sort is a sorting algorithm, U represents random sampling in the interval, min(x) and max(x) represent the minimum and maximum values ​​of the pixels in each channel of the image, respectively, and the image pixel values ​​falling in the interval are copied by the value y. i replace: and i =U(a i ,to i+1 ),i=0,1,...,Num-1 Different quantization interval sizes are selected, and the non-uniform quantizer is used to discretize continuous pixel values, and the image is compressed to different degrees to reduce data complexity while retaining the structural features of the image at different scales.

5. The infrared image data enhancement method based on quantization and diffusion model according to claim 1, characterized in that: The pixel coarse reconstruction module includes three parts: shallow feature extraction, deep feature extraction and image reconstruction; The shallow feature extraction part extracts the shallow features of the quantized image data through a 3×3 convolution layer and divides it into multiple non-overlapping blocks for embedding; The deep feature extraction part extracts deep features from the block embedding through several normalization layers and attention layers; After fusing the shallow features with the deep features, the image reconstruction part performs three interpolation upsampling to obtain reconstructed image data with the same resolution as the original infrared image data, wherein each interpolation is followed by a convolution layer and a Leaky ReLU activation layer.

6. The infrared image data enhancement method based on quantization and diffusion model according to any one of claims 1 or 5, characterized in that: The pixel coarse reconstruction module is trained and optimized with the goal of minimizing the pixel loss between the first reconstructed image data and the input first quantized image data. The loss function of the pixel loss is as follows: Among them, I restore represents a pixel of the first reconstructed image data, I input Represents a pixel of the first quantized image data as input.

7. The infrared image data enhancement method based on quantization and diffusion model according to claim 1, characterized in that: The generation module based on the diffusion model includes a potential diffusion model based on the U-Net network, including an encoder ε and a decoder 8. The infrared image data enhancement method based on quantization and diffusion model according to claim 7 is characterized in that: When the first reconstructed image data is used to train the generation module based on the diffusion model, the first reconstructed image data is used as the input image I in , which is encoded into latent space variables by the encoder ε during training Then gradually add Gaussian noise to get a potential image close to Gaussian distribution Generate denoised image by t-step resampling at last, Through the decoder Generate a resampled image by minimizing the loss function To optimize the generation module based on the diffusion model, so that the generated image is closer to the real image, the loss function as follows: By minimizing the deviation between the noise of the real image and the noise of the generated image, the generation module based on the diffusion model can predict the data distribution of the infrared image and generate infrared image data that is closer to the reality.

9. The infrared image data enhancement method based on quantization and diffusion model according to claim 8, characterized in that: The specific steps of using the diffusion model-based generation module to generate infrared image data include: The input reconstructed image data I is encoded into a potential representation by the encoder ε Then at each time step t, a hyperparameter β is added t ∈(0,1) Gaussian noise, generating the latent image in α t =1-β t , When the value of t is large enough, Approximate to the standard Gaussian distribution, the noise prediction network ∈ θ Trained as a denoising model and passed The loss function is optimized: in, Express expectations, represents a random variable in the latent space, c is a conditional variable, t is the time step, represents a step in the diffusion process, and ∈ is the Gaussian noise sampled from the standard Gaussian noise; Use by The denoising model after the loss function is optimized performs t-step resampling to remove the noise in the potential image and generate a denoised image The denoised image Through the decoder Generate a resampled image, i.e., infrared image data.

10. An infrared image data enhancement system based on quantization and diffusion model, using an infrared image data enhancement method based on quantization and diffusion model as described in any one of claims 1 to 9, characterized in that: The system comprises: A cross quantization unit, configured to perform a non-uniform quantization operation on the original infrared image data using a first quantization parameter to obtain first quantized image data or to perform a non-uniform quantization operation on the original infrared image data using a second quantization parameter to obtain second quantized image data; A modeling unit, used for establishing a data enhancement model including a pixel coarse reconstruction module and a generation module based on a diffusion model; A training unit, configured to train the pixel coarse reconstruction module using the first quantized image data to obtain first reconstructed image data, and to train the diffusion model-based generation module using the first reconstructed image data to obtain the trained data enhancement model; The infrared data image generation unit is used to input the second quantized image data into the trained data enhancement model for inference, obtain the second reconstructed image data through the pixel coarse reconstruction module and input it into the diffusion model-based generation module to generate data-enhanced infrared image data samples.

Images