A method and system for infrared image data enhancement based on quantization and diffusion model

Abstract

The present invention discloses a method for infrared image data enhancement based on quantization and diffusion models, which relates to the field of data enhancement technology. The method comprises: performing non-uniform quantization on infrared image data using a first quantization parameter to obtain first quantized image data; establishing a data enhancement model including a pixel coarse reconstruction module and a generation module based on a diffusion model; training the infrared image data using the first quantized image data to obtain a trained data enhancement model; performing non-uniform quantization on the infrared image data using a second quantization parameter to obtain second quantized image data; and inputting the second quantized image data into the trained data enhancement model for inference to generate new infrared image data samples that have undergone data enhancement. The present invention creates new samples through cross-sampling non-uniform quantization technology and a data enhancement model, providing rich and well-structured training data, thereby improving the training effect of infrared small target detection models.

Description

Technical Field

[0001] The present invention relates to the technical field of infrared image data enhancement, and more particularly to a method and system for infrared image data enhancement based on quantization and diffusion models. Background Art

[0002] With the rapid development of computer vision technology in recent years, high-quality image data has become essential for the success of many vision tasks. However, in the field of infrared imagery, especially small infrared target detection, limited data quality has been a key bottleneck in improving model performance. Infrared small target images, due to weak signals and complex background noise, often exhibit blurred edges and low contrast, posing significant challenges to detection. In particular, under low signal-to-noise ratio conditions, traditional methods struggle to effectively distinguish small targets from the background, limiting their practical applications in remote sensing monitoring, military detection, and automated early warning.

[0003] Objects in infrared images usually appear as small targets due to the long shooting distance. In addition, they are interfered with by various factors, such as environmental factors (drones may be affected by fog or clouds) and equipment factors (sensors inevitably introduce noise). As a result, small infrared targets are submerged in various complex backgrounds, making detection quite difficult.

[0004] With the rapid development of deep learning technology, data-driven infrared small target detection methods have become a hot research area. These methods primarily rely on convolutional neural networks (CNNs) to extract image features and achieve precise target localization and segmentation through iterative training. However, a key prerequisite for achieving excellent detection results with these deep learning methods is the availability of sufficient training datasets. However, due to technical and equipment limitations, complex environmental conditions, privacy and security issues, and limitations on data sharing, acquiring infrared image data has become extremely difficult. Furthermore, collecting and manually annotating large amounts of data is a time-consuming and labor-intensive task, requiring significant human and material resources. Therefore, how to generate large amounts of infrared image data from limited infrared image data has become a pressing issue in current research.

[0005] Existing infrared small target data enhancement methods easily destroy the structural features and pixels of small targets through the generated images, which has an adverse impact on the precise detection task. At the same time, they rarely consider the data distribution information of infrared images, which affects the diversity and authenticity of infrared image data. Summary of the Invention

[0006] The purpose of the present invention is to provide an infrared image data enhancement method and system based on quantization and diffusion models, so as to overcome the problems that the existing infrared small target data enhancement methods easily destroy the structural features and pixels of the small targets through the generated images, which has an adverse effect on the precise detection task. At the same time, the data distribution information of the infrared image is less considered, which affects the diversity and authenticity of the infrared image data.

[0007] To achieve the above object, the technical solution of the present invention is as follows:

[0008] A method for infrared image data enhancement based on quantization and diffusion model includes the following steps:

[0009] Acquire original infrared image data and perform a non-uniform quantization operation using a first quantization parameter to obtain first quantized image data;

[0010] Establish a data enhancement model including a pixel coarse reconstruction module and a generation module based on a diffusion model;

[0011] Using the first quantized image data to train the pixel coarse reconstruction module to obtain first reconstructed image data, and using the first reconstructed image data to train the diffusion model-based generation module to obtain the trained data enhancement model;

[0012] performing a non-uniform quantization operation on the original infrared image data using a second quantization parameter to obtain second quantized image data;

[0013] The second quantized image data is input into the trained data enhancement model for inference, and the second reconstructed image data is obtained through the pixel coarse reconstruction module and input into the diffusion model-based generation module to generate a new infrared image data sample after data enhancement.

[0014] Preferably, the first quantization parameter and the second quantization parameter are obtained by cross sampling the same normal distribution.

[0015] Preferably, the cross sampling to obtain the first quantization parameter and the second quantization parameter is specifically:

[0016] The mean is n and the variance is m 2 The first quantization parameter and the second quantization parameter are obtained by cross sampling based on the normal distribution of X1, X3, ..., X k , the second quantization parameter is X2, X4..., X k-1 , or, the first quantization parameter is X2, X4..., X k-1 , the second quantization parameters are X1, X3, ..., X k, where X k -X k-1 =m, and X k , n, m, and k are all positive integers.

[0017] Preferably, the performing of the non-uniform quantization operation specifically includes:

[0018] A non-uniform quantizer with random intervals and values, wherein the non-uniform quantizer includes a set of non-overlapping intervals S i ={[a i ,a i+1 )}, a set of replicated values ​​y i and a value Num representing the number of intervals, and Num is the first quantization parameter or the second quantization parameter, a i It is obtained by the following formula:

[0019] a0,a1,...,a n-1 =sort(a ′ 0,a ′ 1,...,a ′ n-1 )

[0020] a′ i =U(min(x),max(x)),i=0,1,...,Num-1

[0021] Among them, sort is a sorting algorithm, U represents random sampling in the interval, min(x) and max(x) represent the minimum and maximum values ​​of the pixels in each channel of the image respectively, and the image pixel values ​​falling in the interval are copied by the value y. i replace:

[0022] y i =U(a i ,a i+1 ),i=0,1,...,Num-1

[0023] Different quantization interval sizes are selected, and the non-uniform quantizer is used to discretize continuous pixel values, and the image is compressed to different degrees to reduce data complexity while retaining the structural features of the image at different scales.

[0024] Preferably, the pixel coarse reconstruction module includes three parts: shallow feature extraction, deep feature extraction and image reconstruction;

[0025] The shallow feature extraction part extracts the shallow features of the quantized image data through a 3×3 convolution layer and divides the data into multiple non-overlapping blocks for embedding;

[0026] The deep feature extraction part extracts deep features from the block embedding through several normalization layers and attention layers;

[0027] The image reconstruction part fuses the shallow features and the deep features, and then performs three interpolation upsampling to obtain reconstructed image data with the same resolution as the original infrared image data, wherein each interpolation is followed by a convolution layer and a Leaky ReLU activation layer.

[0028] Preferably, the pixel coarse reconstruction module is trained and optimized with the goal of minimizing the pixel loss between the first reconstructed image data and the input first quantized image data, and the loss function of the pixel loss is as follows:

[0029]

[0030] Among them, I restore represents a pixel of the first reconstructed image data, I input represents a pixel of the first quantized image data as input.

[0031] Preferably, the generation module based on the diffusion model includes a potential diffusion model based on the U-Net network, including an encoder ε and a decoder

[0032] Preferably, when the first reconstructed image data is used to train the generation module based on the diffusion model, the first reconstructed image data is used as the input image I in , which is encoded into latent space variables by the encoder ε during training Then gradually add Gaussian noise to obtain a potential image close to the Gaussian distribution Generate denoised image by t-step resampling at last, Through the decoder Generate a resampled image by minimizing the loss function To optimize the generation module based on the diffusion model, so that the generated image is closer to the real image, the loss function as follows:

[0033]

[0034] By minimizing the deviation between the noise of the real image and the noise of the generated image, the generation module based on the diffusion model can predict the data distribution of the infrared image and generate infrared image data that is closer to the real one.

[0035] Preferably, the specific steps of generating infrared image data using the diffusion model-based generation module include:

[0036] The input reconstructed image data I is encoded into a potential representation by the encoder ε Then at each time step t, we add a hyperparameter β t ∈(0,1) Gaussian noise, generating the potential image

[0037]

[0038] in α t =1-β t , When the value of t is large enough, z t Approximate to the standard Gaussian distribution, the noise prediction network ∈ θ Trained as a denoising model and passed The loss function is optimized:

[0039]

[0040] in, Express expectations, represents a random variable in the latent space, c is the conditional variable, t is the time step, representing a step in the diffusion process, and ∈ is the Gaussian noise sampled from standard Gaussian noise;

[0041] Use by After the loss function is optimized, the denoising model performs t-step resampling to remove noise in the potential image and generate a denoised image. The denoised image Through the decoder Generate a resampled image, i.e., infrared image data.

[0042] To achieve the purpose of the invention, the present invention further provides an infrared image data enhancement system based on quantization and diffusion model, and an infrared image data enhancement method based on quantization and diffusion model using the above technical solution, the system comprising:

[0043] a cross quantization unit, configured to perform a non-uniform quantization operation on the original infrared image data using a first quantization parameter to obtain first quantized image data, or to perform a non-uniform quantization operation on the original infrared image data using a second quantization parameter to obtain second quantized image data;

[0044] A modeling unit, configured to establish a data enhancement model including a pixel coarse reconstruction module and a generation module based on a diffusion model;

[0045] a training unit, configured to train the pixel coarse reconstruction module using the first quantized image data to obtain first reconstructed image data, and to train the diffusion model-based generation module using the first reconstructed image data to obtain the trained data enhancement model;

[0046] The infrared data image generation unit is used to input the second quantized image data into the trained data enhancement model for inference, obtain the second reconstructed image data through the pixel coarse reconstruction module and input it into the diffusion model-based generation module to generate data-enhanced infrared image data samples.

[0047] Compared with the prior art, the present invention has the following beneficial effects:

[0048] The present invention creates new samples by performing multi-scale feature compression on the infrared small target dataset through non-uniform quantization operations, which will not damage the small target pixels and well preserves the structural features of the small targets. Through the generation module based on the diffusion model, the integrity of the target is retained in the process of generating infrared image data, and the Gaussian distribution is used to control the sample diversity. Even in the case of few samples, rich and structurally complete training data can be generated, which significantly improves the training effect of the infrared small target detection model in the few-sample scenario. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] Figure 1 This is a schematic diagram of the steps of an infrared image data enhancement method based on quantization and diffusion model of the present application;

[0050] Figure 2 This is a schematic diagram of the cross sampling process of Example 1 of the present application;

[0051] Figure 3 This is a schematic diagram of the non-uniform quantization process of Example 1 of the present application;

[0052] Figure 4 Result diagram of the quantized image and the output image of the system in Example 2 of the present application;

[0053] Figure 5 A schematic diagram of a process for generating and outputting infrared image data during the inference phase using an infrared image data enhancement system based on a quantization and diffusion model that has been trained using a loss function according to Example 2 of the present application;

[0054] Figure 6 A schematic diagram of the detection results of an infrared small target detection model using the method of the present invention to generate an infrared image training data set;

[0055] Figure 7 The figure is a schematic diagram of the detection results of the infrared small target detection model using the method of the present invention to generate an infrared image training data set. DETAILED DESCRIPTION

[0056] In order to more clearly understand the above-mentioned objects, features and advantages of the present invention, the present invention is further described in detail below in conjunction with the accompanying drawings and specific embodiments. It should be noted that, in the absence of conflict, the embodiments of the present application and the features therein can be combined with each other.

[0057] In the following description, many specific details are set forth to facilitate a full understanding of the present invention. However, the present invention may also be implemented in other ways different from those described herein. Therefore, the scope of protection of the present invention is not limited to the specific embodiments disclosed below.

[0058] Example 1

[0059] See also Figure 1 , Embodiment 1 of the present application provides an infrared image data enhancement method based on quantization and diffusion model, comprising the following steps:

[0060] S1: Acquire original infrared image data and perform a non-uniform quantization operation using a first quantization parameter to obtain first quantized image data.

[0061] Cross Sampling: See Figure 2 , Figure 2 This is a flow chart of cross sampling in Example 1 of the present application. The present invention samples different quantization parameters in the same normal distribution to obtain a first quantization parameter and a second quantization parameter, which are used in the training phase and the inference phase to quantize the image, so as to provide visual diversity. In the training phase, the first quantization parameter is used as the quantization parameter, and in the inference phase, the second quantization parameter is used as the quantization parameter. The cross sampling is as follows: for a mean of n and a variance of m 2 The first quantization parameter and the second quantization parameter are obtained by cross sampling based on the normal distribution of X1, X3, ..., X k , the second quantization parameter is X2, X4..., X k-1 , or, the first quantization parameter is X2, X4..., X k-1 , the second quantization parameters are X1, X3, ..., X k , where X k -X k-1 =m, and X k , n, m, and k are all positive integers.

[0062] Unlike traditional data augmentation methods, this approach does not destroy structural features when generating new samples. By sampling different sets of quantization parameters during training and inference, the generated images have richer textures and more accurate semantic distribution. By controlling sample diversity through a normal distribution, it can provide rich training data.

[0063] Non-uniform quantization: See Figure 3 , Figure 3 This is a flow chart of non-uniform quantization of Example 1 of this application. The present invention generates a large number of training data samples by quantizing and compressing infrared small target images, thereby improving the performance of the infrared small target detection model. Specifically, a non-uniform quantizer with random interval intervals and values ​​is set, and the non-uniform quantizer includes a non-overlapping interval set S i ={[a i ,a i+1 )}, a set of replicated values ​​y i , a value Num representing the number of intervals, and Num is the first quantization parameter or the second quantization parameter. i It is obtained by the following formula:

[0064] a0,a1,...,a n-1 =sort(a′0,a′1,...,a′ n-1 )

[0065] a′ i =U(min(x),max(x)),i=0,1,...,Num-1

[0066] Among them, sort is a sorting algorithm, U represents random sampling within the interval, min(x) and max(x) represent the minimum and maximum pixel values ​​of each channel of the image respectively. The pixel values ​​of the image that fall within the interval are copied by the value y i replace:

[0067] y i =U(a i ,a i+1 ),i=0,1,...,Num-1

[0068] By selecting different quantization interval sizes and discretizing continuous pixel values ​​using a non-uniform quantizer, the image is compressed to varying degrees, reducing data complexity and achieving an image with multiple details compressed and modified. This quantization method, combined with a diffusion model, generates a wider variety of enhanced samples, effectively expanding the dataset and generating more diverse and challenging samples for the detection model.

[0069] S2: Establish a data enhancement model including a pixel coarse reconstruction module and a generation module based on a diffusion model.

[0070] The pixel coarse reconstruction module aims to learn the mapping representation from quantized and compressed images to complex images. By inputting images of different quantization levels into the trained model, target images with different details are generated. The module is mainly divided into three parts: shallow feature extraction module, deep feature extraction module and high-quality image reconstruction module. Specifically, the quantization interval and replication value of each image are randomly selected. The quantized image is subjected to shallow feature extraction through 3x3 convolution and divided into multiple non-overlapping blocks for embedding. Deep features are then extracted through multiple normalization layers and attention layers. After the shallow and deep features are fused, they are upsampled back to the original resolution through cubic interpolation. Each interpolation is followed by a convolution layer and a Leaky ReLU activation layer.

[0071] The generation module based on the diffusion model selects the latent diffusion model to generate prior information. The latent diffusion model reduces computational complexity by gradually "denoising" the data in the latent representation space, decoding it into a complete image, and compressing the image into the latent space. The latent diffusion model is based on the U-Net network structure, which includes an encoder ε and a decoder. The diffusion model's prior mechanism significantly enhances the realism of generated data, providing more representative samples for unsupervised learning and improving the model's learning and generalization capabilities. Images generated and processed by the diffusion model help the detection model learn more features, enhancing robustness and detection accuracy.

[0072] S3: Use the first quantized image data to train the pixel coarse reconstruction module to obtain first reconstructed image data, and use the first reconstructed image data to train the diffusion model-based generation module to obtain the trained data enhancement model.

[0073] The pixel coarse reconstruction module is trained and optimized with the goal of minimizing the pixel loss between the first reconstructed image data and the input first quantized image data. The loss function of the pixel loss is as follows:

[0074]

[0075] Among them, I restore represents a pixel of the first reconstructed image data, I input represents a pixel of the first quantized image data as input.

[0076] When the first reconstructed image data is used to train the generation module based on the diffusion model, the first reconstructed image data is used as the input image I in , which is encoded into latent space variables by the encoder ε during training Then gradually add Gaussian noise to obtain a potential image close to the Gaussian distribution Noise prediction network ε θTrained as a denoising model and passed Loss function for training optimization:

[0077]

[0078] in, Express expectations, represents a random variable in the latent space, c is the conditional variable, t is the time step, representing a step in the diffusion process, and ε is the Gaussian noise sampled from standard Gaussian noise.

[0079] Generate denoised images by performing t-step resampling through the denoising model at last, Through the decoder Generate a resampled image by minimizing the loss function To optimize the generation module based on the diffusion model, so that the generated image is closer to the real image, the loss function as follows:

[0080]

[0081] By minimizing the deviation between the noise of the real image and the noise of the generated image, the generation module based on the diffusion model can predict the data distribution of the infrared image and generate infrared image data that is closer to the real one.

[0082] The trained latent diffusion model learns the image distribution, capturing object shape and texture, thereby generating images that conform to the true distribution and have reasonable semantics. To enable the diffusion model to generate high-quality infrared image data in the target domain, it was fine-tuned using the first reconstructed image data generated by the pixel-based coarse reconstruction module after quantization of the infrared small target dataset. By learning from a large number of infrared small target images, the model understands the image background semantics and the shape and texture characteristics of small targets, thereby generating realistic and high-quality infrared images. By learning the image distribution, the diffusion model incorporates real-world knowledge. This captures latent data characteristics without the need for additional labels, generating more realistic and diverse samples for dataset reconstruction.

[0083] S4: performing a non-uniform quantization operation on the original infrared image data using a second quantization parameter to obtain second quantized image data.

[0084] S5: Input the second quantized image data into the trained data enhancement model for inference, obtain the second reconstructed image data through the pixel coarse reconstruction module and input it into the diffusion model-based generation module to generate a new infrared image data sample after data enhancement.

[0085] The second reconstructed image data output by the pixel coarse reconstruction module is used as the input image I and is compressed into a potential representation by the encoder ε Then at each time step t, we add a hyperparameter β t ∈(0,1) Gaussian noise, generating the potential image

[0086]

[0087] in α t =1-β t and When the value of t is large enough, Approximately a standard Gaussian distribution. Use After the loss function is optimized, the denoising model performs t-step resampling to remove noise in the potential image and generate a denoised image. The denoised image Through the decoder Generate a resampled image, i.e., infrared image data, and complete the data enhancement of the infrared image.

[0088] Example 2

[0089] This embodiment 2 is based on embodiment 1 and applies the infrared image data enhancement method based on quantization and diffusion model in embodiment 1 to provide an infrared image data enhancement system based on quantization and diffusion model. The system is specifically as follows:

[0090] a cross quantization unit, configured to perform a non-uniform quantization operation on the original infrared image data using a first quantization parameter to obtain first quantized image data, or to perform a non-uniform quantization operation on the original infrared image data using a second quantization parameter to obtain second quantized image data;

[0091] A modeling unit, configured to establish a data enhancement model including a pixel coarse reconstruction module and a generation module based on a diffusion model;

[0092] a training unit, configured to train the pixel coarse reconstruction module using the first quantized image data to obtain first reconstructed image data, and to train the diffusion model-based generation module using the first reconstructed image data to obtain the trained data enhancement model;

[0093] The infrared data image generation unit is used to input the second quantized image data into the trained data enhancement model for inference, obtain the second reconstructed image data through the pixel coarse reconstruction module and input it into the diffusion model-based generation module to generate data-enhanced infrared image data samples.

[0094] See also Figure 4 , Figure 4 This is a result diagram of the quantized image and the output image of the system in Example 2 of the present application.

[0095] See also Figure 5 , Figure 5 This is a flow chart illustrating the process of generating and outputting infrared image data during the inference phase using an infrared image data enhancement system based on a quantization and diffusion model that has been trained using a loss function according to Example 2 of the present application. An infrared small target image in an infrared dataset is input as an input image into a cross quantization unit for non-uniform quantization to obtain a quantized image, which is then input into an infrared data image generation unit. A second reconstructed image data is generated through a pixel coarse reconstruction module and input into a generation module based on a diffusion model to generate an output image for expanding the infrared small target image dataset.

[0096] The other steps of this embodiment 2 are the same as those of embodiment 1 and will not be repeated here.

[0097] Example 3

[0098] This embodiment 3 is based on embodiment 1. An infrared image data enhancement method based on quantization and diffusion model in embodiment 1 is used to conduct experiments to verify the effectiveness of the method of the present invention. The details are as follows:

[0099] The NUDT-SIRST dataset and the SIRST dataset were selected for experiments. The SIRST dataset is a dataset specially constructed for single-frame infrared small target detection. It selects representative pictures from infrared sequences of hundreds of different scenes. The dataset contains 427 infrared images and 480 targets. The targets in the images are confirmed by observing the movement sequences to ensure that they are real targets rather than pixel-level impulse noise. The NUDT-SIRST dataset is an artificially synthesized dataset of five main background scenes: city, field, highlight, ocean, and cloud. It contains a variety of small targets such as points, airplanes, drones, and ships. Each image is synthesized from a real background of different targets (e.g., points, airplanes, and extended targets) with different SCRs and rich postures. The total amount of data is 1327.

[0100] 1. Pixel Coarse Reconstruction Module Training Phase: To generate diverse remote sensing images and expand the infrared small target dataset, the pixel coarse reconstruction module of the data augmentation model is trained. During the training phase, we sample from a normal distribution with a mean of 17 and a variance of 4 to obtain the first quantization parameters [7, 9, 11, ..., 23, 25, 27]. Using this first quantization parameter, we perform a non-uniform quantization operation on the original small target remote sensing image to compress the image information. The quantized image is then used as a training sample for the pixel coarse reconstruction module. The training batch size for the pixel coarse reconstruction module is set to 3, the number of training steps is set to 25,000, and the learning rate is set to 0.001. After training, the first reconstructed image data is output as training samples for subsequent modules.

[0101] 2. Diffusion model-based generation module training phase: During the training process, the diffusion model based on the U-Net network structure pre-trained on ImageNet is loaded as the pre-training model of the generation module. The first reconstructed image data quantized and restored by the pixel coarse reconstruction module is used as training data. The batch size, number of training steps, and learning rate are set to 1, 20, and 0.01 respectively for training fine-tuning.

[0102] 3. Generating infrared images for data augmentation inference: After training both modules in the data augmentation model, for the inference phase, another set of quantized values ​​was sampled from the same normal distribution, with the second quantization parameter set to [8, 10, 12, .., 22, 24, 26]. Non-uniform quantization was performed on both NUDT-SIRST and SIRST. The diffusion model-based generation module used time-interval skipping DDPM sampling for resampling, with a time step of 50. Double the image data was generated on the SIRST and NUDT-SIRST training sets.

[0103] 4. Testing and evaluation: In order to evaluate the performance of the infrared image dataset generated by the method of the present invention, the infrared small target detection model is trained with the generated data and tested with the test set. The intersection over union ratio, detection probability and false alarm rate are used as evaluation indicators.

[0104]

[0105]

[0106] Table 1 Experimental results

[0107] See also Figure 6 and Figure 7 , Figure 6 and Figure 7This is a schematic diagram of the detection results of different infrared small targets using the infrared small target detection model generated using the infrared image training dataset using the method of the present invention. Please refer to Table 1. The experimental results show that the data enhancement method of the present invention significantly improves the performance of the detection model, generating higher-quality images and better visual effects. In particular, it demonstrates strong adaptability and robustness in the infrared small target detection task. Through this infrared small target data enhancement method based on quantization operations and diffusion models, combined with an internal and external collaborative data enhancement strategy, the quality of infrared images and the accuracy of detection are improved, providing an efficient and reliable data enhancement solution for infrared small target detection in complex scenarios.

[0108] The other steps of this embodiment 3 are the same as those of embodiment 1 and will not be repeated here.

[0109] In summary, the present invention proposes a method for infrared small target data enhancement based on quantization operation and diffusion model, which improves the performance of the model in complex scenes through a data enhancement strategy of quantization and pixel reconstruction. In terms of quantization operation, cross sampling is used to obtain different quantization parameters for non-uniform quantization. Different quantization parameters are used in the training stage and the inference stage to perform different degrees of quantization compression on the image. In the pixel coarse reconstruction module, shallow and deep features are extracted from the quantized image and fused, and the compressed quantized pixels are coarsely reconstructed. In the generation module based on the diffusion model, the distribution of the data set is learned through the diffusion model, and real-world knowledge is introduced into the image through the resampling strategy, so that the generated infrared image is more diverse and realistic.

[0110] Obviously, the above embodiments of the present invention are merely examples for the purpose of clearly illustrating the present invention, and are not intended to limit the embodiments of the present invention. Those skilled in the art will appreciate that other variations or modifications can be made based on the above description. It is not necessary and impossible to enumerate all embodiments here. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention shall be included within the scope of protection of the claims of the present invention.

Claims

1. A method for infrared image data enhancement based on quantization and diffusion model, characterized in that: The following steps are involved: Acquire original infrared image data and perform a non-uniform quantization operation using a first quantization parameter to obtain first quantized image data; Establish a data enhancement model including a pixel coarse reconstruction module and a generation module based on a diffusion model; Using the first quantized image data to train the pixel coarse reconstruction module to obtain first reconstructed image data, and using the first reconstructed image data to train the diffusion model-based generation module to obtain the trained data enhancement model; performing a non-uniform quantization operation on the original infrared image data using a second quantization parameter to obtain second quantized image data; Inputting the second quantized image data into the trained data enhancement model for inference, obtaining second reconstructed image data through the pixel coarse reconstruction module and inputting the second reconstructed image data into the diffusion model-based generation module to generate a new infrared image data sample after data enhancement; The performing of the non-uniform quantization operation includes: A non-uniform quantizer with random intervals and values, wherein the non-uniform quantizer includes a set of non-overlapping intervals , a set of replicated values and a value representing the number of intervals and That is, the first quantization parameter or the second quantization parameter, It is obtained by the following formula: in, is the sorting algorithm, represents random sampling within the interval, Represents the minimum and maximum values ​​of the pixels of each channel of the image respectively. The image pixel values ​​that fall within the interval are copied by replace: Different quantization interval sizes are selected, and the non-uniform quantizer is used to discretize continuous pixel values, and the image is compressed to different degrees to reduce data complexity while retaining the structural features of the image at different scales.

2. The infrared image data enhancement method based on quantization and diffusion model according to claim 1, characterized in that: The first quantization parameter and the second quantization parameter are obtained by cross-sampling the same normal distribution.

3. The infrared image data enhancement method based on quantization and diffusion model according to claim 2, characterized in that: The cross sampling to obtain the first quantization parameter and the second quantization parameter is specifically: The mean is , the variance is The first quantization parameter and the second quantization parameter are obtained by cross sampling based on the normal distribution of , ,..., , the second quantization parameter is , ..., , or, the first quantization parameter is , ..., , the second quantization parameter is , ,..., ,in ,and 、 、 All are positive integers.

4. The infrared image data enhancement method based on quantization and diffusion model according to claim 1, characterized in that: The pixel coarse reconstruction module includes three parts: shallow feature extraction, deep feature extraction and image reconstruction; The shallow feature extraction part is carried out through 3 The convolutional layer 3 extracts shallow features of the quantized image data and divides it into multiple non-overlapping blocks for embedding; The deep feature extraction part extracts deep features from the block embedding through several normalization layers and attention layers; The image reconstruction part fuses the shallow features and the deep features, and then performs three interpolation upsampling to obtain reconstructed image data with the same resolution as the original infrared image data, wherein each interpolation is followed by a convolution layer and a Leaky ReLU activation layer.

5. The infrared image data enhancement method based on quantization and diffusion model according to any one of claims 1 or 4, characterized in that: The pixel coarse reconstruction module is trained and optimized with the goal of minimizing the pixel loss between the first reconstructed image data and the input first quantized image data. The loss function of the pixel loss is as follows: in, representing pixels of the first reconstructed image data, represents a pixel of the first quantized image data as input.

6. The infrared image data enhancement method based on quantization and diffusion model according to claim 1, characterized in that: The generation module based on the diffusion model includes a potential diffusion model based on the U-Net network, including an encoder and decoder .

7. The infrared image data enhancement method based on quantization and diffusion model according to claim 6, characterized in that: When the generation module based on the diffusion model is trained using the first reconstructed image data, the first reconstructed image data is used as an input image. , through the encoder during training Encoded as latent space variables , and then gradually add Gaussian noise to obtain a potential image close to the Gaussian distribution ,pass Step resampling to generate denoised image ,at last, Through the decoder Generate a resampled image by minimizing the loss function To optimize the generation module based on the diffusion model, so that the generated image is closer to the real image, the loss function as follows: By minimizing the deviation between the noise of the real image and the noise of the generated image, the generation module based on the diffusion model can predict the data distribution of the infrared image and generate infrared image data that is closer to the real one.

8. The infrared image data enhancement method based on quantization and diffusion model according to claim 7, characterized in that: The specific steps of using the diffusion model-based generation module to generate infrared image data include: The reconstructed image data input Through the encoder Encoded as latent representation , then at each time step Add hyperparameters as Gaussian noise, generating the potential image : in , , ,when When the value of is large enough, Approximate to the standard Gaussian distribution, noise prediction network Trained as a denoising model and passed The loss function is optimized: in, Express expectations, represents a random variable in the latent space, is the conditional variable, is the time step, representing a step in the diffusion process, is Gaussian noise sampled from standard Gaussian noise; Use by The denoising model after the loss function is optimized is Step resampling removes noise from the latent image and generates a denoised image , the denoised image Through the decoder Generate a resampled image, i.e., infrared image data.

9. An infrared image data enhancement system based on quantization and diffusion model, applying an infrared image data enhancement method based on quantization and diffusion model as described in any one of claims 1 to 8, characterized in that: The system comprises: a cross quantization unit, configured to perform a non-uniform quantization operation on the original infrared image data using a first quantization parameter to obtain first quantized image data, or to perform a non-uniform quantization operation on the original infrared image data using a second quantization parameter to obtain second quantized image data; A modeling unit, configured to establish a data enhancement model including a pixel coarse reconstruction module and a generation module based on a diffusion model; a training unit, configured to train the pixel coarse reconstruction module using the first quantized image data to obtain first reconstructed image data, and to train the diffusion model-based generation module using the first reconstructed image data to obtain the trained data enhancement model; The infrared data image generation unit is used to input the second quantized image data into the trained data enhancement model for inference, obtain the second reconstructed image data through the pixel coarse reconstruction module and input it into the diffusion model-based generation module to generate data-enhanced infrared image data samples.

Images