An end-to-end turbulence suppression dataset augmentation method based on diffusion model

Abstract

The application provides an end-to-end turbulence suppression data set augmentation method based on a diffusion model, and is used for solving the technical problems that the existing method for accurately characterizing the space-time correlation of a turbulence degradation process has a large amount of calculation, a long calculation time, or the generated phase screen high and low frequency information is insufficient, and the visual effect on a clear true value image is poor. The end-to-end turbulence suppression data set augmentation method based on the diffusion model directly generates a new turbulence degradation image under a turbulence degradation condition image from a clear image through a diffusion model, realizes augmentation of an end-to-end turbulence suppression data set, can provide large-scale, high-quality and unlimitedly close-to-actual-observation image samples for a turbulence degradation image restoration algorithm based on data driving, realizes simulation of complex atmospheric turbulence conditions, and further improves the observation ability of a ground-based large-aperture optical telescope.

Description

TECHNICAL FIELD

[0001] The present application relates to a data set augmentation method, in particular to an end-to-end turbulent flow suppression data set augmentation method based on a diffusion model. BACKGROUND

[0002] In the 400 years of development of telescopes, the aperture of ground-based telescopes has been increasing, and their observation capabilities have been getting stronger, helping humans to constantly reveal the mysteries of the universe. However, in actual observation, the resolution capability of the telescope is limited by the atmospheric turbulence in the upper air and cannot reach the theoretical diffraction limit. Taking a 10m aperture telescope as an example, its corresponding theoretical diffraction limit is about 0.01 arcsec, but even the 10m Keck telescope located on Mauna Kea Mountain in Hawaii, one of the world's best astronomical sites, can only achieve a best resolution of 0.25 arcsec (corresponding to a wavelength of 500 nm) under the seeing limit.

[0003] Due to the highly chaotic nature of atmospheric turbulence itself, traditional algorithms have difficulty accurately depicting the strong spatiotemporal correlation of the turbulence degradation process influenced by multiple factors, thereby serving the data-driven turbulence degradation image restoration task. In an era of significant improvement in computing power, data-driven neural representation algorithms can rely on their strong nonlinear feature representation capabilities for image restoration. However, such algorithms rely on high-quality modeling of the turbulence degradation process, and how to accurately represent the spatiotemporal correlation of the turbulence degradation process becomes a basic prerequisite for data-driven neural representation algorithms.

[0004] Currently, numerical simulation is widely used due to its adjustable parameters and ease of implementation. Among them, the classic step-by-step propagation algorithm, the power spectral density method, and the Zernike polynomial method are commonly used methods for accurately representing the spatiotemporal correlation of the turbulence degradation process. The classic step-by-step propagation algorithm divides the effect of the atmospheric medium on the light wave into multiple phase screens, performs a pixel-by-pixel Fresnel diffraction integral calculation on the image, models the random fluctuation of the light wave phase in space and time caused by turbulence, and ultimately represents dynamic aberration. However, this method requires repeated Fourier transforms and inverse Fourier transforms, which is computationally intensive and time-consuming. The existing power spectral density method lacks low-frequency information due to the constraints of the Fourier transform itself, but existing research has shown that the distribution of turbulence power along the spatial frequency is nonlinear, and the spectral power increases sharply as the frequency moves to the low-frequency region, with turbulence energy mainly concentrated in the low-frequency region. The turbulence phase screen generated by the Zernike polynomial method has good results in the low-frequency part, but the high-frequency components are insufficiently sampled. The lack of high and low frequency information in the phase screen generated by the above methods results in poor visual effects when applied to clear true value images. SUMMARY

[0005] The application aims to solve the technical problems of large amount of calculation, long calculation time, or insufficient high and low frequency information of the generated phase screen, and poor visual effect on the clear true value image of the existing method for accurately characterizing the space-time correlation of the turbulence degradation process, and provide an end-to-end turbulence suppression data set augmentation method based on a diffusion model.

[0006] In order to achieve the above-mentioned purpose, the technical scheme provided by the application is as follows:

[0007] An end-to-end turbulence suppression data set augmentation method based on a diffusion model, characterized by comprising the following steps:

[0008] Step 1, constructing a training set and a test set; the data in the training set includes corresponding turbulence degradation images D, clear images V and turbulence degradation condition images T, and the data in the test set includes corresponding clear images V and turbulence degradation condition images T;

[0009] Step 2, building a diffusion model, the diffusion model includes a forward noise adding unit, a noise prediction network and a reverse denoising unit connected in turn according to input and output;

[0010] Step 3, generating a random Gaussian noise ε with the same size as the turbulence degradation image D, adding the random Gaussian noise ε to the turbulence degradation image D through the forward noise adding unit to obtain a noisy image D i ;

[0011] Step 4, splicing the noisy image D i at different time steps with the clear image V and the turbulence degradation condition image T in the training set respectively according to the channel, and then inputting the spliced image and the corresponding time step into the noise prediction network for iterative training to obtain a trained noise prediction network; when the noise prediction network is iteratively trained, the noise prediction network is optimized through a noise prediction loss function, a structure loss function and an edge loss function;

[0012] Step 5, modifying the tilt direction, tilt action radius, blur action radius and blur degree of the turbulence degradation condition image T in the test set to obtain a new turbulence degradation condition image T new ; at the same time, defining a new random Gaussian noise ε1 with the same size as the turbulence degradation image D, and splicing the new random Gaussian noise ε1, the clear image V in the test set and the new turbulence degradation condition image T new according to the channel, and then inputting the spliced image into the trained noise prediction network to make the trained noise prediction network output a predicted noise ;

[0013] Step 6, obtaining the predicted noise Reverse denoising is performed to obtain a new turbulence degradation condition map T new and a turbulence degradation image D new under the turbulence degradation condition map T.

[0014] Further, in step 5, the tilt direction, tilt action radius, blur action radius and blur degree of the turbulence degradation condition map T in the test set are modified, specifically as follows:

[0015] The turbulence degradation condition map T in the test set includes four channels. Each channel uses an arbitrary real number in the range of 0-2 as the corresponding influence factor to multiply the pixel value of the corresponding field of view of the turbulence degradation condition map T to modify the turbulence degradation condition map T. The modification of the pixel value of each field of view in the first channel of the turbulence degradation condition map T realizes the modification of the tilt direction, the modification of the pixel value of each field of view in the second channel realizes the modification of the tilt action radius, the modification of the pixel value of each field of view in the third channel realizes the modification of the blur action radius, and the modification of the pixel value of each field of view in the fourth channel realizes the modification of the blur degree.

[0016] Further, in step 2, an array is arranged in the forward noise adding unit to store noise scheduling parameters at different time steps.

[0017] Further, in step 2, the noise scheduling parameters at different time steps are obtained by the following steps:

[0018] Step a1, define a sequence η including t numbers, η = [η1, η2, …, η t ];

[0019] Step b1, calculate the scheduling sequence α i according to the following formula:

[0020] α i = 1-η i , i = 1, 2, …, t;

[0021] Step c1, calculate the noise scheduling parameters at different time steps according to the scheduling sequence α i , and store the noise scheduling parameters at different time steps in the array after being defined and initialized according to the specified rules.

[0022] Further, in step 3, the random Gaussian noise ε is added to the turbulence degradation image D by the forward noise adding unit to obtain the noisy image D i at different time steps, specifically as follows:

[0023] The random Gaussian noise ε is added to the turbulence degradation image D by the noise scheduling parameters at different time steps stored in the forward noise adding unit to obtain the noisy image Di , and the specific expression is as follows:

[0024] , i = 1, 2, …, t;

[0025] wherein, is the noise scheduling parameter at time step i.

[0026] Further, in step 4, the noise prediction network is optimized by the noise prediction loss function, the structure loss function, and the edge loss function, specifically:

[0027] Step a2, using the noise prediction loss function, minimizing and optimizing the distance between the output of the noise prediction network during iterative training and the random Gaussian noise ε;

[0028] Step b2, calculate the clean turbulence degradation image at different time steps:

[0029]

[0030] Step c2, using the structure loss function, minimizing and optimizing the distance between the calculated clean turbulence degradation image at different time steps and the turbulence degradation image D;

[0031] Step d2, using the edge loss function, minimizing and optimizing the distance between the calculated clean turbulence degradation image at different time steps and the turbulence degradation image D.

[0032] Further, in step 4, the noise prediction loss function is a Charbonnier loss function;

[0033] The structure loss function is a multi-scale structure function;

[0034] The edge loss function is a total variation (TV) function.

[0035] Further, in step 6, the reverse denoising unit uses a DDIM sampling method for reverse denoising.

[0036] Further, in step 6, the reverse denoising unit performs reverse denoising on the predicted noise obtained in step 5, to obtain a new turbulence degradation image D new under the turbulence degradation condition T new Specifically:

[0037] Step a3, calculate the denoised image at time step t by the following formula: ​

[0038]

[0039] wherein, is the noise schedule parameter at time step t;

[0040] Step b3, calculate the turbulence degradation image at time step t-1 by the following formula :

[0041]

[0042] wherein, is the noise schedule parameter at time step t-1;

[0043] Step c3, iteratively calculate the turbulence degradation image at all time steps by the method of step b, after the iterative calculation is completed, obtain the new turbulence degradation condition image T new under the turbulence degradation image D new .

[0044] Further, in step 1, the turbulence degradation condition image T is obtained by the following way:

[0045] Subtract the pixel values of the corresponding clear image V and turbulence degradation image D at the same position respectively, and then take the absolute value to obtain the turbulence degradation condition image T.

[0046] The beneficial effects of the present application compared with the prior art are as follows:

[0047] 1. The end-to-end turbulence suppression data set augmentation method based on the diffusion model provided by the present application directly generates a new turbulence degradation image under a turbulence degradation condition image from a clear image through a diffusion model, realizes augmentation of an end-to-end turbulence suppression data set, can provide large-scale, high-quality and unlimitedly close-to-actual-observation image samples for a turbulence degradation image restoration algorithm based on data driving, realizes simulation of complex atmospheric turbulence conditions, and further improves the observation ability of a ground-based large-aperture optical telescope; the present application not only omits the complex calculation process of Fourier transform, but also, compared with the existing power spectral density method and Zernike polynomial method, solves a series of problems such as slow simulation speed, low scene adaptability and poor simulation precision by using the powerful nonlinear expression capability of a noise prediction network.

[0048] 2. The end-to-end turbulence suppression data set augmentation method based on the diffusion model can generate a turbulence degradation image with a specified tilt direction, tilt action radius, blur action radius and blur degree.

[0049] 3. The end-to-end turbulence suppression data set augmentation method based on a diffusion model provided by the application can obtain time sequence related turbulence degradation sequences under the same new turbulence degradation condition graph through processing of the clear image for the target with short time motion.

[0050] 4. The application optimizes the noise prediction network through the noise prediction loss function, the structure loss function and the edge loss function to retain high frequency components, thereby realizing fast and high-precision generation of turbulence degradation images under new turbulence degradation condition graphs and effectively overcoming the problem of high frequency detail loss of turbulence degradation images. BRIEF DESCRIPTION OF DRAWINGS

[0051] Figure 1 The distribution difference diagram of the image of the target in step 5 of the embodiment of the application after being affected by atmospheric turbulence in different field of view spaces of the optical system is shown in the figure.

[0052] Figure 2 The change diagram of the tilt action radius and the tilt direction when the turbulence degradation condition graph T in the test set is divided into 4x4 local blocks in step 5 of the embodiment of the application is shown in the figure, wherein (a) is the tilt action radius and the tilt direction diagram of block 1 and block 2 in the 4x4 local block at the initial moment, and (b) is the tilt action radius and the tilt direction diagram of block 1 and block 2 in the 4x4 local block at the next moment after being affected by the change of the atmospheric transmission medium. DETAILED DESCRIPTION

[0053] In order to make the advantages and characteristics of the application more clear, the application will be further described in detail below in combination with the drawings and specific embodiments.

[0054] The end-to-end turbulence suppression data set augmentation method based on a diffusion model specifically includes the following steps:

[0055] Step 1, constructing a training set and a test set.

[0056] The data in the training set includes a turbulence degradation image D, a clear image V and a turbulence degradation condition graph T, and the data in the test set includes a clear image V and a turbulence degradation condition graph T.

[0057] Among them, the turbulence-degraded image D and the clear image V correspond one to one. There are many ways to obtain the turbulence-degraded image D and the clear image V. For example, in the morning in hot and arid areas, when the ground gradually warms up, turbulence is likely to occur. At this time, the target can be photographed to obtain the turbulence-degraded image D, and then the turbulence-degraded image D can be restored using the turbulence image restoration algorithm to obtain the clear image V. In addition, the turbulence-degraded image D and the clear image V of the target can be directly photographed under low-temperature and windless weather conditions, or the clear image V of the target can be photographed under low-temperature and windless weather conditions, and then the clear image V can be degraded using numerical simulation methods to obtain the turbulence-degraded image D.

[0058] Then, the obtained corresponding clear image V and turbulence-degraded image D are subtracted pixel by pixel, that is, the pixel values ​​at the same position of the corresponding clear image V and turbulence-degraded image D are subtracted respectively, and then the absolute value is taken to obtain the turbulence degradation condition map T.

[0059] Step 2: Build a diffusion model.

[0060] The diffusion model includes a forward denoising unit, a noise prediction network and a reverse denoising unit which are connected in sequence according to input and output.

[0061] The forward noise adding unit is provided with an array. First, define a sequence η including t numbers, η=[η1, η2,…, η t ], and then calculate the scheduling sequence α according to the following formula i :

[0062] α i =1-η i , i=1,2,…,t;

[0063] Among them, η1 and η t Usually take 10 -6 and 10 -2 , η1 and η t The intermediate values ​​are obtained by adding the tolerance d in sequence. The calculation formula of the tolerance d is: d= .

[0064] The noise scheduling parameters at different time steps are calculated by the following formula: :

[0065] ;

[0066] The noise scheduling parameters at different time steps Defined and initialized according to the specified rules and stored in an array for later call.

[0067] The reverse denoising unit adopts the DDIM sampling method for reverse denoising, defines the total number of reverse denoising steps total_step as 2000, and the sampling step number num_step as 20. First, a sampling time step array current_timestep is defined, the length of the sampling time step array current_timestep is consistent with the sampling step number num_step, and all elements in the sampling time step array current_timestep are an arithmetic sequence with a common difference of total_step / num_step, i.e. 100. Then, a sampling time step array prev_timestep is defined, the sampling time step array prev_timestep is used to store the arithmetic sequence obtained by adding 0 to the head of the sampling time step array current_timestep and discarding the last element, for example, when the sampling time step array current_timestep is [1 101 201 301 1701 1801 1901], the sampling time step array prev_timestep is [0 1 101 201 1701 1801].

[0068] Then, the sampling time step array current_timestep and the sampling time step array prev_timestep are respectively processed in reverse order, the sampling time step array current_timestep after reverse processing is [1901 1801 1701 301 201 101 1], and the sampling time step array prev_timestep after reverse processing is [1801 1701 201 1011 0].

[0069] When reverse denoising is performed, the number of cycles is the sampling step number num_step, i.e. 20 cycles, and the elements in the sampling time step array current_timestep after reverse processing and the elements in the sampling time step array prev_timestep after reverse processing are taken in turn in the whole cycle process, the element taken from the sampling time step array current_timestep after reverse processing is recorded as the sampling time step t_batch, and the element taken from the sampling time step array prev_timestep after reverse processing is recorded as the sampling time step prev_t_batch. The sampling time step t_batch and the sampling time step prev_t_batch are used as the index sequence number of the time step t corresponding to the access noise scheduling parameter when the sampling time step t_batch and the sampling time step prev_t_batch are taken from the sampling time step array current_timestep after reverse processing and the sampling time step array prev_timestep after reverse processing.

[0070] Step 3, forward noise adding.​

[0071] A random Gaussian noise ε of the same size as the turbulence degradation image D is generated, .

[0072] The random Gaussian noise ε is added to the turbulence degradation image D through the noise scheduling parameters of different time steps stored in the forward noise adding unit, to obtain the noisy image D at different time steps i , and the specific expression is as follows:

[0073] , i = 1, 2, …, t.

[0074] Step 4, the noisy image D at different time steps i is spliced with the clear image V and the turbulence degradation condition image T in the training set by channel, and after splicing, it is sent into the noise prediction network together with the corresponding time step for iterative training to obtain the trained noise prediction network. Among them, the clear image V and the turbulence degradation condition image T are used as the control condition of the noise prediction network, and the output of the noise prediction network during iterative training is denoted as .

[0075] When the noise prediction network is iteratively trained, the present application optimizes the noise prediction network through the noise prediction loss function , the structure loss function and the edge loss function , and the specific expression is as follows:

[0076] ;

[0077] Among them, , , are the weighting coefficients of the noise prediction loss, the structure loss and the edge structure loss respectively, which can be adjusted according to the iterative training process of the noise prediction network.

[0078] The noise prediction loss function in the present embodiment adopts the Charbonnier loss function, which is used to minimize and optimize the distance between the output of the noise prediction network during training and the random Gaussian noise ε.

[0079] The clean turbulence degradation image at different time steps is back calculated by the following formula:

[0080]

[0081] The structure loss function adopts a multi-scale structure function (MS-SSIM), which is used to calculate the clean turbulence degradation image at different time steps​ minimizing the distance between the degraded image D and the clean image Dc.

[0082] The edge loss function adopts a total variation (TV) function, and the clean turbulence-degraded images at different time steps are calculated minimizing the distance between the degraded image D and the clean image Dc.

[0083] Step 5, obtain a new turbulence-degraded condition map T new .

[0084] As shown in Figure 1 , the image of the target formed by the optical system is further degraded after being affected by atmospheric turbulence, and the degree of degradation is different due to the difference in the distribution of atmospheric turbulence in different field-of-view spaces of the system. In order to illustrate the degree of image degradation caused by the difference in the distribution of turbulence, the object plane is divided into different field-of-view regions, and the centroid position of each field-of-view is used to represent the field-of-view region. The coordinate values of the centroid positions of two field-of-views on the object plane are denoted as and , and the atmospheric transmission medium parameter characteristics representing the corresponding field-of-views are denoted as and . When the system observes and images the target, the target is affected by atmospheric turbulence when passing through the atmospheric transmission medium, and the field-of-views and are offset to become and on the image plane . The centroid coordinates and of the aforementioned two field-of-views and of the target should normally correspond to the image plane and , but they correspond to the image plane after passing through different atmospheric transmission medium parameters and , and the offset amounts , It can be seen that the offset amounts of the two targets are different, that is, the difference in the spatial distribution of atmospheric turbulence causes the degrees of image degradation of different field-of-views to be different.

[0085] As shown in Figure 2As shown, the turbulence degradation condition map T in the test set is divided into MxN local blocks, and in this embodiment, M=N=4, and the center of each local block is the center of the tilt action radius and the blur action radius. Taking the tilt action radius as an example, the blocks 1 and 2 in the 4x4 local block at the initial moment have tilt action radii r1 and r2, respectively, and after the influence of the atmospheric transmission medium, the tilt action radii become r3 and r4 at the next moment. The change of the blur action radius is similar to that of the tilt action radius, and the changes of r1, r2 to r3, r4 can be regarded as the changes of the blur action radius on the blocks 1 and 2 in the 4x4 local block. At the initial moment, the tilt directions of the blocks 1 and 2 in the MxN local block are and respectively, and after the influence of the change of the atmospheric transmission medium, the tilt directions become and respectively at the next moment.

[0086] The present application specifies the tilt direction and the tilt action radius pixel by pixel by using the pixel-by-pixel turbulence degradation condition map T, and models the high-order aberrations such as defocus, astigmatism and coma as local blur to specify the blur action radius and the blur degree. Specifically, the turbulence degradation condition map T is expressed as T=[F1,F2]∈ , wherein F1 represents the tilt degradation parameter, F1=[u;v]∈ , u and v represent the tilt direction and the tilt action radius, respectively; F2 represents the blur degradation parameter, F2=[m;n]∈ , m and n represent the blur action radius and the blur degree of the blur aberration, respectively, , and R represents the real number field, H represents the height of the clear image V, W represents the width of the clear image V, and 4 and 2 represent the channel number.

[0087] The turbulence degradation condition map T includes four channels, and when the turbulence degradation condition map T in the test set is modified, different fields of view in each channel of the turbulence degradation condition map T are modified by using random decimals in the range of 0-2 as corresponding influence factors, the influence factors of different fields of view in each channel are multiplied by the pixel value of the corresponding field of view in the turbulence degradation condition map T, that is, the influence factors of different fields of view in each channel are multiplied by the pixel value of the corresponding field of view in the turbulence degradation condition map T, so as to achieve the purpose of modifying the turbulence degradation condition map T. Specifically, the modification of the tilt direction is realized by modifying the pixel value of each field of view in the first channel of the turbulence degradation condition map T, the modification of the tilt action radius is realized by modifying the pixel value of each field of view in the second channel, the modification of the blur action radius is realized by modifying the pixel value of each field of view in the third channel, and the modification of the blur degree is realized by modifying the pixel value of each field of view in the fourth channel. After the tilt direction, the tilt action radius, the blur action radius and the blur degree of the turbulence degradation condition map T in the test set are completed, a new turbulence degradation condition map T new is obtained.

[0088] Step 6, reverse denoising.

[0089] A new random Gaussian noise ε1 with the same size as the turbulence degradation image D is defined as the initial input of the entire reverse denoising process. The new random Gaussian noise ε1, the clear image V in the test set, and the new turbulence degradation condition image T new After channel splicing, the sample time step t_batch stored in the reverse denoising unit is sent to the trained noise prediction network, and the trained noise prediction network outputs the predicted noise .

[0090] The denoised image at time step t is obtained by reverse derivation calculation using the following formula :

[0091]

[0092] wherein, is the noise scheduling parameter at time step t;

[0093] The turbulence degradation image at time step t-1 is calculated using the following formula :

[0094]

[0095] wherein, is the noise scheduling parameter at time step t-1.

[0096] The turbulence degradation images at all time steps are iteratively calculated using the above method. After the iterative calculation is completed, i.e., when the sample time step t_batch is 1 and the sample time step prev_t_batch is 0, the final output of the reverse denoising process is obtained, i.e., the new turbulence degradation condition image T new The turbulence degradation image D new at time step t is obtained, and the end-to-end turbulence suppression data set augmentation based on the diffusion model is completed.

[0097] The above description is only to illustrate the technical solutions of the present application, and not to limit them. For ordinary skilled persons in the art, the specific technical solutions described in the above embodiments can be modified, or some technical features can be replaced by equivalents, and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the scope of the technical solutions protected by the present application.

Claims

1. An end-to-end turbulence suppression dataset augmentation method based on a diffusion model, characterized in that, The method comprises the following steps: Step 1, constructing a training set and a test set; the data in the training set comprises corresponding turbulent degradation images D, clear images V and turbulent degradation condition images T, and the data in the test set comprises corresponding clear images V and turbulent degradation condition images T; Step 2, building a diffusion model, wherein the diffusion model comprises a forward noise adding unit, a noise prediction network and a reverse noise removing unit connected in sequence according to input and output; Step 3, generate random Gaussian noise ε with the same size as the turbulence-degraded image D, add the random Gaussian noise ε to the turbulence-degraded image D through the forward noise adding unit to obtain a noisy image D at different time steps i ; Step 4, noisy images D at different time steps i respectively, and the clear images V and the turbulent degradation condition images T in the training set are spliced by channel, and after splicing, they are sent into the noise prediction network together with the corresponding time step for iterative training to obtain a trained noise prediction network; when the noise prediction network is iteratively trained, the noise prediction network is optimized through a noise prediction loss function, a structure loss function and an edge loss function; The noise prediction loss function is a Charbonnier loss function; the structure loss function is a multi-scale structure function; and the edge loss function is a total variation (TV) function; Step 5, modifying the tilt direction, tilt action radius, blur action radius and blur degree of the turbulence degradation condition graph T in the test set to obtain a new turbulence degradation condition graph T new ; meanwhile, defining a new random Gaussian noise ε1 with the same size as the turbulence degradation image D, and inputting the new random Gaussian noise ε1, the clear image V in the test set and the new turbulence degradation condition graph T new splicing by channel, and then inputting into the trained noise prediction network to make the trained noise prediction network output the predicted noise The modification of the tilt direction, the tilt action radius, the blur action radius and the blur degree of the turbulent degradation condition image T in the test set is specifically as follows: Any real number in the range of 0-2 is used as the corresponding influence factor of each field of view of each channel of the turbulent degradation condition image T in the test set, and different influence factors of each field of view in each channel are multiplied with the turbulent degradation condition image T pixel by pixel to modify the turbulent degradation condition image T; wherein the tilt direction is modified through the first channel of the turbulent degradation condition image T, the tilt action radius is modified through the second channel, the blur action radius is modified through the third channel, and the blur degree is modified through the fourth channel. Step 6, reverse denoising of the predicted noise obtained in step 5 by the reverse denoising unit Reverse denoising is performed to obtain a new turbulence degradation condition map T new Turbulence degraded image D new , complete the end-to-end turbulence suppression data set augmentation based on the diffusion model; the reverse denoising unit adopts the DDIM sampling mode for reverse denoising.

2. The end-to-end turbulent suppression data set augmentation method based on the diffusion model according to claim 1, wherein: In step 2, an array is arranged in the forward noise adding unit, which is used to store noise scheduling parameters at different time steps.

3. The end-to-end turbulent suppression data set augmentation method based on the diffusion model according to claim 2, wherein: In step 2, the noise scheduling parameters at different time steps are obtained through the following steps: Step a1: define a sequence η consisting of t numbers, η=[η1,η2,…,η t ]; Step bl, calculate the scheduling sequence a according to the following formula i : a i = 1 - η i , i = 1, 2,..., t; Step c1, calculate the scheduling sequence a according to the scheduling sequence a i The noise scheduling parameters of different time steps are calculated respectively, and the noise scheduling parameters of different time steps are defined and initialized according to the specified rules and stored in the array.

4. The end-to-end turbulent suppression data set augmentation method based on the diffusion model according to claim 3, wherein: In step 3, the random Gaussian noise ε is added to the turbulence degradation image D by the forward noise adding unit to obtain the noisy image D at different time steps i Specifically: By adding random Gaussian noise ε to the turbulence degradation image D through the noise scheduling parameters of different time steps stored in the forward noise adding unit, the noisy image D at different time steps is obtained i The specific expression is as follows: wherein is the noise scheduling parameter at time step i.

5. The end-to-end turbulent suppression data set augmentation method based on the diffusion model according to claim 4, wherein: In step 4, the noise prediction network is optimized by the noise prediction loss function, the structure loss function and the edge loss function. Step a2, the output ε of the noise prediction network is iteratively trained using a noise prediction loss function θ Minimizes the distance between the random Gaussian noise ε Step b2, computing clean turbulence-degraded images at different time steps Step c2, the computed clean turbulence-degraded images at different time steps are optimized with a structural loss function minimizing the distance to the turbulence-degraded images D; Step d2, using an edge loss function, the computed clean turbulence-degraded images at different time steps are refined The distance between the turbulence-degraded image D and the clean image C is minimized.

6. The end-to-end turbulent suppression data set augmentation method based on the diffusion model according to claim 5, wherein: In step 6, the predicted noise obtained in step 5 is de-noised by the reverse de-noising unit The reverse de-noising is performed to obtain a new turbulence degradation condition map T new The turbulence degradation image D under the new turbulence degradation condition map T new Specifically: Step a3, the denoised image at time step t is calculated by wherein, is a noise scheduling parameter at time step t; Step b3, the turbulence-degraded image D at time step t-1 is calculated by t-1 : wherein is the noise scheduling parameter at time step t-1; Step c3, iteratively calculate the turbulence-degraded image at all time steps using the method of step b, after the iteration is completed, obtain the new turbulence-degraded condition image T new under the turbulence-degraded image D new .

7. The end-to-end turbulent suppression data set augmentation method based on the diffusion model according to claim 1, wherein: In step 1, the turbulent degradation condition image T is obtained by the following method: The corresponding clear image V and the turbulent degradation image D are subtracted pixel by pixel, and then the absolute value is taken to obtain the turbulent degradation condition image T.

Images