Atmospheric turbulence simulation method and atmospheric turbulence simulation device

Abstract

The embodiment of the invention provides an atmospheric turbulence simulation method and an atmospheric turbulence simulation device, and relates to the technical field of atmospheric optics. According to the atmospheric turbulence simulation method, dynamic adjustment of amplitude modulation and phase modulation is performed through the first implicit neural network and the second implicit neural network respectively, so that the influence of atmospheric turbulence on light waves can be simulated more accurately, and the accuracy of a simulation result is improved; and the first correlation coefficient and the second correlation coefficient can be dynamically adjusted according to different simulation turbulence parameters, so that the method can be better generalized to different turbulence conditions and unknown environments.

Description

Technical Field

[0001] The present application relates to the field of atmospheric optics technology, and in particular to an atmospheric turbulence simulation method and an atmospheric turbulence simulation device. Background Art

[0002] During free space transmission, it is easily affected by atmospheric turbulence, which causes turbulence effects such as random drift, expansion, distortion, and flickering of the transmitted light waves. Therefore, atmospheric turbulence is the main factor affecting the performance of long-distance imaging systems such as astronomical observation, monitoring, and navigation. For decades, people have been studying methods to reduce atmospheric turbulence, and objective evaluation of these algorithms requires a large number of image pairs with and without turbulence, which are difficult to obtain for real long-distance imaging. Therefore, atmospheric turbulence simulation methods are needed to provide sufficient sample sets. How to efficiently generate scene-rich image pairs with and without turbulence and improve the image database for atmospheric turbulence distortion simulation has become a problem that people in this field need to solve urgently. Summary of the invention

[0003] In order to at least overcome the above-mentioned deficiencies in the prior art, the purpose of the present application is to provide an atmospheric turbulence simulation method and an atmospheric turbulence simulation device.

[0004] In a first aspect, an embodiment of the present application provides an atmospheric turbulence simulation method, the atmospheric turbulence simulation method comprising:

[0005] Inputting the original turbulence-free image to be processed and the simulated turbulence parameters into the trained atmospheric turbulence simulation model, and obtaining the corresponding first object-space amplitude and first object-space phase based on the original turbulence-free image, wherein the atmospheric turbulence simulation model includes a first implicit neural network and a second implicit neural network;

[0006] Initializing the first correlation coefficient and the second correlation coefficient based on the simulated turbulence parameter;

[0007] modulating the first object-side amplitude based on the first correlation coefficient and the first implicit neural network to obtain a modulated first object-side amplitude;

[0008] modulating the first object-side phase based on the second correlation coefficient and the second implicit neural network to obtain a modulated first object-side phase;

[0009] A turbulence simulation image corresponding to the simulated turbulence parameter is obtained based on the modulated first object-space amplitude and the modulated first object-space phase.

[0010] In a possible implementation, the simulated turbulence parameter includes a relative intensity of atmospheric turbulence, and the step of initializing the first correlation coefficient and the second correlation coefficient based on the simulated turbulence parameter includes:

[0011] The first correlation coefficient and the second correlation coefficient are initialized based on the relative intensity of the atmospheric turbulence, wherein the process of initializing the first correlation coefficient and the second correlation coefficient is as follows:

[0012]

[0013] where D / r0 represents the relative intensity of atmospheric turbulence, randn(x,y,36) represents the coefficient matrix of random normal distribution, Z(r,θ) represents the 36th-order Zernike polynomial, ⊙ represents the Hadamard product, R a (x, y) represents the initialization result of the first correlation coefficient or the second correlation coefficient.

[0014] In a possible implementation, the step of obtaining a turbulence simulation image corresponding to the simulated turbulence parameter based on the modulated first object-space amplitude and the modulated first object-space phase includes:

[0015] synthesizing a corresponding complex amplitude based on the modulated first object-side amplitude and the modulated first object-side phase;

[0016] Performing Fourier transform on the complex amplitude to obtain a turbulence simulation image corresponding to the simulated turbulence parameter, wherein the process of Fourier transform is as follows:

[0017]

[0018] in, represents the first object-side amplitude after the modulation, represents the first object phase after modulation, S i represents the turbulence simulation image, represents the inverse Fourier transform, represents Fourier transform, P(u,v) represents pupil function, ⊙ represents Hadamard product, |*| 2 It means taking the square of the modulus length.

[0019] In a possible implementation, before the step of inputting the original turbulence-free image to be processed and the simulated turbulence parameters into the trained atmospheric turbulence simulation model, the method further includes the step of training the atmospheric turbulence simulation model, which step includes:

[0020] Acquire a training image pair, wherein the training image pair includes a training image with turbulence and a training image without turbulence of the same scene;

[0021] Acquire training turbulence parameters corresponding to the trained turbulence image, input the turbulence environment parameters corresponding to the trained turbulence image and the training turbulence-free image into the atmospheric turbulence simulation model to be trained, and obtain the corresponding second object-space amplitude and second object-space phase based on the training turbulence-free image, wherein the atmospheric turbulence simulation model to be trained includes a first implicit neural network to be trained and a second implicit neural network to be trained;

[0022] Initializing a first correlation coefficient and a second correlation coefficient based on the training turbulence parameter;

[0023] Modulating the second object-side amplitude based on the first correlation coefficient and the first implicit neural network to be trained, calculating a first loss function value based on the modulated second object-side amplitude and the trained turbulence image, and iteratively updating parameters in the first implicit neural network to be trained based on the first loss function value;

[0024] The second object-side phase is modulated based on the second correlation coefficient and the second implicit neural network to be trained, and the second loss function value is calculated based on the modulated first object-side phase and the trained turbulence image. The parameters in the second implicit neural network to be trained are iteratively updated based on the second loss function value. At the end of the iteration, the atmospheric turbulence simulation model after the updated parameters is used as the trained atmospheric turbulence simulation model.

[0025] In a possible implementation, before the step of acquiring the training image pair, the method further includes:

[0026] Acquire collected images of different scenes in different turbulent environments based on an active imaging system, and record the turbulent environment parameters corresponding to each of the collected images;

[0027] A region of interest is extracted from the collected images, and the collected images are paired based on the extracted region of interest, and an image with turbulence and an image without turbulence of the same scene are used as a training image pair.

[0028] In a possible implementation, the step of modulating the second object-space amplitude based on the first correlation coefficient and the first implicit neural network to be trained, calculating a first loss function value based on the modulated second object-space amplitude and the trained turbulence image, and iteratively updating parameters in the first implicit neural network to be trained based on the first loss function value includes:

[0029] Inputting the first correlation coefficient into the first implicit neural network to be trained to obtain a first dynamic result of training;

[0030] extracting high-frequency information and low-frequency information corresponding to the original object space based on the second object space amplitude, multiplying the first dynamic result by the high-frequency information, adding the multiplication result by the low-frequency information, and integrating the spatial amplitude information of the addition result to obtain a modulated first training turbulence simulation image;

[0031] A first loss function value is calculated based on the modulated first training turbulence simulation image and the trained turbulence image, and parameters in the first implicit neural network to be trained are iteratively updated based on the first loss function value. After the iteration is completed, the iteratively updated first implicit neural network is used as the trained first implicit neural network.

[0032] In a possible implementation, the step of modulating the second object-side phase based on the second correlation coefficient and the second implicit neural network to be trained, calculating a second loss function value based on the modulated first object-side phase and the trained turbulence image, iteratively updating parameters in the second implicit neural network to be trained based on the second loss function value, and at the end of the iteration, using the atmospheric turbulence simulation model after the updated parameters as the trained atmospheric turbulence simulation model includes:

[0033] Inputting the second correlation coefficient into the second implicit neural network to be trained to obtain a second dynamic result of training;

[0034] Adding the second dynamic result and the second object-space phase, and integrating the spatial phase information of the addition result to obtain a modulated second training turbulence simulation image;

[0035] A second loss function value is calculated based on the modulated second training turbulence simulation image and the trained turbulence image, and the parameters in the second implicit neural network to be trained are iteratively updated based on the second loss function value. After the iteration, the atmospheric turbulence simulation model with updated parameters is used as the trained atmospheric turbulence simulation model.

[0036] In a possible implementation, the first loss function or the second loss function is as follows:

[0037]

[0038] Wherein, Loss represents the overall loss function value of the first implicit neural network to be trained or the second implicit neural network to be trained, S i represents the first training turbulence simulation graph or the second training turbulence simulation graph, R i represents the training turbulence map, and β represents the selection threshold of the first-order norm loss function and the second-order norm loss function.

[0039] In a second aspect, an embodiment of the present application further provides an atmospheric turbulence simulation device, the device comprising:

[0040] An input module, used for inputting the original turbulence-free image to be processed and the simulated turbulence parameters into the trained atmospheric turbulence simulation model, and obtaining the corresponding first object-space amplitude and first object-space phase based on the original turbulence-free image, wherein the atmospheric turbulence simulation model includes a first implicit neural network and a second implicit neural network;

[0041] An initialization module, used for initializing the first correlation coefficient and the second correlation coefficient based on the simulated turbulence parameter;

[0042] A first modulation module, configured to modulate the first object-side amplitude based on the first correlation coefficient and the first implicit neural network to obtain a modulated first object-side amplitude;

[0043] a second modulation module, configured to modulate the first object-side phase based on the second correlation coefficient and the second implicit neural network to obtain a modulated first object-side phase;

[0044] A synthesis module is used to obtain a turbulence simulation image corresponding to the simulated turbulence parameter based on the modulated first object-space amplitude and the modulated first object-space phase.

[0045] In a third aspect, an embodiment of the present application further provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the atmospheric turbulence simulation method as described in any one of the first aspects.

[0046] Based on any one of the above aspects, the atmospheric turbulence simulation method and the atmospheric turbulence simulation device provided in the embodiments of the present application can not only more accurately simulate the influence of atmospheric turbulence on light waves and improve the accuracy of simulation results, but also dynamically adjust the first correlation coefficient and the second correlation coefficient according to different simulated turbulence parameters, so as to better generalize to different turbulence conditions and unknown environments, by dynamically adjusting the amplitude modulation and the phase modulation of the first implicit neural network and the second implicit neural network respectively. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required in the embodiments will be briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present application and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without creative work.

[0048] Figure 1 A schematic diagram of a flow chart of an atmospheric turbulence simulation method provided in an embodiment of the present application;

[0049] Figure 2 Another schematic diagram of the flow chart of the atmospheric turbulence simulation method provided in the embodiment of the present application;

[0050] Figure 3 for Figure 2 Schematic diagram of the sub-step flow chart of step S140;

[0051] Figure 4 for Figure 2 Schematic diagram of the sub-step flow chart of step S150;

[0052] Figure 5 A functional block diagram of an atmospheric turbulence simulation device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0053] In order to make the purpose, technical solution and advantages of the embodiments of the present application clearer, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, rather than all the embodiments. The components of the embodiments of the present application described and shown in the drawings here can be arranged and designed in various different configurations.

[0054] Therefore, the following detailed description of the embodiments of the present application provided in the accompanying drawings is not intended to limit the scope of the present application for which protection is sought, but merely represents selected embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in the field without creative work are within the scope of protection of the present application.

[0055] It should be noted that similar reference numerals and letters denote similar items in the following drawings, and therefore, once an item is defined in one drawing, further definition and explanation thereof is not required in subsequent drawings.

[0056] In the description of this application, it should be noted that the terms "upper", "lower", etc. indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings, or the orientation or positional relationship in which the product of the application is usually placed when in use, which is only for the convenience of describing this application and simplifying the description, and does not indicate or imply that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as a limitation on this application. In addition, the terms "first", "second", etc. are only used to distinguish the description, and cannot be understood as indicating or implying relative importance.

[0057] In the description of this application, it should also be noted that, unless otherwise clearly specified and limited, the terms "disposed", "connected", "connected", and "connected" should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection, or it can be indirectly connected through an intermediate medium, or it can be the internal connection of two elements. For ordinary technicians in this field, the specific meanings of the above terms in this application can be understood according to specific circumstances.

[0058] It should be noted that, in the absence of conflict, different features in the embodiments of the present application may be combined with each other.

[0059] The inventors found that the relevant technologies usually use the phase screen method or the data-driven neural network method to simulate atmospheric turbulence. The statistical phase screen method specifically refers to a numerical simulation method based on the Kolmogorov turbulence theory. However, this method often needs to include many meteorological parameters. When the research conditions lack sufficient equipment to measure turbulence information, it will be difficult to perform optical turbulence parametric simulation analysis. Although the data-driven neural network method can solve key turbulence problems such as the measurement and intensity estimation of non-Kolmogorov turbulence, and the turbulence prediction results are better than the traditional phase screen-based method, this method lacks sufficient physical explanation and has the problem of insufficient generalization.

[0060] To solve the problems in the prior art, please refer to Figure 1 , this application embodiment provides an atmospheric turbulence simulation method, please refer to Figure 1 , Figure 1 This is a flow chart of the atmospheric turbulence simulation method. Figure 1 Each step of the atmospheric turbulence simulation method is described in detail.

[0061] Step S210: inputting the original turbulence-free image to be processed and the simulated turbulence parameters into the trained atmospheric turbulence simulation model, and obtaining the corresponding first object-space amplitude and first object-space phase based on the original turbulence-free image.

[0062] In this step, the original turbulence-free image can be expressed as I i (x,y), its original object square can be expressed as Among them, the first object-side amplitude can be expressed as The first object phase is φ(x,y). The simulated turbulence parameters may include temperature range, wind speed range, etc. Temperature and wind speed may affect the atmospheric coherence length r0. According to the atmospheric coherence length r0 and the imaging aperture size D, the relative intensity D / r0 of the atmospheric turbulence may be obtained.

[0063] The atmospheric turbulence simulation model includes the first implicit neural network and the second implicit neural network. The output of the first implicit neural network can adjust the multi-order combination coefficients of the original high-frequency information of the object to simulate the changes in high-frequency information caused by atmospheric turbulence. The second implicit neural network can modulate the phase information of the original turbulence-free image, change the amplitude, phase and polarization state of the light field, and thus better simulate the influence of atmospheric turbulence on the phase of the light wave.

[0064] Step S220: Initializing the first correlation coefficient and the second correlation coefficient based on the simulated turbulence parameters.

[0065] In this step, the first correlation coefficient and the second correlation coefficient may be initialized based on the relative intensity of the atmospheric turbulence, wherein the initialization process of the first correlation coefficient and the second correlation coefficient is as follows:

[0066]

[0067] Where D / r0 represents the relative intensity of atmospheric turbulence, randn(x,y,36) represents the coefficient matrix of random normal distribution, Z(r,θ) represents the 36th-order Zernike polynomial, ⊙ represents the Hadamard product, R a (x, y) represents the initialization result of the first correlation coefficient or the second correlation coefficient, where the relative intensity of atmospheric turbulence D / r0 is related to the atmospheric coherence length r0 and the imaging aperture size D, and the atmospheric coherence length r0 is related to the temperature range and wind speed.

[0068] Step S230: modulating the first object-side amplitude based on the first correlation coefficient and the first implicit neural network to obtain a modulated first object-side amplitude.

[0069] In this step, since atmospheric turbulence mainly affects the distribution of high-frequency information on the object side, the original low-frequency information on the object side can be retained. By dynamically adjusting the laser wavelength, propagation distance, atmospheric coherence length, and Zernike distortion intensity, the degree of degradation of multi-order high-frequency information is changed, thereby simulating the real turbulence effect.

[0070] Specifically, the first correlation coefficient can be input into the first implicit neural network to obtain the first dynamic result. Then, the high-pass filter kernel can be used to extract the high-frequency information of various forms of the original object space corresponding to the original turbulence-free image, and the high-frequency information corresponding to the original turbulence-free image and the first dynamic result can be multiplied respectively, and then added to the object space low-frequency information obtained by the low-pass filter kernel, and finally the spatial amplitude information of the addition result is integrated through the flow field grid sampler to obtain the modulated first object space amplitude. Among them, the modulation process of the first object space amplitude can be expressed as:

[0071]

[0072] in, represents the first object amplitude after modulation, represents the sampling function, represents the low-pass filter kernel, represents the mth high-pass filter kernel,

[0073] ⊙ represents Hadamard product, MLP1 represents the first implicit neural network, wherein the number of high-pass filter kernels can be selected according to the specific situation. Exemplarily, the number of high-pass filter kernels can be 100, that is, M=100. represents the flow field grid, R t (x,y) represents the coefficient matrix related to tilt distortion, which is defined as follows:

[0074]

[0075] Where λ represents the laser wavelength, d represents the imaging distance, D / r0 represents the relative intensity of atmospheric turbulence, randn(x,y,2) represents the grid matrix of random normal distribution, B(ξ,N) represents the Bessel integral correlation matrix, and ⊙ represents the Hadamard product.

[0076] Step S240: modulating the first object-side phase based on the second correlation coefficient and the second implicit neural network to obtain a modulated first object-side phase.

[0077] In this step, the second correlation coefficient can be first input into the second implicit neural network to obtain the second dynamic result. Then, the flow field grid sampler integrates the spatial phase information of the addition result of the second dynamic result and the first object phase to obtain the modulated first object phase, that is, when modulating the first object phase, a Zernike phase with a certain distortion intensity can be superimposed on the first object phase, wherein the Zernike order can be adjusted according to the actual turbulence conditions.

[0078] Specifically, the modulation process of the first object-side phase can be expressed as:

[0079]

[0080] in, represents the first object phase after modulation, represents the sampling function, represents the flow field grid, MLP2 represents the second implicit neural network, Represents element-wise addition. t (x, y) represents a coefficient matrix related to tilt distortion, and the implementation method is the same as that shown in step S230.

[0081] Step S250: obtaining a turbulence simulation image corresponding to the simulated turbulence parameters based on the modulated first object-space amplitude and the modulated first object-space phase.

[0082] In this embodiment, the corresponding complex amplitude can be synthesized based on the modulated first object-side amplitude and the modulated first object-side phase, and then the complex amplitude is Fourier transformed through the pupil plane to obtain a turbulence simulation image corresponding to the simulated turbulence parameters, wherein the Fourier transform process is as follows:

[0083]

[0084] Among them, S i represents the turbulence simulation image, represents the inverse Fourier transform, represents Fourier transform, P(u,v) represents pupil function, ⊙ represents Hadamard product, |*| 2 It means taking the square of the modulus length.

[0085] In this embodiment, by dynamically adjusting the amplitude modulation and the phase modulation respectively by the first implicit neural network and the second implicit neural network, not only can the influence of atmospheric turbulence on light waves be simulated more accurately and the accuracy of the simulation results be improved, but the first correlation coefficient and the second correlation coefficient can also be dynamically adjusted according to different simulated turbulence parameters, so as to better generalize to different turbulence conditions and unknown environments.

[0086] In some possible embodiments, before step S210, the atmospheric turbulence simulation method provided by the present application further includes the step of training an atmospheric turbulence simulation model. For details, please refer to Figure 2 , the following combination Figure 2 A detailed description of the steps for training an atmospheric turbulence simulation model is given.

[0087] Step S110: Acquire training image pairs.

[0088] In this step, the training image pair includes a training image with turbulence and a training image without turbulence of the same scene. Specifically, the acquired images of different scenes under different turbulent environments can be first acquired based on the active imaging system, and the turbulent environment parameters corresponding to each acquired image can be recorded, wherein the turbulent environment parameters can include temperature range and wind speed range, etc., and the temperature and wind speed can affect the atmospheric coherence length r0. According to the atmospheric coherence length r0 and the imaging aperture size D, the relative intensity D / r0 of the atmospheric turbulence can be obtained. Then, the region of interest can be extracted from each acquired image, and the acquired images can be paired based on the extracted region of interest, and the image with turbulence and the image without turbulence of the same scene can be used as a training image pair.

[0089] Step S120: inputting the turbulence environment parameters corresponding to the trained turbulence image and the trained turbulence-free image into the atmospheric turbulence simulation model to be trained, and obtaining the corresponding second object-space amplitude and second object-space phase based on the trained turbulence-free image.

[0090] In this step, the atmospheric turbulence simulation model to be trained includes a first implicit neural network to be trained and a second implicit neural network to be trained. Exemplarily, the first implicit neural network to be trained may include a multilayer perceptron of 3 modules, the hidden layer dimension of each module may be 100, and ReLU is used as the activation layer. The second implicit neural network to be trained may include a multilayer perceptron of 8 modules, the hidden layer dimension of each module may be 16, and ReLU is used as the activation layer.

[0091] Step S130: Initializing the first correlation coefficient and the second correlation coefficient based on the training turbulence parameters.

[0092] In this step, the training turbulence parameters include the relative intensity of atmospheric turbulence, and the first correlation coefficient and the second correlation coefficient can be initialized based on the relative intensity of atmospheric turbulence, where the initialization is a random normal distribution in the range of [0,2π]. Specifically, the initialization process of the first correlation coefficient and the second correlation coefficient is as follows:

[0093]

[0094] Where D / r0 represents the relative intensity of atmospheric turbulence, randn(x,y,36) represents the coefficient matrix of random normal distribution, Z(r,θ) represents the 36th-order Zernike polynomial, ⊙ represents the Hadamard product, R a (x, y) represents the initialization result of the first correlation coefficient or the second correlation coefficient.

[0095] Step S140: modulate the second object-space amplitude based on the first correlation coefficient and the first implicit neural network to be trained, calculate the first loss function value based on the modulated second object-space amplitude and the trained turbulence image, and iteratively update the parameters in the first implicit neural network to be trained based on the first loss function value.

[0096] In this step, when the first implicit neural network is iteratively updated, the second object-side phase can be randomly initialized, and the phase modulation of the second implicit neural network can be temporarily ignored, so that the first implicit neural network to be trained can focus on amplitude modulation. After the first implicit neural network to be trained completes the second object-side amplitude modulation, the first loss function value is calculated based on the image generated by the modulated second object-side amplitude and the second object-side phase and the turbulence image, and the parameters in the first implicit neural network to be trained are iteratively updated based on the first loss function value until the first loss function value is less than the preset loss function value.

[0097] Step S150: modulate the second object-side phase based on the second correlation coefficient and the second implicit neural network to be trained, calculate the second loss function value based on the modulated first object-side phase and the trained turbulence image, iteratively update the parameters in the second implicit neural network to be trained based on the second loss function value, and at the end of the iteration, use the atmospheric turbulence simulation model after the updated parameters as the trained atmospheric turbulence simulation model.

[0098] In this step, when modulating the second object-side phase, a Zernike phase with a certain distortion intensity is superimposed on the second object-side phase, wherein the Zernike order can be adjusted according to the actual turbulence conditions. After the second implicit neural network to be trained finishes modulating the second object-side phase, the second loss function value is calculated based on the image generated by the modulated second object-side phase and the second object-side amplitude and the turbulence image, and the parameters in the second implicit neural network to be trained are iteratively updated based on the second loss function value until the second loss function value is less than the preset loss function value. After fixing the parameters of the first implicit neural network, the second implicit neural network to be trained is trained so that the second implicit neural network to be trained can focus on phase modulation.

[0099] In this embodiment, the first implicit neural network and the second implicit neural network are trained independently, which can not only reduce the complexity of network training, but also make the first implicit neural network and the second implicit neural network focus on the tasks of amplitude modulation and phase modulation respectively, avoiding the situation that the general neural network parameters are fixed after training, and the generalization ability is insufficient due to the relatively fixed function mapping relationship. In addition, by training the two neural networks independently, the corresponding network can be adjusted or optimized separately for amplitude modulation or phase modulation without affecting the other network.

[0100] Further, step S140 can be implemented by the following method.

[0101] Step S141: inputting the first correlation coefficient into the first implicit neural network to be trained to obtain a first dynamic result of the training.

[0102] Step S142: extracting high-frequency information and low-frequency information corresponding to the original object space based on the second object space amplitude, multiplying the first dynamic result by the high-frequency information, and adding the multiplication result to the low-frequency information, integrating the spatial amplitude information of the addition result, and obtaining a modulated first training turbulence simulation image.

[0103] In this step, the high-pass filter kernel can be used to extract the high-frequency information of various forms of the original object space corresponding to the training turbulence-free image, and then the corresponding high-frequency information and the first dynamic result are multiplied, and then the object space low-frequency information obtained by the low-pass filter is added, and then the spatial amplitude information of the addition result is integrated through the flow field grid sampler to obtain the modulated second object space amplitude. Finally, the complex amplitude synthesized by the modulated second object space amplitude and the second object space phase is Fourier transformed to obtain the modulated first training turbulence simulation image.

[0104] Step S143: Calculate a first loss function value based on the modulated first training turbulence simulation image and the trained turbulence image, iteratively update the parameters in the first implicit neural network to be trained based on the first loss function value, and after the iteration, use the iteratively updated first implicit neural network as the trained first implicit neural network.

[0105] In this step, when the parameters in the first implicit neural network to be trained are iteratively updated based on the first loss function, the iteration can be stopped when the first loss function is less than the preset loss function domain or the number of iterations reaches the preset number, and the first implicit neural network parameters are fixed. The first loss function can be expressed as:

[0106]

[0107] Among them, Loss represents the overall loss function value of the first implicit neural network to be trained, S i represents the first training turbulence simulation graph, R i It indicates that the turbulence map is trained, and β indicates the selection threshold of the first-order norm loss function and the second-order norm loss function.

[0108] In this embodiment, the first implicit neural network is trained alone, and the phase modulation of the second implicit neural network is temporarily ignored. This reduces the difficulty of model training and enables the first implicit neural network to be trained to better learn the relationship between turbulence parameters and amplitude, thereby improving the accuracy and generalization ability of amplitude modulation.

[0109] Furthermore, step S150 can be implemented by the following method.

[0110] Step S151: inputting the second correlation coefficient into the second implicit neural network to be trained to obtain a second dynamic result of the training.

[0111] Step S152: Add the second dynamic result and the second object-space phase, and integrate the spatial phase information of the addition result to obtain a modulated second training turbulence simulation image.

[0112] In this step, the flow field grid sampler can integrate the spatial phase information of the addition result of the second dynamic result and the second object phase to obtain the modulated second object phase. Finally, the complex amplitude synthesized by the second object amplitude and the modulated second object phase is Fourier transformed to obtain the modulated second training turbulence simulation image.

[0113] Step S153: Calculate the second loss function value based on the modulated second training turbulence simulation image and the trained turbulence image, iteratively update the parameters in the second implicit neural network to be trained based on the second loss function value, and after the iteration, use the atmospheric turbulence simulation model with updated parameters as the trained atmospheric turbulence simulation model.

[0114] In this step, when the parameters in the second implicit neural network to be trained are iteratively updated based on the second loss function value, the iteration can be stopped when the second loss function value is less than the preset loss function threshold value or the number of iterations reaches the preset number of times. After the iteration is completed, the atmospheric turbulence simulation model with updated parameters is used as the trained atmospheric turbulence simulation model. The second loss function can be expressed as:

[0115]

[0116] Among them, Loss represents the overall loss function value of the second implicit neural network to be trained, S i Represents the second training turbulence simulation graph, R i It indicates that the turbulence map is trained, and β indicates the selection threshold of the first-order norm loss function and the second-order norm loss function.

[0117] In this embodiment, the second implicit neural network is trained separately, which reduces the difficulty of model training and enables the second implicit neural network to be trained to better learn the relationship between turbulence parameters and phase, thereby improving the accuracy and generalization ability of phase modulation.

[0118] It should be noted that this application can use deep learning network frameworks such as PyTorch, TensorFlow, Keras, and MXNet for training, and use the optimizer of the adaptive learning rate optimization algorithm for iterative updates, gradually reducing the loss function value until the network converges. Exemplarily, the optimizer includes but is not limited to the Adam (Adaptive Moment Estimation) optimizer. The learning rate is dynamically adjusted using the cosine annealing method, and the range of the learning rate can be (10 -6 ,10 -4 ), the number of warm-up iterations is 100.

[0119] Based on the same inventive concept, the present application also provides an atmospheric turbulence simulation device, please refer to FIG. *, FIG. * is a functional module schematic diagram of the atmospheric turbulence simulation device 200 provided in the embodiment of the present application. The embodiment of the present application can divide the functional modules of the atmospheric turbulence simulation device 200 according to the method embodiment executed by the computer device, that is, the following functional modules corresponding to the atmospheric turbulence simulation device 200 can be used to execute the above-mentioned various method embodiments. Among them, the atmospheric turbulence simulation device 200 may include an input module 210, an initialization module 220, a first modulation module 230, a second modulation module 240 and a synthesis module 250. The functions of each functional module of the atmospheric turbulence simulation device 200 are described in detail below.

[0120] The input module 210 is used to input the original turbulence-free image to be processed and the simulated turbulence parameters into the trained atmospheric turbulence simulation model, and obtain the corresponding first object space amplitude and first object space phase based on the original turbulence-free image.

[0121] Specifically, the original turbulence-free image can be expressed as I i (x,y), its original object square can be expressed as Among them, the first object-side amplitude can be expressed as The first object phase is φ(x,y). The simulated turbulence parameters may include temperature range, wind speed range, etc. Temperature and wind speed may affect the atmospheric coherence length r0. According to the atmospheric coherence length r0 and the imaging aperture size D, the relative intensity D / r0 of the atmospheric turbulence may be obtained.

[0122] The atmospheric turbulence simulation model includes the first implicit neural network and the second implicit neural network. The output of the first implicit neural network can adjust the multi-order combination coefficients of the original high-frequency information of the object to simulate the changes in high-frequency information caused by atmospheric turbulence. The second implicit neural network can modulate the phase information of the original turbulence-free image, change the amplitude, phase and polarization state of the light field, and thus better simulate the influence of atmospheric turbulence on the phase of the light wave.

[0123] In this embodiment, the receiving module can be used to execute the above step S210. The detailed implementation of the input module 210 can refer to the above detailed description of step S210.

[0124] The initialization module 220 is used to initialize the first correlation coefficient and the second correlation coefficient based on the simulated turbulence parameters.

[0125] Specifically, the first correlation coefficient and the second correlation coefficient may be initialized based on the relative intensity of the atmospheric turbulence, wherein the initialization process of the first correlation coefficient and the second correlation coefficient is as follows:

[0126]

[0127] Where D / r0 represents the relative intensity of atmospheric turbulence, randn(x,y,36) represents the coefficient matrix of random normal distribution, Z(r,θ) represents the 36th-order Zernike polynomial, ⊙ represents the Hadamard product, R a (x, y) represents the initialization result of the first correlation coefficient or the second correlation coefficient, where the relative intensity of atmospheric turbulence D / r0 is related to the atmospheric coherence length r0 and the imaging aperture size D, and the atmospheric coherence length r0 is related to the temperature range and wind speed.

[0128] In this embodiment, the initialization module 220 may be used to execute the above step S220. For the detailed implementation of the initialization module 220, reference may be made to the above detailed description of step S220.

[0129] The first modulation module 230 is used to modulate the first object-side amplitude based on the first correlation coefficient and the first implicit neural network to obtain a modulated first object-side amplitude.

[0130] Since atmospheric turbulence mainly affects the distribution of high-frequency information on the object side, the original low-frequency information on the object side is retained. By dynamically adjusting the laser wavelength, propagation distance, atmospheric coherence length, and Zernike distortion intensity, the degradation degree of multi-order high-frequency information is changed to simulate the real turbulence effect.

[0131] Specifically, the first correlation coefficient can be input into the first implicit neural network to obtain the first dynamic result. Then, the high-pass filter kernel can be used to extract the high-frequency information of various forms of the original object space corresponding to the original turbulence-free image, and the high-frequency information corresponding to the original turbulence-free image and the first dynamic result can be multiplied respectively, and then added to the object space low-frequency information obtained by the low-pass filter kernel, and finally the spatial amplitude information of the addition result is integrated by the flow field grid sampler to obtain the modulated first object space amplitude.

[0132] Specifically, the modulation process of the first object-side amplitude can be expressed as:

[0133]

[0134] in, represents the first object amplitude after modulation, represents the sampling function, represents the low-pass filter kernel, represents the mth high-pass filter kernel,

[0135] ⊙ represents Hadamard product, MLP1 represents the first implicit neural network, wherein the number of high-pass filter kernels can be selected according to the specific situation. Exemplarily, the number of high-pass filter kernels can be 100, that is, M=100. represents the flow field grid, R t (x,y) represents the coefficient matrix related to tilt distortion, which is defined as follows:

[0136]

[0137] Where λ represents the laser wavelength, d represents the imaging distance, D / r0 represents the relative intensity of atmospheric turbulence, randn(x,y,2) represents the grid matrix of random normal distribution, B(ξ,N) represents the Bessel integral correlation matrix, and ⊙ represents the Hadamard product.

[0138] In this embodiment, the first modulation module 230 may be used to execute the above step S230. For the detailed implementation of the first modulation module 230, reference may be made to the above detailed description of step S230.

[0139] The second modulation module 240 is used to modulate the first object-side phase based on the second correlation coefficient and the second implicit neural network to obtain a modulated first object-side phase.

[0140] Specifically, the second correlation coefficient can be first input into the second implicit neural network to obtain the second dynamic result. Then, the flow field grid sampler integrates the spatial phase information of the addition result of the second dynamic result and the first object phase to obtain the modulated first object phase, that is, when modulating the first object phase, a Zernike phase with a certain distortion intensity can be superimposed on the first object phase, wherein the Zernike order can be adjusted according to the actual turbulence conditions.

[0141] Among them, the modulation process of the first object phase can be expressed as:

[0142]

[0143] in, represents the first object phase after modulation, represents the sampling function, represents the flow field grid, MLP2 represents the second implicit neural network, Represents element-wise addition. t (x, y) represents a coefficient matrix related to tilt distortion, and the implementation method is the same as that shown in step S230.

[0144] In this embodiment, the second modulation module 240 may be used to execute the above step S240. The detailed implementation of the second modulation module 240 may refer to the above detailed description of step S240.

[0145] The synthesis module 250 is used to obtain a turbulence simulation image corresponding to the simulated turbulence parameters based on the modulated first object-space amplitude and the modulated first object-space phase.

[0146] Specifically, the corresponding complex amplitude can be synthesized based on the modulated first object amplitude and the modulated first object phase, and then the complex amplitude is Fourier transformed through the pupil plane to obtain a turbulence simulation image corresponding to the simulated turbulence parameters, wherein the Fourier transform process is as follows:

[0147]

[0148] Among them, S i represents the turbulence simulation image, represents the inverse Fourier transform, represents Fourier transform, P(u,v) represents pupil function, ⊙ represents Hadamard product, |*| 2 It means taking the square of the modulus length.

[0149] In this embodiment, the synthesis module 250 may be used to execute the above step S250. The detailed implementation of the synthesis module 250 may refer to the above detailed description of step S250.

[0150] In this embodiment, by dynamically adjusting the amplitude modulation and the phase modulation respectively by the first implicit neural network and the second implicit neural network, not only can the influence of atmospheric turbulence on light waves be simulated more accurately and the accuracy of the simulation results be improved, but the first correlation coefficient and the second correlation coefficient can also be dynamically adjusted according to different simulated turbulence parameters, so as to better generalize to different turbulence conditions and unknown environments.

[0151] In addition, an embodiment of the present application also provides a computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the atmospheric turbulence simulation method as described in any one of the above embodiments is implemented.

[0152] In summary, the embodiments of the present application provide an atmospheric turbulence simulation method and an atmospheric turbulence simulation device, which dynamically adjust the amplitude modulation and the phase modulation by the first implicit neural network and the second implicit neural network respectively, which can not only more accurately simulate the influence of atmospheric turbulence on light waves and improve the accuracy of the simulation results, but also dynamically adjust the first correlation coefficient and the second correlation coefficient according to different simulated turbulence parameters, so as to better generalize to different turbulence conditions and unknown environments.

[0153] The above description is only the preferred embodiment of the present application and is not intended to limit the present application. For those skilled in the art, the present application may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. A method for simulating atmospheric turbulence, characterized in that: The method comprises: Inputting the original turbulence-free image to be processed and the simulated turbulence parameters into the trained atmospheric turbulence simulation model, and obtaining the corresponding first object-space amplitude and first object-space phase based on the original turbulence-free image, wherein the atmospheric turbulence simulation model includes a first implicit neural network and a second implicit neural network; Initializing the first correlation coefficient and the second correlation coefficient based on the simulated turbulence parameter; modulating the first object-side amplitude based on the first correlation coefficient and the first implicit neural network to obtain a modulated first object-side amplitude; modulating the first object-side phase based on the second correlation coefficient and the second implicit neural network to obtain a modulated first object-side phase; A turbulence simulation image corresponding to the simulated turbulence parameter is obtained based on the modulated first object-space amplitude and the modulated first object-space phase.

2. The atmospheric turbulence simulation method according to claim 1, characterized in that: The simulated turbulence parameters include relative intensity of atmospheric turbulence, and the step of initializing the first correlation coefficient and the second correlation coefficient based on the simulated turbulence parameters includes: The first correlation coefficient and the second correlation coefficient are initialized based on the relative intensity of the atmospheric turbulence, wherein the process of initializing the first correlation coefficient and the second correlation coefficient is as follows: where D / r0 represents the relative intensity of atmospheric turbulence, randn(x,y,36) represents the coefficient matrix of random normal distribution, Z(r,θ) represents the 36th-order Zernike polynomial, ⊙ represents the Hadamard product, R a (x, y) represents the initialization result of the first correlation coefficient or the second correlation coefficient.

3. The atmospheric turbulence simulation method according to claim 1, characterized in that: The step of obtaining a turbulence simulation image corresponding to the simulated turbulence parameter based on the modulated first object-space amplitude and the modulated first object-space phase comprises: synthesizing a corresponding complex amplitude based on the modulated first object-side amplitude and the modulated first object-side phase; Performing Fourier transform on the complex amplitude to obtain a turbulence simulation image corresponding to the simulated turbulence parameter, wherein the process of Fourier transform is as follows: in, represents the first object-side amplitude after the modulation, represents the first object phase after modulation, S i represents the turbulence simulation image, represents the inverse Fourier transform, represents Fourier transform, P(u,v) represents pupil function, ⊙ represents Hadamard product, |*| 2 It means taking the square of the modulus length.

4. The atmospheric turbulence simulation method according to claim 1, characterized in that: Before the step of inputting the original turbulence-free image to be processed and the simulated turbulence parameters into the trained atmospheric turbulence simulation model, the method further includes the step of training the atmospheric turbulence simulation model, which step includes: Acquire a training image pair, wherein the training image pair includes a training image with turbulence and a training image without turbulence of the same scene; Acquire training turbulence parameters corresponding to the trained turbulence image, input the turbulence environment parameters corresponding to the trained turbulence image and the training turbulence-free image into the atmospheric turbulence simulation model to be trained, and obtain the corresponding second object-space amplitude and second object-space phase based on the training turbulence-free image, wherein the atmospheric turbulence simulation model to be trained includes a first implicit neural network to be trained and a second implicit neural network to be trained; Initializing a first correlation coefficient and a second correlation coefficient based on the training turbulence parameter; Modulating the second object-side amplitude based on the first correlation coefficient and the first implicit neural network to be trained, calculating a first loss function value based on the modulated second object-side amplitude and the trained turbulence image, and iteratively updating parameters in the first implicit neural network to be trained based on the first loss function value; The second object-side phase is modulated based on the second correlation coefficient and the second implicit neural network to be trained, and the second loss function value is calculated based on the modulated first object-side phase and the trained turbulence image. The parameters in the second implicit neural network to be trained are iteratively updated based on the second loss function value. At the end of the iteration, the atmospheric turbulence simulation model after the updated parameters is used as the trained atmospheric turbulence simulation model.

5. The atmospheric turbulence simulation method according to claim 4, characterized in that: Before the step of acquiring the training image pair, the method further includes: Acquire collected images of different scenes in different turbulent environments based on an active imaging system, and record the turbulent environment parameters corresponding to each of the collected images; A region of interest is extracted from the collected images, and the collected images are paired based on the extracted region of interest, and an image with turbulence and an image without turbulence of the same scene are used as a training image pair.

6. The atmospheric turbulence simulation method according to claim 4, characterized in that: The step of modulating the second object-side amplitude based on the first correlation coefficient and the first implicit neural network to be trained, calculating a first loss function value based on the modulated second object-side amplitude and the trained turbulence image, and iteratively updating parameters in the first implicit neural network to be trained based on the first loss function value comprises: Inputting the first correlation coefficient into the first implicit neural network to be trained to obtain a first dynamic result of training; extracting high-frequency information and low-frequency information corresponding to the original object space based on the second object space amplitude, multiplying the first dynamic result by the high-frequency information, adding the multiplication result by the low-frequency information, and integrating the spatial amplitude information of the addition result to obtain a modulated first training turbulence simulation image; A first loss function value is calculated based on the modulated first training turbulence simulation image and the trained turbulence image, and parameters in the first implicit neural network to be trained are iteratively updated based on the first loss function value. After the iteration is completed, the iteratively updated first implicit neural network is used as the trained first implicit neural network.

7. The atmospheric turbulence simulation method according to claim 4, characterized in that: The step of modulating the second object-side phase based on the second correlation coefficient and the second implicit neural network to be trained, calculating a second loss function value based on the modulated first object-side phase and the trained turbulence image, iteratively updating parameters in the second implicit neural network to be trained based on the second loss function value, and using the atmospheric turbulence simulation model after the updated parameters as the trained atmospheric turbulence simulation model at the end of the iteration includes: Inputting the second correlation coefficient into the second implicit neural network to be trained to obtain a second dynamic result of training; Adding the second dynamic result and the second object-space phase, and integrating the spatial phase information of the addition result to obtain a modulated second training turbulence simulation image; A second loss function value is calculated based on the modulated second training turbulence simulation image and the trained turbulence image, and the parameters in the second implicit neural network to be trained are iteratively updated based on the second loss function value. After the iteration, the atmospheric turbulence simulation model with updated parameters is used as the trained atmospheric turbulence simulation model.

8. The atmospheric turbulence simulation method according to claim 4, characterized in that: The first loss function or the second loss function is as follows: Wherein, Loss represents the overall loss function value of the first implicit neural network to be trained or the second implicit neural network to be trained, S i represents the first training turbulence simulation graph or the second training turbulence simulation graph, R i represents the training turbulence map, and β represents the selection threshold of the first-order norm loss function and the second-order norm loss function.

9. An atmospheric turbulence simulation device, characterized in that: The device comprises: An input module, used for inputting the original turbulence-free image to be processed and the simulated turbulence parameters into the trained atmospheric turbulence simulation model, and obtaining the corresponding first object-space amplitude and first object-space phase based on the original turbulence-free image, wherein the atmospheric turbulence simulation model includes a first implicit neural network and a second implicit neural network; An initialization module, used for initializing the first correlation coefficient and the second correlation coefficient based on the simulated turbulence parameter; A first modulation module, configured to modulate the first object-side amplitude based on the first correlation coefficient and the first implicit neural network to obtain a modulated first object-side amplitude; a second modulation module, configured to modulate the first object-side phase based on the second correlation coefficient and the second implicit neural network to obtain a modulated first object-side phase; A synthesis module is used to obtain a turbulence simulation image corresponding to the simulated turbulence parameter based on the modulated first object-space amplitude and the modulated first object-space phase.

10. A computer-readable storage medium, characterized in that: A computer program is stored thereon, and when the computer program is executed by a processor, the atmospheric turbulence simulation method as claimed in any one of claims 1 to 8 is implemented.

Images