Image atmospheric turbulence elimination method and system

Abstract

The present invention provides an image atmospheric turbulence elimination method and system, and relates to the field of image processing technology. The method includes: building a light field imaging device and collecting a target image of a target object; building a turbulence simulation scene, collecting a turbulence image and wavefront information of the target object in the turbulence simulation scene; building a turbulence image reconstruction network, and training the turbulence image reconstruction network, building a total loss function, optimizing the trained turbulence image reconstruction network through the total loss function, and obtaining an optimized turbulence image reconstruction network; reconstructing the turbulence image to be reconstructed through the optimized turbulence image reconstruction network to obtain a reconstructed target image. The mapping relationship between the turbulence image and the corresponding target image is learned through the constructed turbulence image reconstruction network, and the network is optimized in conjunction with the wavefront information, and the wavefront aberration of the turbulence image is corrected to reconstruct the target image.

Description

Technical Field

[0001] The present invention mainly relates to the technical field of image processing, and in particular to a method and system for eliminating atmospheric turbulence in an image. Background Art

[0002] Adaptive optics is an optical technology that has developed rapidly in recent years. It actively controls the wavefront shape of the optical system to eliminate the effects of atmospheric turbulence, optical component manufacturing errors, etc. on the performance of the optical system, thereby improving the imaging quality and resolution of the optical system. With the continuous advancement of technology and the continuous growth of application needs, the research on adaptive optics is also deepening.

[0003] At present, adaptive optical technology has been widely used in many fields, such as astronomical observation, laser weapons, medical imaging, etc. In the future, with the continuous development of technology and the continuous growth of application needs, the research and application of adaptive optical technology will be more extensive and in-depth. However, adaptive optical systems are usually more complex, and their optical components are expensive, and the system construction cost is high; it also involves the process of fine calibration of multiple components, which increases the difficulty of building and maintaining the system, and also reduces the reliability and stability of the system. Summary of the invention

[0004] The technical problem to be solved by the present invention is to provide a method and system for eliminating atmospheric turbulence in an image in view of the deficiencies in the prior art.

[0005] The technical solution of the present invention to solve the above technical problems is as follows:

[0006] A method for eliminating atmospheric turbulence in an image comprises the following steps:

[0007] Building a light field imaging device, and collecting a target image of a target object through the light field imaging device;

[0008] A heat source device is placed in front of the light field imaging device to build a turbulence simulation scene, and the light field imaging device is optimized, and a turbulence image of a target object is collected in the turbulence simulation scene by the optimized light field imaging device, and wavefront information is collected by the optimized light field imaging device;

[0009] constructing an initial turbulence image reconstruction network based on a self-attention mechanism, training the initial turbulence image reconstruction network through the target image, the turbulence image and the wavefront information to obtain a turbulence image reconstruction network, constructing a total loss function, optimizing the turbulence image reconstruction network through the total loss function to obtain an optimized turbulence image reconstruction network;

[0010] The turbulence image to be reconstructed is reconstructed through the optimized turbulence image reconstruction network to obtain a reconstructed target image.

[0011] Further, the light field imaging device includes a primitive nematic phase liquid crystal composite microlens array;

[0012] Before the step of building the light field imaging device, the following steps are also included:

[0013] By applying different voltages to the original nematic liquid crystal composite microlens array, the arrangement of liquid crystal molecules in the original nematic liquid crystal composite microlens array is changed to obtain the focal length corresponding to each voltage, and an imaging relationship model is constructed according to all voltages and their corresponding focal lengths.

[0014] Furthermore, the light field imaging device is constructed as follows:

[0015] The original nematic liquid crystal composite microlens array is subjected to voltage transformation calculation through the imaging relationship model to obtain a first voltage, the first voltage is applied to the original nematic liquid crystal composite microlens array to obtain a nematic liquid crystal composite microlens array, and a light field imaging device is constructed through the nematic liquid crystal composite microlens array and basic imaging components.

[0016] Furthermore, the light field imaging device is optimized, specifically:

[0017] The nematic liquid crystal composite microlens array is subjected to voltage transformation calculation through the imaging relationship model to obtain a second voltage, the second voltage is applied to the nematic liquid crystal composite microlens array to obtain an optimized nematic liquid crystal composite microlens array, the nematic liquid crystal composite microlens array of the light field imaging device is replaced with the optimized nematic liquid crystal composite microlens array to obtain an optimized light field imaging device.

[0018] Further, the optimized light field imaging device comprises a wavefront sensor, which is composed of the optimized nematic liquid crystal composite microlens array and a photodetector CCD arranged in sequence along an imaging path, wherein the optimized nematic liquid crystal composite microlens array is composed of a plurality of optimized nematic liquid crystal composite microlenses;

[0019] The wavefront information is collected by the optimized light field imaging device, specifically:

[0020] When collecting the turbulence image, an incident light wave is irradiated onto the plurality of optimized nematic phase liquid crystal composite microlenses, and the incident light wave is focused into a plurality of corresponding light spots by the plurality of optimized nematic phase liquid crystal composite microlenses respectively, and the plurality of corresponding light spots are irradiated onto a focal plane;

[0021] The photoelectric detector CCD collects multiple light spots on the focal plane, and calculates the center position of each light spot. The center position of each light spot is calculated by a wavefront restoration algorithm to obtain wavefront information.

[0022] Furthermore, the center position of each light spot is calculated by the wavefront restoration algorithm to obtain the wavefront information, which is specifically:

[0023] Calculating the center position of each light spot with the corresponding theoretical center position collected in advance to obtain a plurality of corresponding offsets, wherein the offsets include an x-axis offset component and a y-axis offset component;

[0024] The offset components of multiple x-axes are calculated by the wavefront slope expression to obtain multiple corresponding sub-wavefront slopes of the x-axes. The offset components of multiple y-axes are calculated by the wavefront slope expression to obtain multiple corresponding sub-wavefront slopes of the y-axes. The wavefront slope expression is:

[0025] S x =Δx / f,

[0026] S y =Δy / f,

[0027] Among them, S x is the sub-wavefront slope of the x-axis, S y is the sub-wavefront slope of the x-axis, Δx is the offset component of the x-axis, Δy is the offset component of the y-axis, and f is the focal length of the optimized nematic liquid crystal composite microlens array;

[0028] The wavefront information is obtained by integrating the plurality of sub-wavefront slopes through an integral calculation expression, and the integral calculation expression is:

[0029] Φ(x,y)=∫S x (x,y)dx+∫S y (x,y)dy,

[0030] Among them, Φ(x,y) is the wavefront information.

[0031] Furthermore, the initial turbulence image reconstruction network is constructed based on the self-attention mechanism, specifically:

[0032] The initial turbulence image reconstruction network is constructed through a refinement block, n encoding blocks and n decoding blocks, including:

[0033] At the input end, connect n encoding blocks one by one, connect the first encoding block to the input port, and connect the last encoding block to the input of the refinement block;

[0034] At the output end, the n decoding blocks are connected one by one, and the output of the refinement block is connected to the first decoding block, and the last decoding block is connected to the output port;

[0035] Connect the output of the first encoding block to the input of the last decoding block, and connect the output of the second encoding block to the input of the n-1th decoding block, and so on, to construct the initial turbulence image reconstruction network;

[0036] The initial turbulence image reconstruction network is trained by using the target image, the turbulence image and the wavefront information to obtain the turbulence image reconstruction network, specifically:

[0037] The target image, the turbulence image and the wavefront information are feature extracted by the first coding block, and the extracted feature maps are respectively input into the second coding block and the last decoding block; the second coding block to the n-1th coding block all perform feature extraction on the feature map extracted by the previous coding block, and the corresponding extracted feature maps are respectively input into the corresponding next coding block and the corresponding decoding block; the last coding block performs feature extraction on the feature map extracted by the n-1th coding block, and the extracted feature maps are respectively input into the refinement block and the first decoding block;

[0038] Performing a texture refinement operation on the feature map extracted by the last encoding block through the refinement block, and inputting the refined feature map into the first decoding block;

[0039] The first decoding block performs a feature recovery operation on the feature map extracted by the last encoding block and the feature map refined by the refinement block, and the recovered feature map is input into the second decoding block. The second decoding block to the last decoding block all perform a feature recovery operation on the feature map recovered by the previous decoding block and the feature map extracted by the corresponding encoding block. The last decoding block obtains an initial reconstructed target image after the feature recovery operation, and outputs the initial reconstructed target image to the output port, completing the training to obtain a turbulent image reconstruction network.

[0040] Furthermore, the encoding blocks each include an encoding convolution layer, an encoding self-attention mechanism module, an encoding pooling layer, and an encoding forward feedback layer;

[0041] The feature extraction of the target image, the turbulence image and the wavefront information is performed by the first coding block, specifically:

[0042] The wavefront information is calculated through the coding convolution layer to obtain a degradation operator, the turbulence features of the turbulence image degraded from the target image are extracted according to the degradation operator, the turbulence image is reconstructed according to the turbulence features to obtain a local reconstructed feature map, the local reconstructed feature map is weighted summed through the coding self-attention mechanism module to obtain a coding weighted feature map, the coding weighted feature map is downsampled through the coding pooling layer, and the downsampled feature map is residually connected through the coding forward feedback layer;

[0043] The decoding blocks each include a decoding convolutional layer, a decoding self-attention mechanism module, a decoding pooling layer, and a decoding forward feedback layer;

[0044] The performing of a feature recovery operation on the feature map extracted by the last coding block and the feature map refined by the refinement block by the first decoding block is specifically as follows:

[0045] The feature map downsampled by the last encoding block and the feature map refined by the refinement block are fused through the decoding convolution layer to obtain a decoding fused feature map, the decoding fused feature map is weighted summed up through the decoding self-attention mechanism module to obtain a decoding weighted feature map, the decoding weighted feature map is upsampled through the decoding pooling layer, and the upsampled feature map is residually connected through the decoding forward feedback layer.

[0046] Furthermore, the total loss function is constructed by scaling parameters of the reconstruction loss function and the turbulence estimation loss function;

[0047] The turbulence image reconstruction network is optimized by the total loss function to obtain an optimized turbulence image reconstruction network, specifically:

[0048] The initial reconstructed target image and the target image are calculated by using the reconstruction loss function expression to obtain the reconstruction loss. The reconstruction loss function expression is:

[0049]

[0050] Among them, L0 is the reconstruction loss function, is the initial reconstructed target image, I is the target image;

[0051] The target image is subjected to turbulence degradation by the degradation operator to obtain an estimated turbulence image, and the estimated turbulence image and the turbulence image are calculated by the turbulence estimation loss function expression to obtain a turbulence estimation loss. The turbulence estimation loss function expression is:

[0052]

[0053] Among them, L1 is the turbulence estimation loss function, To estimate the turbulence image, is a turbulence image;

[0054] The reconstruction loss and the turbulence estimation loss are calculated by using a total loss function expression to obtain a total loss, and the total loss function expression is:

[0055] L=L 0 +αL 1 ,

[0056] Among them, L is the total loss function and α is the scaling parameter;

[0057] The turbulence image reconstruction network is iteratively optimized through the total loss to obtain an optimized turbulence image reconstruction network.

[0058] Another technical solution of the present invention to solve the above technical problems is as follows:

[0059] An image atmospheric turbulence elimination system, comprising:

[0060] A data acquisition unit is used to build a light field imaging device, collect a target image of a target object through the light field imaging device, place a heat source device in front of the light field imaging device to build a turbulence simulation scene, and optimize the light field imaging device, collect a turbulence image of the target object in the turbulence simulation scene through the optimized light field imaging device, and collect wavefront information through the optimized light field imaging device;

[0061] Constructing a network unit, which is used to construct an initial turbulence image reconstruction network based on a self-attention mechanism, training the initial turbulence image reconstruction network through the target image, the turbulence image and the wavefront information to obtain a turbulence image reconstruction network, constructing a total loss function, and optimizing the turbulence image reconstruction network through the total loss function to obtain an optimized turbulence image reconstruction network;

[0062] The reconstruction unit is used to reconstruct the turbulence image to be reconstructed through the optimized turbulence image reconstruction network to obtain a reconstructed target image.

[0063] The beneficial effects of the present invention are: learning the mapping relationship between turbulent images and corresponding target images through the constructed turbulent image reconstruction network, combining wavefront information to perform network optimization, correcting the wavefront aberration of the turbulent image to efficiently and accurately reconstruct the target image, so as to realize the reconstruction of the wavefront-perceived turbulent image, and then eliminating the image blurring phenomenon caused by the atmospheric turbulence effect when acquiring the image. BRIEF DESCRIPTION OF THE DRAWINGS

[0064] Figure 1A flowchart of a method for eliminating atmospheric turbulence in an image provided by an embodiment of the present invention;

[0065] Figure 2 A schematic diagram of imaging a turbulence simulation scene provided by an embodiment of the present invention;

[0066] Figure 3 A structural diagram of a turbulence image reconstruction network provided by an embodiment of the present invention;

[0067] Figure 4 A structural diagram of an encoder of a turbulent image reconstruction network provided by an embodiment of the present invention;

[0068] Figure 5 A structural diagram of a decoder of a turbulent image reconstruction network provided by an embodiment of the present invention;

[0069] Figure 6 A unit block diagram of an image atmospheric turbulence elimination system provided by an embodiment of the present invention.

[0070] In the accompanying drawings, the names of the components represented by the various marks are as follows:

[0071] 1. Camera; 2. Liquid crystal microlens array; 3. Main lens; 4. Heat source device; 5. Target object. DETAILED DESCRIPTION

[0072] The principles and features of the present invention are described below in conjunction with the accompanying drawings. The examples given are only used to explain the present invention and are not used to limit the scope of the present invention.

[0073] Atmospheric turbulence is a complex form of atmospheric motion, which is caused by the interaction of various waves and vortices in the atmosphere. In turbulence, the flow direction and speed of air will change continuously, causing corresponding changes in parameters such as air density, temperature and pressure. The distortion caused by turbulence is formed by the interweaving of spatial variation blur, geometric distortion and sensor noise. Turbulence reconstruction research is one of the important research directions in the current field of optics, aiming to improve the clarity and resolution of turbulence images through reconstruction algorithms and optimization techniques.

[0074] Due to the complexity and uncertainty of turbulence phenomena, existing turbulence-degraded image reconstruction systems often have difficulty accurately reconstructing the original image. The reconstruction results may have problems such as blurring and distortion, which cannot meet the needs of practical applications. In order to solve these problems, it is necessary to further study and explore new methods and technologies, such as optimizing algorithms and introducing deep learning technology, to improve the performance and reliability of turbulence-degraded image reconstruction systems.

[0075] like Figure 1 As shown, an embodiment of the present invention provides a method for eliminating atmospheric turbulence in an image, comprising the following steps:

[0076] Building a light field imaging device, and collecting a target image of a target object through the light field imaging device;

[0077] A heat source device is placed in front of the light field imaging device to build a turbulence simulation scene, and the light field imaging device is optimized, and a turbulence image of a target object is collected in the turbulence simulation scene by the optimized light field imaging device, and wavefront information is collected by the optimized light field imaging device;

[0078] constructing an initial turbulence image reconstruction network based on a self-attention mechanism, training the initial turbulence image reconstruction network through the target image, the turbulence image and the wavefront information to obtain a turbulence image reconstruction network, constructing a total loss function, optimizing the turbulence image reconstruction network through the total loss function to obtain an optimized turbulence image reconstruction network;

[0079] The turbulence image to be reconstructed is reconstructed through the optimized turbulence image reconstruction network to obtain a reconstructed target image.

[0080] Among them, it is necessary to repeat the process of iteratively optimizing the trained turbulent image reconstruction network through the total loss function until the network converges.

[0081] Specifically, since high surface temperature can cause non-uniform air media to form a disordered flow field, the refractive index of different regions along the optical path changes, and the air is no longer regarded as a single medium, thereby generating turbulence. Therefore, the turbulence effect is simulated by heating the air along the imaging path, a turbulence simulation scene.

[0082] In the embodiment of the present invention, a turbulence simulation scene is built to collect wavefront information of turbulence images, which greatly improves the utilization rate of scene information collected by the light field imaging device and effectively improves the quality of turbulence image reconstruction; a deep learning neural network is constructed to enable it to reconstruct turbulence images and reconstruct high-definition images;

[0083] Among them, an optimized light field imaging device is used to obtain the incident wavefront information, and a deep learning neural network (i.e., the constructed turbulence image reconstruction network) is used to learn the mapping relationship between the turbulence image and the corresponding target image. The wavefront information is combined to perform network optimization, and the wavefront aberration of the turbulence image is corrected to efficiently and accurately reconstruct the target image, thereby realizing the reconstruction of the wavefront-perceived turbulence image.

[0084] Preferably, the light field imaging device comprises a primitive nematic liquid crystal composite microlens array;

[0085] Before the step of building the light field imaging device, the following steps are also included:

[0086] By applying different voltages to the original nematic liquid crystal composite microlens array, the arrangement of liquid crystal molecules in the original nematic liquid crystal composite microlens array is changed to obtain the focal length corresponding to each voltage, and an imaging relationship model is constructed according to all voltages and their corresponding focal lengths.

[0087] Specifically, applying voltage to the original nematic liquid crystal composite microlens array causes the liquid crystal molecules to rearrange, thereby changing the refractive index of the original nematic liquid crystal composite microlens array. The change in refractive index affects the propagation path of the light wave, thereby changing the imaging result. According to Snell's law, the expression of the refraction law of light is obtained as follows:

[0088] n 1 *sinθ 1 =n 2 *sinθ 2 ,

[0089] Among them, n 1 and n 2 are the refractive indices of the two media, θ 1 and θ 2 are the angles of incidence and refraction of light in the two media respectively.

[0090] More specifically, when voltage is applied, the liquid crystal molecules of the original nematic phase liquid crystal composite microlens array will gradually rise, the focal length of the original nematic phase liquid crystal composite microlens array will change, and the imaging result will become clearer. In addition, the liquid crystal electrically controlled adjustable microlens array can also adjust the shape and refractive index of the microlens by controlling the voltage, thereby achieving precise control of light and optimization of imaging.

[0091] Since the liquid crystal electrically controlled adjustable microlens array (i.e., the original nematic phase liquid crystal composite microlens array) images the target object by zooming, the focal length of the liquid crystal electrically controlled adjustable microlens array is adjusted by adjusting the voltage of the liquid crystal electrically controlled adjustable microlens array, and then the imaging relationship between the voltage and the liquid crystal device (i.e., the original nematic phase liquid crystal composite microlens array) is derived. According to the corresponding relationship between the voltage applied to the original nematic phase liquid crystal composite microlens array and the corresponding focal length change, an imaging relationship model is constructed.

[0092] It should be understood that the voltage applied to the original nematic liquid crystal composite microlens array includes an initial driving voltage and an external voltage. The initial driving voltage needs to be selected and adjusted according to factors such as experimental conditions, liquid crystal material properties, microlens structure (i.e., original nematic liquid crystal composite microlens array structure) to obtain the best imaging effect. The range of the applied voltage can be randomly selected between 10Vrms-14Vrms, and the parameters of the photodetector CCD are adjusted to obtain the best imaging effect.

[0093] In the embodiment of the present invention, since the focal length of the light wave after passing through the liquid crystal electrically-controlled adjustable microlens array during imaging will lead to unclear imaging, the voltage of the liquid crystal electrically-controlled adjustable microlens array is adjusted to rearrange it, so as to obtain a focal length that can make the imaging clear, so as to derive the optical imaging characteristics of the liquid crystal electrically-controlled adjustable microlens array, that is, the imaging relationship between the voltage and the liquid crystal electrically-controlled adjustable microlens array is an adjustable and controllable mapping relationship, and then the imaging relationship model is constructed according to the imaging characteristics.

[0094] Preferably, the light field imaging device is constructed by:

[0095] The original nematic liquid crystal composite microlens array is subjected to voltage transformation calculation through the imaging relationship model to obtain a first voltage, the first voltage is applied to the original nematic liquid crystal composite microlens array to obtain a nematic liquid crystal composite microlens array, and a light field imaging device is constructed through the nematic liquid crystal composite microlens array and basic imaging components.

[0096] Specifically, the basic imaging components include a main lens, a polarizer, and a camera (i.e., a CCD camera, i.e., a photodetector CCD). The target object, the main lens, the polarizer, the liquid crystal microlens array (i.e., a nematic liquid crystal composite microlens array), and the photodetector CCD are placed on the same optical axis in this order.

[0097] In the embodiment of the present invention, a corresponding voltage is applied to the original nematic liquid crystal composite microlens array according to the imaging relationship model to change the imaging focal length, so that the image collected by the photoelectric detector CCD is clearer.

[0098] Preferably, the collecting the target image of the target object by the light field imaging device is specifically:

[0099] The incident light wave passes through the main lens and the polarizer respectively and then illuminates multiple nematic liquid crystal composite microlenses (i.e., nematic liquid crystal composite microlens array). Then, the incident light wave is focused on the photodetector CCD to form a target image.

[0100] Among them, the liquid crystal microlens array is a nematic liquid crystal composite microlens array. Because the nematic liquid crystal composite microlens array uses liquid crystal materials, the focal length of the liquid crystal microlens array can be freely and quickly adjusted under the action of an external electric field. By controlling the loaded voltage and adjusting the focal length to adapt to the collection of turbulent images, the imaging quality can be improved.

[0101] In the embodiment of the present invention, the target image is collected so as to subsequently construct a training set to train the turbulence image reconstruction network.

[0102] Preferably, if Figure 2 As shown, the heat source device is placed in front of the light field imaging device to build a turbulence simulation scene, specifically:

[0103] According to the target object 5, the heat source device 4, the main lens 3, the polarizing plate (not in Figure 2 ), the liquid crystal microlens array 2, and the camera 1 (ie, the photodetector CCD) are placed on the same optical axis in this order.

[0104] Preferably, the light field imaging device is optimized, specifically:

[0105] The nematic liquid crystal composite microlens array is subjected to voltage transformation calculation through the imaging relationship model to obtain a second voltage, the second voltage is applied to the nematic liquid crystal composite microlens array to obtain an optimized nematic liquid crystal composite microlens array, the nematic liquid crystal composite microlens array of the light field imaging device is replaced with the optimized nematic liquid crystal composite microlens array to obtain an optimized light field imaging device.

[0106] In the embodiment of the present invention, due to the change of the acquisition environment (i.e., acquisition in a turbulence simulation scene), the medium of light propagation changes, so a corresponding voltage is applied to the nematic liquid crystal composite microlens array according to the imaging relationship model to change the imaging focal length, so that the image collected by the photodetector CCD is clearer.

[0107] Preferably, the optimized light field imaging device comprises a wavefront sensor, which is composed of the optimized nematic liquid crystal composite microlens array and a photodetector CCD arranged in sequence along an imaging path, wherein the optimized nematic liquid crystal composite microlens array is composed of a plurality of optimized nematic liquid crystal composite microlenses;

[0108] The wavefront information is collected by the optimized light field imaging device, specifically:

[0109] When collecting the turbulence image, an incident light wave is irradiated onto the plurality of optimized nematic phase liquid crystal composite microlenses, and the incident light wave is focused into a plurality of corresponding light spots by the plurality of optimized nematic phase liquid crystal composite microlenses respectively, and the plurality of corresponding light spots are irradiated onto a focal plane;

[0110] The photoelectric detector CCD collects multiple light spots on the focal plane, and calculates the center position of each light spot. The center position of each light spot is calculated by a wavefront restoration algorithm to obtain wavefront information.

[0111] It should be understood that the wavefront sensing technology is used to collect information about the phase of the incident light and obtain the distorted wavefront information (i.e., wavefront information). The photodetector CCD collects multiple light spots on the focal plane, calibrates the positions of the multiple light spots, transmits the calibrated positions of the multiple light spots (i.e., the center positions of the light spots) to the computer, and calculates the calibrated positions of the light spots through the wavefront restoration algorithm to obtain the wavefront information.

[0112] The wavefront sensor is a Shack-Hartmann wavefront sensor, which combines the Shack-Hartmann wavefront sensing technology to collect the wavefront information of the turbulence image, greatly improving the utilization rate of the scene information collected by the light field imaging device. The Shack-Hartmann wavefront sensing technology is used to illuminate the wavefront generated in the turbulence simulation scene onto the optimized light field imaging device to obtain the incident wavefront information.

[0113] In an embodiment of the present invention, the influence of atmospheric turbulence on light waves is manifested as phase difference, which will affect the imaging of the light field imaging device. Therefore, the wavefront information generated by the atmospheric turbulence is collected, and phase compensation is performed based on the wavefront information to eliminate the phase difference between the turbulence image and the target image, and the turbulence image is reconstructed to obtain the target image.

[0114] Preferably, the center position of each light spot is calculated by a wavefront restoration algorithm to obtain wavefront information, specifically:

[0115] Calculating the center position of each light spot with the corresponding theoretical center position collected in advance to obtain a plurality of corresponding offsets, wherein the offsets include an x-axis offset component and a y-axis offset component;

[0116] The offset components of multiple x-axes are calculated by the wavefront slope expression to obtain multiple corresponding sub-wavefront slopes of the x-axes. The offset components of multiple y-axes are calculated by the wavefront slope expression to obtain multiple corresponding sub-wavefront slopes of the y-axes. The wavefront slope expression is:

[0117] S x =Δx / f,

[0118] S y =Δy / f,

[0119] Among them, S x is the sub-wavefront slope of the x-axis, S y is the sub-wavefront slope of the x-axis, Δx is the offset component of the x-axis, Δy is the offset component of the y-axis, and f is the focal length of the optimized nematic liquid crystal composite microlens array;

[0120] The wavefront information is obtained by integrating the plurality of sub-wavefront slopes through an integral calculation expression, and the integral calculation expression is:

[0121] Φ(x,y)=∫Sx(x,y)dx+∫Sy(x,y)dy,

[0122] Among them, Φ(x,y) is the wavefront information.

[0123] It should be understood that the theoretical center position of the light spot is the center position of the light wave without wavefront information irradiated on the focal plane. The multiple sub-wavefront slopes are integrated to obtain phase information. Since phase compensation needs to be performed based on the wavefront information when reconstructing the turbulence image, the phase information obtained by integration is the wavefront information.

[0124] Specifically, the focal length of the optimized nematic phase liquid crystal composite microlens array is calculated in advance, and the focal length calculation expression of the liquid crystal microlens array is:

[0125]

[0126] Where, f is the focal length of the optimized nematic liquid crystal composite microlens array, n max represents the maximum refractive index of the liquid crystal, r is the radius of the aperture of a single lens in the liquid crystal microlens array, n(r) is the liquid crystal refractive index distribution function formed in the liquid crystal layer under the action of an applied voltage, and d is the thickness of the liquid crystal layer.

[0127] In the embodiment of the present invention, the wavefront information is calculated to facilitate training of the turbulence image reconstruction network, so that the network can learn the compensation phase required for the atmospheric turbulence effect.

[0128] Preferably, the initial turbulence image reconstruction network is constructed based on the self-attention mechanism, specifically:

[0129] The initial turbulence image reconstruction network is constructed through a refinement block, n encoding blocks and n decoding blocks, including:

[0130] At the input end, connect n encoding blocks one by one, connect the first encoding block to the input port, and connect the last encoding block to the input of the refinement block;

[0131] At the output end, the n decoding blocks are connected one by one, and the output of the refinement block is connected to the first decoding block, and the last decoding block is connected to the output port;

[0132] Connect the output of the first encoding block to the input of the last decoding block, and connect the output of the second encoding block to the input of the n-1th decoding block, and so on, to construct the initial turbulence image reconstruction network;

[0133] The initial turbulence image reconstruction network is trained by using the target image, the turbulence image and the wavefront information to obtain the turbulence image reconstruction network, specifically:

[0134] The target image, the turbulence image and the wavefront information are feature extracted by the first coding block, and the extracted feature maps are respectively input into the second coding block and the last decoding block; the second coding block to the n-1th coding block all perform feature extraction on the feature map extracted by the previous coding block, and the corresponding extracted feature maps are respectively input into the corresponding next coding block and the corresponding decoding block; the last coding block performs feature extraction on the feature map extracted by the n-1th coding block, and the extracted feature maps are respectively input into the refinement block and the first decoding block;

[0135] Performing a texture refinement operation on the feature map extracted by the last encoding block through the refinement block, and inputting the refined feature map into the first decoding block;

[0136] The first decoding block performs a feature recovery operation on the feature map extracted by the last encoding block and the feature map refined by the refinement block, and the recovered feature map is input into the second decoding block. The second decoding block to the last decoding block all perform a feature recovery operation on the feature map recovered by the previous decoding block and the feature map extracted by the corresponding encoding block. The last decoding block obtains an initial reconstructed target image after the feature recovery operation, and outputs the initial reconstructed target image to the output port, completing the training to obtain a turbulent image reconstruction network.

[0137] Specifically, Figure 3 As shown in the figure, the turbulence image reconstruction network based on the self-attention mechanism is a U-Net network. Based on the self-attention mechanism module, the encoder block and the decoder block are respectively constructed, and are set to three respectively. The three encoder blocks are connected one by one, and then a refinement block is connected. The refinement block is connected to the three decoder blocks one by one, wherein the output of each encoder block is skip-connected to the input of the decoder block, specifically:

[0138] The input port is connected to the input of the first encoding block (i.e., the first encoding block), the output of the first encoding block is connected to the input of the second encoding block, the output of the second encoding block is connected to the input of the third encoding block, the output of the third encoding block (i.e., the last encoding block) is connected to the input of the refinement block, the output of the refinement block is connected to the input of the first decoding block (i.e., the first decoding block), the output of the first decoding block is connected to the input of the second decoding block, the output of the second decoding block is connected to the input of the third decoding block (i.e., the last decoding block), and the output of the third decoding block is connected to the output port; the output of the first encoding block is jump-connected to the input of the third decoding block, the output of the second encoding block is jump-connected to the input of the second decoding block, and the output of the third encoding block is jump-connected to the input of the first decoding block to construct a turbulent image reconstruction network.

[0139] It should be understood that the jump connection provides a connection method based on identity mapping, which helps to aggregate features of different layers so that the underlying features (i.e., the features output by the encoding block) and the high-level features (i.e., the features output by the decoding block) can communicate and fuse. This direct connection helps maintain the stability of the gradient and makes the network easier to train, thereby improving the performance and learning ability of the network. By adding or splicing the feature map in the encoder to the feature map of the corresponding layer in the decoder, the decoder can use the underlying features extracted in the encoder stage when generating a new feature map, thereby retaining more detailed information.

[0140] Building a turbulent image reconstruction network based on the self-attention mechanism can help the network capture the interactions and dependencies between different positions in the turbulent image, not just local information, so that the network can learn how to correct the image to eliminate the blur and distortion caused by turbulence.

[0141] When the turbulence image reconstruction network is trained, the target image, the turbulence image and the wavefront information are input into the input port; when the image is reconstructed by optimizing the turbulence image reconstruction network, the turbulence image is input into the input port.

[0142] In an embodiment of the present invention, in a U-Net network, the self-attention mechanism can help the network better understand the global context of the image, thereby improving the accuracy of reconstruction. In the encoder, the self-attention mechanism module can help the network focus on calculating the similarity between each pixel in the image and other pixels, thereby capturing global dependencies. In the decoder, the self-attention mechanism module can help the network focus on the image area that is most relevant to the current pixel reconstruction, thereby providing finer features at the edges and detail areas of the image.

[0143] Preferably, if Figure 4As shown, the encoder blocks (Encoder Block, EB) include an encoding convolutional layer (EB-Convolutional Layer), an encoding self-attention mechanism module (EB-Self-Attention), an encoding pooling layer (EB-Pooling Layer) and an encoding forward feedback layer (EB-Feed Forward).

[0144] like Figure 5 As shown, the decoding blocks (Decoder Block, DB) include a decoding convolutional layer (DB-Convolutional Layer), a decoding self-attention mechanism module (DB-Self-Attention), a decoding pooling layer (DB-Pooling Layer) and a decoding forward feedback layer (DB-Feed Forward).

[0145] Specifically, the low-frequency information of the target image, turbulence image and wavefront information (i.e., input) is extracted through the encoder's encoding convolution layer; the image pixels are weighted summed through the encoding self-attention mechanism module, and the weight of each convolution kernel is updated through the back-propagation algorithm during the training process; the image dimension is reduced through the encoding pooling layer, the amount of calculation is reduced while retaining the main features, and downsampling is achieved; the feature map in the encoder is connected to the feature map of the corresponding layer of the decoder through the encoding forward feedback layer to achieve image feature fusion, which helps to restore more accurate image details during the decoding process.

[0146] The input feature map is fused through the decoder's decoding convolution layer; the image pixels are weighted summed through the decoding self-attention mechanism module; the spatial resolution of the feature map is restored through the decoding pooling layer, and new features are learned to achieve upsampling; the decoding forward feedback layer is connected with the corresponding layer to output the feature map; among them, the final reconstructed image (i.e., the output terminal Output) is output through the decoding forward feedback layer of the last decoder.

[0147] Among them, the jump connection is: the encoding forward feedback layer in the encoding block is jump-connected with the decoding convolutional layer in the decoding block.

[0148] Preferably, the initial turbulence image reconstruction network is trained by the target image, the turbulence image and the wavefront information, and the processing process of the three encoding blocks, the three decoding blocks and the refinement block is:

[0149] The target image, turbulence image and wavefront information are input into the first coding convolution layer of the first coding block through the input port, and the target image, turbulence image and wavefront information are feature extracted through the first coding block, specifically:

[0150] The wavefront information is calculated through the first coding convolution layer to obtain a degradation operator, the turbulence features of the turbulence image degenerated from the target image are extracted according to the degradation operator, the turbulence image is reconstructed according to the turbulence features to obtain a first local reconstructed feature map, the first local reconstructed feature map is weighted summed through the first coding self-attention mechanism module to obtain a first coding weighted feature map, the first coding weighted feature map is downsampled through the first coding pooling layer to obtain a first coding feature map, the first coding feature map is residually connected through the first coding forward feedback layer, and the first coding feature map is input into the second coding convolution layer of the second coding block and the third decoding convolution layer of the third decoding block.

[0151] The second encoding block is used to extract features from the first encoding feature map, specifically:

[0152] The first encoding feature map is subjected to local feature extraction through the second encoding convolution layer to obtain a second local reconstructed feature map, the second local reconstructed feature map is subjected to weighted summation through the second encoding self-attention mechanism module to obtain a second encoding weighted feature map, the second encoding weighted feature map is subjected to downsampling operation through the second encoding pooling layer to obtain a second encoding feature map, the second encoding feature map is subjected to residual connection through the second encoding forward feedback layer, and the second encoding feature map is input into the third encoding convolution layer of the third encoding block and the second decoding convolution layer of the second decoding block.

[0153] The third encoding block is used to extract features from the second encoding feature map, specifically:

[0154] The second encoded feature map is subjected to local feature extraction through the third encoded convolution layer to obtain a third local reconstructed feature map, the third local reconstructed feature map is subjected to weighted summation through the third encoded self-attention mechanism module to obtain a third encoded weighted feature map, the third encoded weighted feature map is subjected to downsampling operation through the third encoded pooling layer to obtain a third encoded feature map, the third encoded feature map is subjected to residual connection through the third encoded forward feedback layer, and the third encoded feature map is input into the refinement block and the first decoding convolution layer of the first decoding block.

[0155] The texture refinement operation is performed on the third encoded feature map through the refinement block, specifically:

[0156] The texture refinement operation is performed on the third encoding feature map through the refinement convolution layer to obtain an initial refinement feature map; the initial refinement feature map is activated through the activation layer to obtain a refinement feature map and input into the first decoding convolution layer of the first decoding block.

[0157] The first decoding block performs feature recovery operations on the third encoded feature map and the refined feature map, specifically:

[0158] The third encoding feature map and the refined feature map are fused through the first decoding convolution layer to obtain a first decoding fusion feature map; the first decoding fusion feature map is weighted summed through the first decoding self-attention mechanism module to obtain a first decoding weighted feature map; the first decoding weighted feature map is upsampled through the first decoding pooling layer to obtain a first restored feature map; the first restored feature map is residually connected through the first decoding forward feedback layer, and the first restored feature map is input into the second decoding convolution layer of the second decoding block.

[0159] The second decoding block performs a feature recovery operation on the second encoded feature map and the first restored feature map, specifically:

[0160] The second encoding feature map and the first restored feature map are fused through the second decoding convolution layer to obtain a second decoding fused feature map; the second decoding fused feature map is weighted summed through the second decoding self-attention mechanism module to obtain a second decoding weighted feature map; the second decoding weighted feature map is upsampled through the second decoding pooling layer to obtain a second restored feature map; the second restored feature map is residually connected through the second decoding forward feedback layer, and the second restored feature map is input into the third decoding convolution layer of the third decoding block.

[0161] The third decoding block performs a feature recovery operation on the first encoded feature map and the second restored feature map, specifically:

[0162] The first encoding feature map and the second recovery feature map are fused through the third decoding convolution layer to obtain a third decoding fusion feature map; the third decoding fusion feature map is weighted summed through the third decoding self-attention mechanism module to obtain a third decoding weighted feature map; the third decoding weighted feature map is upsampled through the third decoding pooling layer to obtain an initial reconstructed target image; the initial reconstructed target image is residually connected through the third decoding forward feedback layer, and the initial reconstructed target image is input to the output port.

[0163] Preferably, the total loss function is constructed by scaling parameters of a reconstruction loss function and a turbulence estimation loss function;

[0164] The turbulence image reconstruction network is optimized by the total loss function to obtain an optimized turbulence image reconstruction network, specifically:

[0165] The initial reconstructed target image and the target image are calculated by using the reconstruction loss function expression to obtain the reconstruction loss. The reconstruction loss function expression is:

[0166]

[0167] Among them, L0 is the reconstruction loss function, is the initial reconstructed target image, I is the target image;

[0168] The target image is subjected to turbulence degradation by the degradation operator to obtain an estimated turbulence image, and the estimated turbulence image and the turbulence image are calculated by the turbulence estimation loss function expression to obtain a turbulence estimation loss. The turbulence estimation loss function expression is:

[0169]

[0170] Among them, L1 is the turbulence estimation loss function, To estimate the turbulence image, is a turbulence image;

[0171] The reconstruction loss and the turbulence estimation loss are calculated by using a total loss function expression to obtain a total loss, and the total loss function expression is:

[0172] L=L 0 +αL 1 ,

[0173] Among them, L is the total loss function, and α is a learnable scaling parameter;

[0174] The turbulence image reconstruction network is iteratively optimized through the total loss to obtain an optimized turbulence image reconstruction network.

[0175] It should be understood that the degradation operator τ calculated based on the wavefront information can enable the turbulence image reconstruction network to learn the phase compensation between the target image and the turbulence image. Therefore, the degradation operator is optimized according to the turbulence estimation loss function, and the turbulence image reconstruction network is trained by the optimized degradation operator to obtain a more accurate turbulence image reconstruction network.

[0176] In an embodiment of the present invention, the output image of the network is compared with the corresponding target image, the loss function is calculated, and based on the gradient information of the loss function, the reconstruction loss and the wavefront sensing loss of the turbulent image reconstruction network are jointly optimized. The weights and parameters of the network are iteratively updated through the back propagation algorithm to reduce the loss and improve the network performance.

[0177] Preferably, the step of training and optimizing the constructed initial turbulence image reconstruction network may also be:

[0178] The target image, the turbulence image and the wavefront information are constructed into a set of data, and multiple sets of data are collected to construct a data set, and the data set is divided into a training set and a verification set;

[0179] The initial turbulent image reconstruction network is trained by the training set to obtain a turbulent image reconstruction network, the turbulent image reconstruction network is iteratively optimized by the total loss function to obtain an iterative turbulent image reconstruction network, the iterative turbulent image reconstruction network is evaluated by the validation set, and the iterative turbulent image reconstruction network is optimized according to the evaluation result to obtain an optimized turbulent image reconstruction network.

[0180] It should be understood that the training set includes multiple groups of training data, including training target images, training turbulence images and training wavefront information; the verification set includes multiple groups of verification data, including verification target images, verification turbulence images and verification wavefront information.

[0181] In the embodiment of the present invention, the turbulence image reconstruction network is trained by the training set, the network parameters are adjusted, and the network reconstruction accuracy of the turbulence image is improved. The turbulence image reconstruction network is evaluated by the validation set to prevent overfitting during network optimization, thereby resulting in low model generalization ability and decreased reconstruction accuracy.

[0182] like Figure 6 As shown, an image atmospheric turbulence elimination system provided by an embodiment of the present invention includes:

[0183] A data acquisition unit is used to build a light field imaging device, collect a target image of a target object through the light field imaging device, place a heat source device in front of the light field imaging device to build a turbulence simulation scene, and optimize the light field imaging device, collect a turbulence image of the target object in the turbulence simulation scene through the optimized light field imaging device, and collect wavefront information through the optimized light field imaging device;

[0184] Constructing a network unit, which is used to construct an initial turbulence image reconstruction network based on a self-attention mechanism, training the initial turbulence image reconstruction network through the target image, the turbulence image and the wavefront information to obtain a turbulence image reconstruction network, constructing a total loss function, and optimizing the turbulence image reconstruction network through the total loss function to obtain an optimized turbulence image reconstruction network;

[0185] The reconstruction unit is used to reconstruct the turbulence image to be reconstructed through the optimized turbulence image reconstruction network to obtain a reconstructed target image.

[0186] For the above-mentioned image atmospheric turbulence elimination system, the implementation contents and beneficial effects thereof can be referred to the above-mentioned detailed description of an image atmospheric turbulence elimination method, which will not be described in detail here.

[0187] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device.

[0188] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A method for eliminating atmospheric turbulence in an image, characterized in that: The steps include: Building a light field imaging device, and collecting a target image of a target object through the light field imaging device; A heat source device is placed in front of the light field imaging device to build a turbulence simulation scene, and the light field imaging device is optimized, and a turbulence image of a target object is collected in the turbulence simulation scene by the optimized light field imaging device, and wavefront information is collected by the optimized light field imaging device; constructing an initial turbulence image reconstruction network based on a self-attention mechanism, training the initial turbulence image reconstruction network through the target image, the turbulence image and the wavefront information to obtain a turbulence image reconstruction network, constructing a total loss function, optimizing the turbulence image reconstruction network through the total loss function to obtain an optimized turbulence image reconstruction network; Reconstructing the turbulence image to be reconstructed through the optimized turbulence image reconstruction network to obtain a reconstructed target image; The initial turbulence image reconstruction network is constructed based on the self-attention mechanism, specifically: The initial turbulence image reconstruction network is constructed through a refinement block, n encoding blocks and n decoding blocks, including: At the input end, connect n encoding blocks one by one, connect the first encoding block to the input port, and connect the last encoding block to the input of the refinement block; At the output end, the n decoding blocks are connected one by one, and the output of the refinement block is connected to the first decoding block, and the last decoding block is connected to the output port; Connect the output of the first encoding block to the input of the last decoding block, and connect the output of the second encoding block to the input of the n-1th decoding block, and so on, to construct the initial turbulence image reconstruction network; The initial turbulence image reconstruction network is trained by using the target image, the turbulence image and the wavefront information to obtain the turbulence image reconstruction network, specifically: The target image, the turbulence image and the wavefront information are feature extracted by the first coding block, and the extracted feature maps are respectively input into the second coding block and the last decoding block; the second coding block to the n-1th coding block all perform feature extraction on the feature map extracted by the previous coding block, and the corresponding extracted feature maps are respectively input into the corresponding next coding block and the corresponding decoding block; the last coding block performs feature extraction on the feature map extracted by the n-1th coding block, and the extracted feature maps are respectively input into the refinement block and the first decoding block; Performing a texture refinement operation on the feature map extracted by the last encoding block through the refinement block, and inputting the refined feature map into the first decoding block; The first decoding block performs a feature recovery operation on the feature map extracted by the last encoding block and the feature map refined by the refinement block, and the restored feature map is input into the second decoding block. The second decoding block to the last decoding block all perform a feature recovery operation on the feature map restored by the previous decoding block and the feature map extracted by the corresponding encoding block. The last decoding block obtains an initial reconstructed target image after the feature recovery operation, and outputs the initial reconstructed target image to the output port, completing the training, and obtaining a turbulent image reconstruction network; The encoding blocks each include an encoding convolution layer, an encoding self-attention mechanism module, an encoding pooling layer, and an encoding forward feedback layer; The feature extraction of the target image, the turbulence image and the wavefront information is performed by the first coding block, specifically: The wavefront information is calculated through the coding convolution layer to obtain a degradation operator, the turbulence features of the turbulence image degraded from the target image are extracted according to the degradation operator, the turbulence image is reconstructed according to the turbulence features to obtain a local reconstructed feature map, the local reconstructed feature map is weighted summed through the coding self-attention mechanism module to obtain a coding weighted feature map, the coding weighted feature map is downsampled through the coding pooling layer, and the downsampled feature map is residually connected through the coding forward feedback layer; The decoding blocks each include a decoding convolutional layer, a decoding self-attention mechanism module, a decoding pooling layer, and a decoding forward feedback layer; The performing of a feature recovery operation on the feature map extracted by the last coding block and the feature map refined by the refinement block by the first decoding block is specifically as follows: The feature map downsampled by the last coding block and the feature map refined by the refinement block are fused through the decoding convolution layer to obtain a decoding fused feature map, the decoding fused feature map is weighted summed through the decoding self-attention mechanism module to obtain a decoding weighted feature map, the decoding weighted feature map is upsampled through the decoding pooling layer, and the upsampled feature map is residually connected through the decoding forward feedback layer; The total loss function is constructed by a reconstruction loss function, a turbulence estimation loss function and a scaling parameter; The turbulence image reconstruction network is optimized by the total loss function to obtain an optimized turbulence image reconstruction network, specifically: The initial reconstructed target image and the target image are calculated by using the reconstruction loss function expression to obtain the reconstruction loss. The reconstruction loss function expression is: Among them, L0 is the reconstruction loss function, is the initial reconstructed target image, I is the target image; The target image is subjected to turbulence degradation by the degradation operator to obtain an estimated turbulence image, and the estimated turbulence image and the turbulence image are calculated by the turbulence estimation loss function expression to obtain a turbulence estimation loss. The turbulence estimation loss function expression is: Among them, L1 is the turbulence estimation loss function, To estimate the turbulence image, is a turbulence image; The reconstruction loss and the turbulence estimation loss are calculated by using a total loss function expression to obtain a total loss, and the total loss function expression is: L=L0+αL1, Among them, L is the total loss function and α is the scaling parameter; The turbulence image reconstruction network is iteratively optimized through the total loss to obtain an optimized turbulence image reconstruction network.

2. The method for eliminating atmospheric turbulence in an image according to claim 1, characterized in that: The light field imaging device comprises a primitive nematic phase liquid crystal composite microlens array; Before the step of building the light field imaging device, the following steps are also included: By applying different voltages to the original nematic liquid crystal composite microlens array, the arrangement of liquid crystal molecules in the original nematic liquid crystal composite microlens array is changed to obtain the focal length corresponding to each voltage, and an imaging relationship model is constructed according to all voltages and their corresponding focal lengths.

3. The method for eliminating atmospheric turbulence in an image according to claim 2, characterized in that: The light field imaging device is constructed as follows: The original nematic liquid crystal composite microlens array is subjected to voltage transformation calculation through the imaging relationship model to obtain a first voltage, the first voltage is applied to the original nematic liquid crystal composite microlens array to obtain a nematic liquid crystal composite microlens array, and a light field imaging device is constructed through the nematic liquid crystal composite microlens array and basic imaging components.

4. The method for eliminating atmospheric turbulence in an image according to claim 3, characterized in that: The optimization of the light field imaging device is specifically as follows: The nematic liquid crystal composite microlens array is subjected to voltage transformation calculation through the imaging relationship model to obtain a second voltage, the second voltage is applied to the nematic liquid crystal composite microlens array to obtain an optimized nematic liquid crystal composite microlens array, the nematic liquid crystal composite microlens array of the light field imaging device is replaced with the optimized nematic liquid crystal composite microlens array to obtain an optimized light field imaging device.

5. The method for eliminating atmospheric turbulence in an image according to claim 4, characterized in that: The optimized light field imaging device comprises a wavefront sensor, which is composed of the optimized nematic liquid crystal composite microlens array and a photodetector CCD arranged in sequence along an imaging path, wherein the optimized nematic liquid crystal composite microlens array is composed of a plurality of optimized nematic liquid crystal composite microlenses; The wavefront information is collected by the optimized light field imaging device, specifically: When collecting the turbulence image, an incident light wave is irradiated onto the plurality of optimized nematic phase liquid crystal composite microlenses, and the incident light wave is focused into a plurality of corresponding light spots by the plurality of optimized nematic phase liquid crystal composite microlenses respectively, and the plurality of corresponding light spots are irradiated onto a focal plane; The photoelectric detector CCD collects multiple light spots on the focal plane, and calculates the center position of each light spot. The center position of each light spot is calculated by a wavefront restoration algorithm to obtain wavefront information.

6. The method for eliminating atmospheric turbulence in an image according to claim 5, characterized in that: The center position of each light spot is calculated by the wavefront restoration algorithm to obtain the wavefront information, which is specifically: Calculating the center position of each light spot with the corresponding theoretical center position collected in advance to obtain a plurality of corresponding offsets, wherein the offsets include an x-axis offset component and a y-axis offset component; The offset components of multiple x-axes are calculated by the wavefront slope expression to obtain multiple corresponding sub-wavefront slopes of the x-axes. The offset components of multiple y-axes are calculated by the wavefront slope expression to obtain multiple corresponding sub-wavefront slopes of the y-axes. The wavefront slope expression is: S x =Δx / f, S y =Δy / f, Among them, S x is the sub-wavefront slope of the x-axis, S y is the sub-wavefront slope of the x-axis, Δx is the offset component of the x-axis, Δy is the offset component of the y-axis, and f is the focal length of the optimized nematic liquid crystal composite microlens array; The wavefront information is obtained by integrating the plurality of sub-wavefront slopes through an integral calculation expression, and the integral calculation expression is: Φ(x,y)=∫S x (x,y)dx+∫S y (x,y)dy, Among them, Φ(x,y) is the wavefront information.

7. An image atmospheric turbulence elimination system, characterized in that: include: A data acquisition unit is used to build a light field imaging device, collect a target image of a target object through the light field imaging device, place a heat source device in front of the light field imaging device to build a turbulence simulation scene, and optimize the light field imaging device, collect a turbulence image of the target object in the turbulence simulation scene through the optimized light field imaging device, and collect wavefront information through the optimized light field imaging device; Constructing a network unit, which is used to construct an initial turbulence image reconstruction network based on a self-attention mechanism, training the initial turbulence image reconstruction network through the target image, the turbulence image and the wavefront information to obtain a turbulence image reconstruction network, constructing a total loss function, and optimizing the turbulence image reconstruction network through the total loss function to obtain an optimized turbulence image reconstruction network; A reconstruction unit, used for reconstructing the turbulence image to be reconstructed through the optimized turbulence image reconstruction network to obtain a reconstructed target image; The initial turbulence image reconstruction network is constructed based on the self-attention mechanism, specifically: The initial turbulence image reconstruction network is constructed through a refinement block, n encoding blocks and n decoding blocks, including: At the input end, connect n encoding blocks one by one, connect the first encoding block to the input port, and connect the last encoding block to the input of the refinement block; At the output end, the n decoding blocks are connected one by one, and the output of the refinement block is connected to the first decoding block, and the last decoding block is connected to the output port; Connect the output of the first encoding block to the input of the last decoding block, and connect the output of the second encoding block to the input of the n-1th decoding block, and so on, to construct the initial turbulence image reconstruction network; The initial turbulence image reconstruction network is trained by using the target image, the turbulence image and the wavefront information to obtain the turbulence image reconstruction network, specifically: The target image, the turbulence image and the wavefront information are feature extracted by the first coding block, and the extracted feature maps are respectively input into the second coding block and the last decoding block; the second coding block to the n-1th coding block all perform feature extraction on the feature map extracted by the previous coding block, and the corresponding extracted feature maps are respectively input into the corresponding next coding block and the corresponding decoding block; the last coding block performs feature extraction on the feature map extracted by the n-1th coding block, and the extracted feature maps are respectively input into the refinement block and the first decoding block; Performing a texture refinement operation on the feature map extracted by the last encoding block through the refinement block, and inputting the refined feature map into the first decoding block; The first decoding block performs a feature recovery operation on the feature map extracted by the last encoding block and the feature map refined by the refinement block, and the restored feature map is input into the second decoding block. The second decoding block to the last decoding block all perform a feature recovery operation on the feature map restored by the previous decoding block and the feature map extracted by the corresponding encoding block. The last decoding block obtains an initial reconstructed target image after the feature recovery operation, and outputs the initial reconstructed target image to the output port, completing the training, and obtaining a turbulent image reconstruction network; The encoding blocks each include an encoding convolution layer, an encoding self-attention mechanism module, an encoding pooling layer, and an encoding forward feedback layer; The feature extraction of the target image, the turbulence image and the wavefront information is performed by the first coding block, specifically: The wavefront information is calculated through the coding convolution layer to obtain a degradation operator, the turbulence features of the turbulence image degraded from the target image are extracted according to the degradation operator, the turbulence image is reconstructed according to the turbulence features to obtain a local reconstructed feature map, the local reconstructed feature map is weighted summed through the coding self-attention mechanism module to obtain a coding weighted feature map, the coding weighted feature map is downsampled through the coding pooling layer, and the downsampled feature map is residually connected through the coding forward feedback layer; The decoding blocks each include a decoding convolutional layer, a decoding self-attention mechanism module, a decoding pooling layer, and a decoding forward feedback layer; The performing of a feature recovery operation on the feature map extracted by the last coding block and the feature map refined by the refinement block by the first decoding block is specifically as follows: The feature map downsampled by the last coding block and the feature map refined by the refinement block are fused through the decoding convolution layer to obtain a decoding fused feature map, the decoding fused feature map is weighted summed through the decoding self-attention mechanism module to obtain a decoding weighted feature map, the decoding weighted feature map is upsampled through the decoding pooling layer, and the upsampled feature map is residually connected through the decoding forward feedback layer; In the network construction unit, the total loss function is constructed by the reconstruction loss function, the turbulence estimation loss function and the scaling parameter; The turbulence image reconstruction network is optimized by the total loss function to obtain an optimized turbulence image reconstruction network, specifically: The initial reconstructed target image and the target image are calculated by using the reconstruction loss function expression to obtain the reconstruction loss. The reconstruction loss function expression is: Among them, L0 is the reconstruction loss function, is the initial reconstructed target image, I is the target image; The target image is subjected to turbulence degradation by the degradation operator to obtain an estimated turbulence image, and the estimated turbulence image and the turbulence image are calculated by the turbulence estimation loss function expression to obtain a turbulence estimation loss. The turbulence estimation loss function expression is: Among them, L1 is the turbulence estimation loss function, To estimate the turbulence image, is a turbulence image; The reconstruction loss and the turbulence estimation loss are calculated by using a total loss function expression to obtain a total loss, and the total loss function expression is: L=L0+αL1, Among them, L is the total loss function and α is the scaling parameter; The turbulence image reconstruction network is iteratively optimized through the total loss to obtain an optimized turbulence image reconstruction network.

Images