Three-dimensional atmospheric turbulence image simulation method based on deep perception and physical model driving

Abstract

This invention discloses a three-dimensional atmospheric turbulence image simulation method based on depth perception and physical model driving. First, a clear image and its corresponding relative depth map are obtained. Then, the depth map is mapped to the physical propagation distance at the spatial location, which is then used to generate a depth modulation map. Next, a blurred image is generated by spatial variation fuzzy interpolation of the clear image using the depth modulation map. At the same time, the pixel-level displacement vector based on phase derivation is scaled using the depth modulation map to obtain depth-perceived geometric distortion. Finally, the geometric distortion is applied to the blurred image to generate the final turbulence image.

Description

Technical Field

[0001] This invention belongs to the field of image processing technology, and more specifically, relates to a three-dimensional atmospheric turbulence image simulation method based on depth perception and physical model driven. Background Technology

[0002] Long-range imaging systems are often subject to severe degradation caused by atmospheric turbulence, a physical phenomenon primarily driven by refractive index fluctuations along the optical path. In the image domain, atmospheric turbulence manifests as a complex interplay of spatially varying blur and non-rigid geometric deformation / tilt.

[0003] Currently, data-driven atmospheric turbulence mitigation (image restoration) methods heavily rely on accurate simulation techniques to generate paired training data for deep learning models. However, physical wave propagation models are computationally expensive. While existing 3D wave propagation models and step-by-step simulation methods are physically rigorous, their computational costs are extremely high, failing to meet the demands of generating large-scale datasets. Image domain simulators lack depth perception: To improve efficiency, existing techniques often employ image domain simulators, such as phase-to-space (P2S) transformation methods. This method efficiently maps Zernike polynomial coefficients to spatially varying point spread functions (PSFs) and dense displacement fields. However, standard P2S simulators rely on idealized isoplanatic or flat-field assumptions. They typically impose a uniform, homogeneous turbulence intensity across the entire image space.

[0004] Existing technologies cause "domain gaps" such as Figure 1 As shown, the standard simulator's application of uniform turbulence intensity deviates from the optical physics of the real world. In real physical environments, turbulence intensity accumulates dynamically along the line-of-sight path, exhibiting inherent depth-dependent characteristics (i.e., foreground objects are less affected, while distant objects are severely distorted). As shown in the left figure, atmospheric turbulence exhibits weaker intensity at near points and stronger intensity at far points, in relation to scene depth. Existing techniques such as P2S ignore this path accumulation effect. As shown in the right figure, P2S uses uniform degradation at both near and far points, resulting in synthesized degradation images that are independent of depth. Consequently, restoration models trained on these data face a severe "sim-to-real" domain gap. Summary of the Invention

[0005] The purpose of this invention is to overcome the shortcomings of the prior art and provide a three-dimensional atmospheric turbulence image simulation method based on depth perception and physical model driving, so as to generate physically consistent and scene depth-related degraded images, thereby reducing the inter-domain gap between simulation data and real-world data.

[0006] To achieve the above-mentioned objectives, this invention provides a three-dimensional atmospheric turbulence image simulation method based on depth perception and physical model-driven approach, characterized by comprising the following steps:

[0007] (1) Obtain a clear image of a certain scene and its corresponding relative depth map , Indicates the spatial location of a pixel;

[0008] (2) Depth map Mapped to spatial location Physical propagation distance at the location :

[0009] ;

[0010] in, The total optical path length, This represents the baseline offset of the scene.

[0011] (3) Construct a spatial change modulation map for depth perception;

[0012] Based on Kolmogorov's turbulence theory, using physical propagation distance... Distance corresponding to the maximum cumulative turbulence intensity Constructing spatially varying depth modulation maps :

[0013] ;

[0014] (4) Synthesize spatially blurred images with depth perception;

[0015] Through depth modulation map right Perform spatial variation fuzzy interpolation to generate a blurred image. :

[0016] ;

[0017] in, Represents the spatial variation point spread function generated by the Zernike polynomial basis;

[0018] (5) Obtain geometric distortions for depth perception;

[0019] Through depth modulation map Scale the pixel-level displacement vector based on phase derivation to obtain depth-aware geometric distortion. :

[0020] ;

[0021] in, This represents a frequency-domain phase distortion screen controlled by Kolmogorov's turbulence statistics. This is the inverse Fourier transform operation;

[0022] (6) Geometric distortion of depth perception Applied to blurred images This generates the final turbulence image. :

[0023] ;

[0024] The objective of this invention is achieved as follows:

[0025] This invention is based on a three-dimensional atmospheric turbulence image simulation method driven by depth perception and physical models. First, a clear image and its corresponding relative depth map are obtained. Then, the depth map is mapped to the physical propagation distance at the spatial location, which is then used to generate a depth modulation map. Next, a blurred image is generated by spatial variation fuzzy interpolation of the clear image using the depth modulation map. At the same time, the pixel-level displacement vector based on phase derivation is scaled using the depth modulation map to obtain depth-perceived geometric distortion. Finally, the geometric distortion is applied to the blurred image to generate the final turbulence image.

[0026] Meanwhile, the three-dimensional atmospheric turbulence image simulation method based on depth perception and physical model driven by the present invention also has the following beneficial effects:

[0027] (1) It breaks through the limitation of global uniform degradation based on the "isohalo hypothesis" or "flat field hypothesis" in the existing atmospheric turbulence image simulation technology. By mapping the relative depth to the physical propagation distance and constructing a spatially varying depth modulation map, it accurately reflects the physical effect of the dynamic accumulation of turbulence intensity with the optical path distance in the real scene. This technology effectively avoids the problem that the synthesized degradation image is not related to depth and significantly reduces the interdomain gap between simulation data and real-world data.

[0028] (2) It provides excellent structural supervision signals for deep learning models, enabling the trained models to effectively straighten distorted edges and recover more realistic texture details when dealing with complex real degradation that varies with depth; its quantitative evaluation and qualitative visual effects are significantly better than models trained based on synthetic data from standard P2S simulators.

[0029] (3) The present invention overcomes the shortcomings of the existing three-dimensional wave propagation model, which has extremely high computational cost. The depth perception synthesis technology of the present invention can efficiently meet the needs of generating large-scale datasets. The simulation calculation and data generation processes can be deployed on computing devices equipped with high-performance graphics processors for parallel acceleration, thereby flexibly and cost-effectively generating a large number of paired turbulence training and test data of different intensity levels. Attached Figure Description

[0030] Figure 1 This is a diagram illustrating the "inter-domain gap" caused by existing technology; Figure 2 This is a flowchart of the three-dimensional atmospheric turbulence image simulation method based on depth perception and physical model driven by the present invention; Figure 3 It consists of a clear image taken and its corresponding relative depth map; Figure 4 It is a synthetic depth-sensing image of spatially varying blurred features; Figure 5 It is a turbulence image; Figure 6 These are comparison images of depth perception simulations. Detailed Implementation

[0031] The specific embodiments of the present invention will now be described with reference to the accompanying drawings to enable those skilled in the art to better understand the invention. It should be particularly noted that in the following description, detailed descriptions of known functions and designs that might obscure the main content of the invention will be omitted here.

[0032] Example

[0033] In this embodiment, as Figure 2 As shown, the present invention provides a three-dimensional atmospheric turbulence image simulation method based on depth perception and physical model driven by the following steps:

[0034] (1) Obtain clear images of outdoor scenes by taking pictures with a camera. ,like Figure 3 The left image shows the image and its corresponding relative depth map. ,like Figure 3 As shown in the right figure. Indicates the spatial location of a pixel;

[0035] (2) Mapping of relative depth to physical propagation distance;

[0036] Depth map Mapped to spatial location Physical propagation distance at the location :

[0037] ;

[0038] in, The total optical path length, This represents the baseline offset of the scene.

[0039] In this embodiment, The value is set to 0.48, introducing... The aim is to flexibly calibrate the dynamic range of the physical distance to ensure that the synthesized degradation accurately reflects the true path-cumulative turbulence intensity;

[0040] (3) Construct a spatial change modulation map for depth perception;

[0041] Based on Kolmogorov's turbulence theory, using physical propagation distance... Distance corresponding to the maximum cumulative turbulence intensity Constructing spatially varying depth modulation maps This replaces the non-physical uniform mapping in standard simulators, making far points more affected by turbulence degradation than near points.

[0042] ;

[0043] (4) Synthesize spatially blurred images with depth perception;

[0044] Through depth modulation map right Perform spatial variation blur interpolation to generate an image where the degree of blur varies with distance. ,like Figure 4 As shown;

[0045] ;

[0046] in, Represents the spatial variation point spread function generated by the Zernike polynomial basis;

[0047] (5) Obtain geometric distortions for depth perception;

[0048] Through depth modulation map Scale the pixel-level displacement vector based on phase derivation to obtain depth-aware geometric distortion. :

[0049] ;

[0050] in, This represents a frequency-domain phase distortion screen controlled by Kolmogorov's turbulence statistics. This is the inverse Fourier transform operation, used to map the phase representation into a spatially dense displacement field;

[0051] (6) Geometric distortion of depth perception Applied to blurred images This generates the final turbulence image. ,like Figure 5 As shown;

[0052] ;

[0053] To verify the effectiveness of the depth-sensing simulation simulator proposed in this invention in eliminating the gap between the "sim-to-real" domain, this embodiment compares it with the existing unified phase space (standard P2S) simulator.

[0054] In the comparative experiments, two parallel datasets were constructed using the same physical parameters, and the same baseline image restoration network (e.g., the TurbNet model) was trained from scratch on both datasets. Subsequently, the two trained models were evaluated for cross-domain generalization on a real, unlabeled atmospheric turbulence dataset (RLR-AT dataset).

[0055] Quantitative evaluation results: On the no-reference image quality evaluation metric, the model trained using the synthetic data of this invention significantly outperforms the model trained using synthetic data from a standard P2S simulator. Specific test data are shown in the table below:

[0056] P2S 7.397 46.554 Ours 6.980 51.996

[0057] Qualitative visual effect comparison: In the actual visual effect comparison, the input turbulence image contained severe blurring and distortion degradation, such as... Figure 6 As shown in Figure a, the traditional P2S model struggles to handle complex depth-dependent real-world degradation, and the restored results still retain severe geometric distortions, such as... Figure 6 As shown in b, the depth-aware synthesis technology of this invention provides the model with excellent structural supervision signals, enabling the finally trained model to effectively straighten distorted edges and recover more realistic texture details, such as... Figure 6 As shown in c in the figure.

[0058] Although the illustrative specific embodiments of the present invention have been described above to enable those skilled in the art to understand the invention, it should be understood that the invention is not limited to the scope of the specific embodiments. For those skilled in the art, various changes are obvious as long as they are within the spirit and scope of the invention as defined and determined by the appended claims, and all inventions utilizing the concept of the present invention are protected.

Claims

1. A three-dimensional atmospheric turbulence image simulation method based on depth perception and physical model driven by the method, characterized in that, Includes the following steps: (1) Obtain a clear image of a certain scene and its corresponding relative depth map , Indicates the spatial location of a pixel; (2) Depth map Mapped to spatial location Physical propagation distance at the location : ； in, The total optical path length, This represents the baseline offset of the scene. (3) Construct a spatial change modulation map for depth perception; Based on Kolmogorov's turbulence theory, using physical propagation distance... Distance corresponding to the maximum cumulative turbulence intensity Constructing spatially varying depth modulation maps : ； (4) Synthesize spatially blurred images with depth perception; Through depth modulation map right Perform spatial variation fuzzy interpolation to generate a blurred image. : ； in, Represents the spatial variation point spread function generated by the Zernike polynomial basis; (5) Obtain geometric distortions for depth perception; Through depth modulation map Scale the pixel-level displacement vector based on phase derivation to obtain depth-aware geometric distortion. : ； in, This represents a frequency-domain phase distortion screen controlled by Kolmogorov's turbulence statistics. This is the inverse Fourier transform operation; (6) Geometric distortion of depth perception Applied to blurred images This generates the final turbulence image. : 。

Images