Infrared turbulence degradation image simulation method based on unreal engine

Abstract

This invention proposes a method for simulating infrared turbulence degradation images based on Unreal Engine. The steps are as follows: constructing a 3D mountain simulation scene based on Unreal Engine; generating a blurred kernel texture image based on a calculated 2D point spread function; rendering the 3D mountain scene using Unreal Engine, convolving the rendered result with the blurred kernel texture image, and obtaining the simulated infrared turbulence degradation image through the convolution result. This invention achieves real-time turbulence blurring of the mountain scene by performing parallel convolution on each frame of the rendered 3D mountain scene with the blurred kernel texture image generated based on the 2D point spread function. The obtained real-time turbulence blurring effect preserves the details of the terrain well. It also considers the influence of the spatial frequency response information of the imaging system itself and the attenuation effect of atmospheric turbulence on image detail transmission, effectively improving simulation accuracy. Furthermore, parallel convolution reduces the complexity of generating the real-time turbulence blurring effect of the mountain scene, exhibiting good real-time performance.

Description

Technical Field

[0001] This invention belongs to the field of image simulation technology and relates to an infrared turbulence degradation image simulation method based on Unreal Engine, which can be applied to aerospace, security monitoring, meteorological monitoring and other fields. Background Technology

[0002] Infrared imaging technology boasts wide applicability and powerful functionality, characterized by its versatility, strong anti-interference capabilities, and high environmental adaptability. However, acquiring realistic turbulence images requires specific environmental conditions, necessitating not only high-precision measurement equipment but also overcoming environmental limitations, thus incurring high costs. Infrared imaging simulation technology involves simulating the infrared radiation distribution of objects in terms of time, space, spectrum, and radiance. This technology can generate infrared images that are difficult to obtain under constrained conditions. Images are the most commonly used information carrier in infrared imaging systems, and high-quality, high-resolution images can better convey target information. However, atmospheric turbulence can significantly degrade the imaging quality of optical systems. The non-uniformity of atmospheric density and temperature leads to changes in atmospheric refractive index. Spatially, the farther away from the target and the longer the exposure time, the more severe the atmospheric disturbance, resulting in a blurred image. Temporally, different regions of the image will experience random jitter over time. Therefore, atmospheric turbulence can cause blurring, noise, phase distortion, and content loss in images acquired by infrared imaging systems.

[0003] Infrared turbulence degradation image simulation uses a constructed atmospheric turbulence physical model and combines it with the transmission characteristics of an infrared imaging system to simulate the random interference effect of atmospheric turbulence on infrared signal transmission, thereby generating degradation images that closely resemble real observation conditions.

[0004] Existing typical infrared turbulence degradation image simulations are based on the OGRE engine. This method first constructs a 3D terrain scene containing the infrared target and imaging system using the OGRE engine. A point spread function is generated using the atmospheric modulation transfer function (AMF), and a blur kernel is obtained after sampling and normalization. The blur kernel is then converted into a texture image format and convolved with the rendering result of the simulation scene. This operation is performed in a CPU serial computing mode. Finally, a simulated image with turbulence degradation effects is output through the simulation imaging system. This method utilizes the AMMF to generate a point spread function that conforms to the physical laws of turbulence. The degradation image simulated using this point spread function has higher simulation accuracy than those simulated by traditional methods. Then, the point spread function is sampled and normalized into a blur kernel texture image, which is then rendered in real-time with the constructed 3D scene, solving the problem of insufficient real-time performance in traditional static scene simulations. However, in order to ensure the smooth operation of the scene, the terrain details are compressed and simplified to a certain extent during the construction of the 3D terrain scene, resulting in differences from the high-frequency details of the real terrain. In addition, the generated point spread function only considers the atmospheric modulation transfer function and ignores the influence of the imaging system on the imaging results, thus affecting the simulation accuracy. At the same time, the method convolves the texture image and the simulation scene rendering result through the serial computing mode of the CPU, which is relatively complex and time-consuming, affecting the real-time performance of the simulation. Summary of the Invention

[0005] The purpose of this invention is to overcome the defects of the existing technology and propose an infrared turbulence degradation image simulation method based on Unreal Engine to solve the technical problems of poor accuracy and real-time performance of simulation results in the existing technology.

[0006] To achieve the above objectives, the technical solution adopted by the present invention includes the following steps:

[0007] (1) Constructing a 3D mountain simulation scene based on Unreal Engine:

[0008] The Unreal Engine was used to construct a virtual 3D mountain scene, as well as a 3D mountain simulation scene with multiple infrared radiation feature targets and imaging systems.

[0009] (2) Calculate the two-dimensional point spread function:

[0010] Modulation transfer function (MTF) through the imaging system diff With atmospheric turbulence modulation transfer function (MTF) turb Calculate the integrated modulation transfer function (MTF) optics and MTF optics Perform an inverse Fourier transform to obtain the two-dimensional point spread function h in the spatial domain;

[0011] (3) Generating texture images with blur kernels based on two-dimensional point spread functions:

[0012] After radial sampling of the two-dimensional point spread function h, each sampled data is normalized, and the two-dimensional blur kernel H of all sampled data is saved as a texture image in EXR format with the same specifications.

[0013] (4) After rendering the 3D mountain scene using Unreal Engine, the rendering result is convolved with the blurred kernel texture image:

[0014] The 3D mountain scene was rendered multiple times in real time using Unreal Engine, and each frame of the rendered image was convolved in parallel with the blur kernel texture image to obtain the real-time turbulent blur effect of the mountain scene.

[0015] (5) Obtain the simulation results of infrared turbulence degradation images:

[0016] Based on the imaging system, the real-time turbulence blurring effect of mountain scenes is imaged to obtain infrared turbulence degradation images.

[0017] Compared with the prior art, the present invention has the following advantages: real-time turbulence blurring effect in mountain scenes;

[0018] 1. The three-dimensional mountain simulation scene constructed in this invention contains height data from the digital elevation model of the real mountain scene. The generated two-dimensional point spread function contains the spatial frequency response information of the imaging system itself and the attenuation information of atmospheric turbulence on the transmission of image details. By performing parallel convolution between each frame of the rendered three-dimensional mountain scene and the blurred kernel texture image generated based on the two-dimensional point spread function, the real-time turbulence blurring effect of the mountain scene not only preserves the details of the terrain well, but also takes into account the influence of the imaging system on the image, effectively improving the simulation accuracy.

[0019] 2. This invention utilizes the parallel convolution of multiple threads in the GPU parallel architecture of Unreal Engine to perform parallel convolution on pixels and blur kernel texture images in the rendered image. This avoids the drawback of long processing time caused by the complex calculation process of serial convolution using the CPU in the prior art, and has better real-time performance. Attached Figure Description

[0020] Figure 1 This is a flowchart illustrating the implementation of the present invention.

[0021] Figure 2 This is a fuzzy kernel texture map according to an embodiment of the present invention.

[0022] Figure 3 This is a rendering of a three-dimensional mountain scene from the present invention.

[0023] Figure 4This is a simulation result of the infrared turbulence degradation image of the present invention. Detailed Implementation

[0024] The present invention will now be described in further detail with reference to the accompanying drawings and specific embodiments.

[0025] Reference Figure 1 The present invention includes the following steps:

[0026] Step 1) Use Unreal Engine to build a detailed 3D mountain scene;

[0027] Based on a digital elevation model acquired through satellite remote sensing, the elevation data of real mountains in the model was extracted using the GDAL terrain processing software library and stored in a 16-bit grayscale height map. The grayscale height map uses shades of gray to represent the undulations of the mountain terrain. Virtual geometry technology was applied to analyze the grayscale height map in the Unreal Engine terrain editor system. Based on the changes in the elevation data in the grayscale height map, the number of triangles was reasonably determined. The number of triangles was increased in areas with high grayscale values ​​and decreased in areas with low grayscale values. The scene was built by combining numerous triangles to ensure that the 3D mountain scene maintained geometric consistency with the real terrain.

[0028] Multiple infrared radiation feature targets are added to this simulation scenario, based on the temperature T of the target surface area. tem The thermal radiation zones are divided by the spectral reflectance ρ(λ), and then T tem The values ​​of ρ(λ) and ρ(λ) are stored in the target UV texture map respectively. Figure 3 The green and blue channels are used in the data; the infrared radiance L(λ) of the infrared target is calculated by dynamically sampling the UV texture map using the Unreal Engine material system; finally, the radiation distribution of the infrared target is presented using UV mapping technology to obtain the infrared radiation characteristic target, where the formula for L(λ) is:

[0029] L(λ)=ρ(λ)·T tem

[0030] λ represents the infrared wavelength of the simulated scene.

[0031] Step 2) Calculate the two-dimensional point spread function:

[0032] Modulation transfer function (MTF) through the imaging system diff With atmospheric turbulence modulation transfer function (MTF) turb The combined calculation yields the integrated modulation transfer function (MTF). optics This function simultaneously reflects the spatial frequency response of the imaging system itself and the attenuation effect of atmospheric turbulence on image detail transmission. Imaging system modulation transfer function (MTF) diffThis describes the ability of an imaging system to maintain contrast and detail at different spatial frequencies during the process from object to image; while the atmospheric turbulence modulation transfer function (MTF) turb This quantitatively characterizes the phase disturbances and high-frequency attenuation effects caused by atmospheric turbulence, which typically lead to image blurring and loss of detail. (Synthetic Modulation Transfer Function MTF) optics The calculation formula is:

[0033] MTF optics =MTF diff ×MTF turb

[0034]

[0035] v nor =f x / f oco

[0036] f oco =D / λ

[0037] Among them, v nor Let represent the normalized frequency of the imaging system, exp{·} represent the exponential operation with base e, D represent the pupil diameter of the imaging system, r0 represent the atmospheric coherence length, α represent the long and short exposure coefficients, and R represent the distance between the imaging system and the infrared target. f represents the atmospheric refractive index structure constant. x f represents the spatial angular frequency of the imaging system. oco This indicates the cutoff frequency of the imaging system.

[0038] The obtained MTF optics Perform inverse Fourier transform, where MTF optics The synthesized modulation transfer function (MTF) in the u-th row and v-th column of the mid-frequency domain optics The formula for performing the inverse Fourier transform on the result is (u,v):

[0039]

[0040]

[0041] Where h(x,y) represents the two-dimensional point spread function in the x-th row and y-th column of a spatial domain of size N×N, ∫{·} represents the integration operation, and du and dv represent the accumulation operation.

[0042] Step 3) Generate a texture image with a blur kernel based on a two-dimensional point spread function:

[0043] Using the center point of the numerical peak of the two-dimensional point spread function h in the spatial domain, perform m radial samplings outward along the positive and negative directions of x and y respectively, and normalize all the sampled data to obtain a two-dimensional blur kernel H with a size of (2m+1)×(2m+1). Then save H as a texture image in EXR format with the same specifications as H.

[0044] The formula for calculating the fuzzy kernel value H[i,j] in the i-th row and j-th column is:

[0045]

[0046] Where Δx and Δy represent the sampling step size along the x and y directions, respectively, and (i,j)∈[0,2m+1]. h(iΔx,jΔy) represents the point spread function value of the i-th Δx row and j-th Δy column.

[0047] In this embodiment, m=4, and the resulting two-dimensional blur kernel H has a size of 9×9. The corresponding blur kernel texture image is as follows. Figure 2 As shown in the figure, it can be clearly observed that the value of the blurred kernel texture image reaches its highest in the central region, and the value gradually decreases as it spreads outwards, showing a distribution pattern of high value in the center and low value around the edges.

[0048] Step 4) After rendering the 3D mountain scene using Unreal Engine, convolve the rendering result with the blur kernel texture image:

[0049] Unreal Engine at resolution (P) row ,P col In the rendering window, the geometric data in the 3D scene is rasterized into 2D pixels, and then pixel shaders and fragment shaders are used to color the pixels to achieve real-time scene rendering. The scene rendering frame rate is T, which is affected by hardware performance and rendering window resolution. Each frame rendering result is convolved with the blur kernel texture image to simulate the blurring effect of turbulence. This convolution operation is based on the parallel architecture of the GPU, which assigns each pixel in the rendered image to a parallel thread in the GPU parallel architecture for convolution. Multiple parallel threads can be performed simultaneously to achieve synchronous convolution of multiple pixels, thereby significantly reducing computational latency, accelerating the overall rendering process, and ensuring the efficiency and accuracy of real-time image processing. The convolution formula for the pixel I(p,q,t) in the p-th row and q-th column of the t-th frame rendered image and the blur kernel value H[i,j] in the i-th row and j-th column is:

[0050]

[0051] Where (p,q)∈(P) row ,P col), t∈[1,T], S(pi,qj,t) represents the value in the (pi)th row and (qj)th column of the rendering result of the tth frame.

[0052] This invention utilizes Unreal Engine to render a 3D mountain scene, and the result is as follows: Figure 3 As shown in the image, the mountain outlines, grasslands, and trees in the mountain scene built with Unreal Engine are clearly visible, and the objects in the scene have detailed and rich shapes.

[0053] Step 5) Obtain the simulation results of infrared turbulence degradation images:

[0054] The pixel values ​​of the real-time turbulence blur effect of the mountain scene observed within the viewport of the imaging system are saved in a format such as... Figure 4 The image shown is an infrared image of turbulence degradation. (Compared to...) Figure 3 In contrast, after being affected by the degradation effect of turbulence, the infrared turbulence degradation image shows obvious blurring as a whole. The clarity of the mountain outline, grassland and trees in the mountain scene is significantly reduced, and the shape details of the target in the scene are lost. This change is consistent with the effect of turbulence on infrared imaging.

Claims

1. A method for simulating infrared turbulence degradation images based on Unreal Engine, characterized in that... Includes the following steps: (1) Constructing a 3D mountain simulation scene based on Unreal Engine: Based on Unreal Engine, a virtual 3D mountain scene is constructed, as well as a 3D mountain simulation scene with multiple infrared radiation feature targets and an imaging system added. The multiple infrared radiation feature targets are multiple infrared radiation feature targets containing temperature, spectral reflectance and infrared radiation brightness information of different target surface areas. (2) Calculate the two-dimensional point spread function: Modulation transfer function through imaging system With atmospheric turbulence modulation transfer function Calculate the integrated modulation transfer function and to Performing an inverse Fourier transform yields the two-dimensional point spread function h in the spatial domain, where... The calculation formula is: ； ； ； ； ； ； in, Represents the normalized frequency of the imaging system. Indicated by Exponentiation with base 0. Indicates the pupil diameter of the imaging system. Indicates the atmospheric coherence length. This represents the infrared wavelength in the simulation scenario. Indicates the exposure factor for long and short exposures. Indicates the distance between the imaging system and the infrared target. This represents the atmospheric refractive index structure constant. Represents the spatial angular frequency of the imaging system. Indicates the cutoff frequency of the imaging system; (3) Generating texture images with blur kernels based on two-dimensional point spread functions: Two-dimensional point spread function After radial sampling, each sampled data point is normalized, and the normalized two-dimensional fuzzy kernel of all sampled data is then used. Save the texture image as an EXR format image with the same specifications; (4) After rendering the 3D mountain scene using Unreal Engine, convolve the rendering result with the blurred kernel texture image: The 3D mountain scene was rendered multiple times in real time using Unreal Engine, and each frame of the rendered image was convolved in parallel with the blur kernel texture image to obtain the real-time turbulent blur effect of the mountain scene. (5) Obtain the simulation results of infrared turbulence degradation images: Based on the imaging system, the real-time turbulence blurring effect of mountain scenes is imaged to obtain infrared turbulence degradation images.

2. The method according to claim 1, characterized in that, The three-dimensional mountain scene described in step (1) is constructed as follows: The height data in the digital elevation model of the real mountain scene is converted into a grayscale height map. Then, using the Unreal Engine terrain system, multiple virtual triangular patches are created based on the height data in the grayscale height map. Finally, the multiple triangular patches are combined to form a three-dimensional mountain scene with the same geometric dimensions as the real mountain scene.

3. The method according to claim 1, characterized in that, The step (2) described above Perform inverse Fourier transform, where... In the mid-frequency domain, the first Line number Synthetic Modulation Transfer Function of the Column The formula for performing an inverse Fourier transform on the result is: ； ； in Indicates size is The two-dimensional point spread function in the x-th row and y-th column of the spatial domain. This indicates the integration operation. and This indicates an accumulation operation.

4. The method according to claim 1, characterized in that, The steps described in step (3) for radial sampling of the two-dimensional point spread function h and normalization of each sampled data are as follows: The numerical peak center point of the two-dimensional point spread function h in the spatial domain is along... , The positive and negative directions are respectively directed outwards. After radial sampling and normalization of each sampled data, a size of [size missing] is obtained. Two-dimensional fuzzy kernel Among them, for the first Line number The formula for normalizing the sampled data is: ； in, This represents the fuzzy kernel after normalizing the sampled data in the i-th row and j-th column. , They represent along , Sampling step size in direction, , , Indicates the first OK The point spread function value of the column.

5. The method according to claim 1, characterized in that, The real-time rendering of the 3D mountain scene based on Unreal Engine described in step (4) is implemented as follows: A 3D mountain scene was created using pixel shaders and fragment shaders in Unreal Engine. The second real-time rendering yielded the following resolutions for the rows and columns in the rendering window: , of Frame rendering image, where, .

6. The method according to claim 5, characterized in that, The parallel convolution of the rendering result with the blur kernel texture image frame by frame, as described in step (4), is specifically achieved by using multiple threads in parallel on the GPU parallel architecture of Unreal Engine to simultaneously convolve the pixels in the rendered image with the blur kernel texture image. The first frame of the rendered image Line number Column pixels With the Line number Column fuzzy kernel The convolution formula is: ； in, , , Indicates the first The first frame in the rendering result Line number The value of the column.

7. The method according to claim 1, characterized in that, The method for imaging the real-time turbulence blur effect of the mountain scene described in step (5) is to save the value of each pixel in the real-time turbulence blur effect of the mountain scene observed in the viewport of the imaging system, and obtain an infrared turbulence degradation image with the same resolution as the imaging system.

Images