Heterogeneous hardware light chasing acceleration method for space-based infrared remote sensing imaging simulation

Abstract

The invention provides a heterogeneous hardware light chasing acceleration method for space-based infrared remote sensing imaging simulation. The method comprises the following steps: step 1, constructing a space-based infrared remote sensing scene database; 2, the scene database is analyzed, configuration of a hardware light tracing pipeline is carried out at a CPU end, and scene loading is completed; 3, at the GPU end, executing a core infrared light tracking calculation task according to the configured hardware light tracking pipeline, and completing scene configuration; and step 4, performing image rendering and processing, and generating space-based infrared remote sensing simulation images under different conditions according to scene loading and scene configuration and actual requirements. A CPU + GPU heterogeneous hardware architecture is adopted for ray tracing acceleration, and the complex instruction processing capacity of the CPU and the high parallel computing capacity of the GPU are organically combined. According to the method, the calculation efficiency is improved by dozens of times to hundreds of times, so that real-time generation of high-quality images is realized, and strict requirements of space-based infrared remote sensing imaging on real-time performance and high efficiency are met.

Description

Technical Field

[0001] The invention relates to a heterogeneous hardware ray tracing acceleration method for space-based infrared remote sensing imaging simulation, and belongs to the technical field of image processing. Background Art

[0002] In recent years, space-based infrared imaging systems have become a research hotspot, mainly used to solve the problem of insufficient imaging capability of space-based visible light detection systems when the target enters the shadow area of ​​the earth. However, the development cycle of space-based infrared imaging cameras is long, the testing conditions are harsh and the cost is high. With the development of computer simulation technology, simulating space environment and imaging equipment through simulation algorithms has become an effective way to avoid harsh experimental conditions. Space-based infrared remote sensing imaging usually needs to process complex geographic information and meteorological scenes. Traditional rasterization technology has limitations in realism and image detail performance, while ray tracing technology based on physical optics principles can more realistically simulate the propagation and reflection of light and generate realistic simulated images. Although ray tracing technology has high-precision imaging capabilities, its computational complexity is high and it is difficult to meet the requirements of real-time simulation. In recent years, with major breakthroughs in algorithms and hardware architectures by leading chip companies such as Intel and NVIDIA, the computational efficiency of ray tracing has been greatly improved, making efficient space-based infrared remote sensing imaging simulation possible. However, a single type of hardware architecture is difficult to meet the requirements of high precision and real-time performance at the same time, while heterogeneous hardware architecture technology combining different types of processors such as CPU and GPU can give full play to their respective computing advantages. The CPU has powerful instruction processing capabilities and is suitable for handling complex logic and control tasks, while the GPU has highly parallel computing capabilities and is suitable for executing a large number of relatively independent and complex ray tracing computing tasks. Through the design of heterogeneous hardware architecture, the computing efficiency of ray tracing can be significantly improved, taking into account the accuracy and real-time requirements of simulation, thus providing a more efficient solution for space-based infrared remote sensing imaging.

[0003] Defects and shortcomings of the existing technology:

[0004] 1. Limited imaging quality: When dealing with complex geographic information and meteorological conditions in space-based infrared remote sensing scenes, the traditional rasterization technology has significantly limited the realism and detail of the image. This method cannot accurately simulate the propagation, reflection and refraction of light in the scene, resulting in the generated infrared remote sensing images being insufficient in terms of high precision and fidelity, making it difficult to meet the high-standard space-based infrared imaging requirements;

[0005] 2. Low computational efficiency: The existing space-based infrared remote sensing imaging simulation system is difficult to generate high-quality images in real time due to its high computational complexity. The slow speed of the computational process affects the actual application effect of the system, especially in scenarios that require fast response and efficient processing, the performance is poor;

[0006] 3. Insufficient use of hardware resources: Current technologies generally rely on CPUs for calculations. Due to the limited parallel processing capabilities of CPUs, the powerful computing capabilities of modern GPUs are not fully utilized. This results in slow calculation speeds and the inability to achieve efficient simulation calculations, thus limiting the performance and real-time nature of the system. Summary of the invention

[0007] The present invention aims to solve the technical problems of limited imaging quality, low computing efficiency and insufficient utilization of hardware resources in the prior art, and further proposes a heterogeneous hardware ray tracing acceleration method for space-based infrared remote sensing imaging simulation.

[0008] The technical solution adopted by the present invention to solve the above problems is: the present invention proposes a heterogeneous hardware ray tracing acceleration method for space-based infrared remote sensing imaging simulation, comprising:

[0009] Step 1: Build a space-based infrared remote sensing scene database;

[0010] Step 2: Parse the scene database, configure the hardware ray tracing pipeline on the CPU side, and complete scene loading;

[0011] Step 3: On the GPU side, perform the core infrared ray tracing calculation tasks according to the configured hardware ray tracing pipeline to complete the scene configuration;

[0012] Step 4: Render and process the image, load the scene and configure the scene, and generate infrared remote sensing images under different conditions according to actual needs.

[0013] Optionally, the steps of building a scene database in step 1 include:

[0014] Step 1.1: Construct a space-based infrared remote sensing scene database, which includes but is not limited to an earth elevation model and a three-dimensional geometric model of cloud top surface height;

[0015] Step 1.2: Use MODTRAN software to calculate the environmental infrared radiation transmission model, which includes but is not limited to the atmospheric radiation model, the direct sunlight model, and the atmospheric transmittance model;

[0016] Step 1.3: Store the scene data of the space-based infrared remote sensing scene database and the environmental infrared radiation transmission model in the scene file Infrared_Render.json that supports flexible loading and parameter modification to complete the construction of the scene database.

[0017] Optionally, in step 2, the CPU is used for data loading, parsing and initialization configuration to create an acceleration structure for the ray tracing pipeline. Data loading, parsing and initialization configuration include loading infrared remote sensing scene data, parsing model geometry information, physical material information and environmental parameters.

[0018] Optionally, the steps of configuring the ray tracing pipeline on the CPU side in step 2 include:

[0019] Step 2.1: Create a device context, create a shader module, shader instance, shader pipeline and bind SBT;

[0020] Step 2.2: Analyze the ambient infrared radiation transmission model and build the acceleration structure Traversable to complete the configuration of the ray tracing pipeline.

[0021] Optionally, in step 3, the GPU is used to execute core infrared ray tracing computing tasks, which include but are not limited to ray generation, scene traversal and infrared radiation calculations to generate high-quality infrared remote sensing images.

[0022] Optionally, the step of executing the core ray tracing calculation task in step 3 includes:

[0023] Step 3.1: Write a ".cu" file to implement the space-based infrared remote sensing ray tracing kernel algorithm on the GPU side, generate primary rays and define their propagation characteristics;

[0024] Step 3.2: Write a CUDA function module, perform ray generation, miss processing and hit processing based on the CUDA function module, and use the parallel computing capability of the GPU to accelerate the recursive calculation in the ray tracing process;

[0025] Step 3.3: Bind scene data and shader instances to the space-based infrared remote sensing ray tracing kernel, accurately simulate the physical behavior of infrared radiation, complete the core ray tracing calculation, and accurately simulate the physical behavior of infrared radiation including the calculation of infrared self-emitted radiation, reflected radiation, and projected radiation.

[0026] Optionally, the steps of processing and rendering the image in step 4 include:

[0027] Step 4.1: Introduce a deep learning-based image denoising algorithm and progressive rendering function to weighted average the current frame image with the previous rendering results;

[0028] Step 4.2: Use the ImGUI library function to design and implement the host computer interface, so that users can select corresponding functions and configure the rendering bands of space-based infrared remote sensing images;

[0029] Step 4.3: Based on the scene loading and scene configuration functions provided in steps 2 and 3, infrared remote sensing images under different conditions are generated according to the actual needs of the user.

[0030] The beneficial effects of the present invention are:

[0031] 1. High-quality imaging: This invention combines infrared radiometry with ray tracing algorithms, which can accurately simulate the physical behavior of infrared radiation, generate high-quality realistic infrared remote sensing simulation images, and significantly improve the realism and detail expression of the images.

[0032] 2. Efficient computing: The present invention uses a CPU+GPU heterogeneous hardware architecture to accelerate ray tracing, organically combining the CPU's complex instruction processing capabilities with the GPU's high parallel computing capabilities. Compared with traditional ray tracing rendering methods, this heterogeneous architecture acceleration technology improves computing efficiency by tens to hundreds of times, thereby achieving real-time generation of high-quality images and meeting the strict requirements of space-based infrared remote sensing imaging for real-time and high efficiency.

[0033] 3. Flexible scene rendering and configuration: The present invention supports the generation of infrared remote sensing simulation images under a variety of different conditions and provides flexible scene loading and configuration functions. Through the design of heterogeneous hardware architecture, the system can be adjusted according to actual needs, further enhancing the application flexibility and adaptability of the simulation system. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] Figure 1 A flowchart of a heterogeneous hardware ray tracing acceleration method for space-based infrared remote sensing imaging simulation provided by the present invention;

[0035] Figure 2 A flowchart of the pipeline function initialization configuration provided by the present invention;

[0036] Figure 3 A schematic diagram of the structure of the scene database provided by the present invention;

[0037] Figure 4 A schematic diagram of the flow of the shader program provided by the present invention;

[0038] Figure 5 This is a schematic diagram of the front-end and back-end module structures of the space-based infrared remote sensing imaging simulation and hardware acceleration design provided by the present invention. DETAILED DESCRIPTION

[0039] Combination Figure 1-5To explain this embodiment, a heterogeneous hardware ray tracing acceleration method for space-based infrared remote sensing imaging simulation described in this embodiment is coordinated by the CPU side (Host) and the GPU side (Device), and is accelerated using heterogeneous hardware architecture technology. On the CPU side, it is mainly responsible for loading and parsing the input space-based infrared remote sensing scene data, including obtaining the model's geometric information, physical material information, temperature field information, etc., completing the construction of the acceleration structure, and providing input data for subsequent ray tracing calculations on the GPU side. On the GPU side, relying on the powerful parallel computing capabilities of the GPU, the core and time-consuming infrared radiation ray tracing calculation tasks are executed. This includes primary and secondary ray generation, scene recursive traversal, and core infrared radiation calculations, and ultimately completes the caching and output of image data.

[0040] like Figure 1 As shown, the steps of a heterogeneous hardware ray tracing acceleration method for space-based infrared remote sensing imaging simulation described in this embodiment include:

[0041] S1: Build a space-based infrared remote sensing scene database;

[0042] S101: Figure 3 As shown, the space-based infrared remote sensing scene database includes but is not limited to the earth elevation model and the three-dimensional geometric model of the cloud top surface height. The MODTRAN software is used to calculate the environmental infrared radiation transmission model for a preset time period. The environmental infrared radiation transmission model includes but is not limited to the atmospheric radiation model, the direct sunlight model, the atmospheric transmittance model, etc. The radiation model of the remote sensing image surface and cloud layer is preset. The above data is preset in the scene file Infrared_Render.json to facilitate scene parameter modification and flexible loading.

[0043] S2: pipeline function initialization configuration;

[0044] S201: Figure 2 and Figure 3 As shown on the left, the CPU side (Host) completes the pipeline function initialization configuration according to the scene data model pre-loaded and parsed in S1. The pipeline function initialization configuration includes but is not limited to creating a device context (finding and starting the GPU), creating shader modules, shader instances, shader pipelines, and binding SBT. The core task of the CPU side is to parse the ambient infrared radiation transmission model and build the acceleration structure Traversable to improve the efficiency of light traversal and prepare for complex ray tracing calculations on the GPU side.

[0045] S3: Recursive calculation of space-based remote sensing infrared ray tracing;

[0046] S301: Write a function file type ".cu" file that can be executed on the GPU to implement the infrared remote sensing scene ray tracing kernel algorithm, generate primary light according to the "camera viewpoint" set in the .json file and define the propagation characteristics of the light;

[0047] S302: Write necessary CUDA function modules, including ray generation, miss processing and hit processing (solving the nearest intersection point);

[0048] Figure 4In the code, initOptix(), createInfrared_Context(), createInfrared_Module(), createRaygenPrograms(), createMissPrograms(), createHitgroupPrograms(), createInfrared_Pipeline(), buildInfrared_SBT(), and downloadPixels() are all designed shader program codes. Among them, initOptix() is used to initialize the OptiX framework, set the necessary context and device parameters, and prepare for subsequent ray tracing operations. createInfrared_Context() is used to create and configure the OptiX context of the infrared remote sensing scene, which is used to manage the overall status and resources of ray tracing. createInfrared_Module() is used to create and configure the OptiX context of the infrared remote sensing scene, which is used to manage the overall status and resources of ray tracing. odule() is used to load and compile the OptiX program module for infrared remote sensing simulation, which contains the CUDA code required for ray tracing. createRaygenPrograms() is used to create and configure ray generation programs to define the generation method and initial direction of light in the scene. createMissPrograms() is used to create and configure Miss programs to define the behavior or output when the light does not intersect with any object. createHitgroupPrograms() is used to create and configure Hitgroup programs to handle operations when the light intersects with objects in the scene, such as calculating reflection, refraction or absorption effects. createInfrared_Pipeline() is used to build the OptiX pipeline for infrared remote sensing simulation, integrating ray generation, Hitgroup and Miss programs to ensure smooth execution of the ray tracing process. buildInfrared_SBT() is used to build the Shader Binding Table (SBT). The ShaderBinding Table (SBT) is the core data structure in the OptiX pipeline and is used to link different ray tracing programs. downloadPixels() is used to download rendered pixel data from the GPU memory and save it to the host memory or file for further processing or display.

[0049] Infrared_Render.cu, Infrared_Render.cu.ptx and (char)Infrared_Shader.c are file formats related to GPU computing. Infrared_Render.cu is a CUDA source file, which contains the core ray tracing algorithm and rendering logic for infrared remote sensing simulation, and performs efficient calculations through the GPU. Infrared_Render.cu.ptx is a PTX (Parallel Thread Execution) intermediate code file generated by the CUDA compiler, which contains the low-level instructions compiled by Infrared_Render.cu for execution by the OptiX framework on the GPU. Infrared_Shader.c is a C language source file that defines the shader code used in infrared simulation, which is used to process special effects or material calculations related to ray tracing.

[0050] like Figure 4 The "createRaygenPrograms(), createMissPrograms(), createHitgroupPrograms()" shader programs shown on the right recursively call the corresponding shader programs to handle the interaction between light and scene objects, including reflection, refraction, and absorption characteristics, to ensure that the light can accurately simulate the propagation of infrared radiation in the scene and complete: the core and time-consuming recursive calculation of space-based remote sensing infrared ray tracing.

[0051] In order to accurately simulate infrared radiation, the space-based infrared ray tracing kernel needs to access detailed data in the scene, including the geometric shape of the object, material properties and environmental parameters, and the environmental parameters include but are not limited to atmospheric transmittance. This implementation binds the scene data loaded in S1 and the shader instance created in S2 to the ray tracing kernel to ensure that the kernel algorithm can use these data, such as the surface temperature field information of the ground and cloud layers, in Kelvin (symbol K) for accurate calculation. During the propagation of each ray, the light propagation path and the corresponding infrared radiation value are calculated according to the physical material property value set in the scene in the .json file, involving spontaneous radiation (calculated by Planck's blackbody radiation law), reflected radiation (supporting Cook-Torrance BRDF and Sandford-Robertson BRDF models) and transmitted radiation characteristic calculations. These calculations are efficiently executed through the parallel computing capabilities of the GPU. The powerful parallel computing of the GPU significantly accelerates the recursive calculations in the space-based infrared ray tracing process. Compared with the solution that relies solely on the CPU, while ensuring image quality, the rendering efficiency is greatly improved.

[0052] Figure 5It is a schematic diagram of the front-end and back-end module structure of space-based infrared remote sensing imaging simulation and hardware acceleration design. "Imgui.hpp, Imgui.cpp", "GLFWindow.hpp and GLFWindow.cpp" are open source library files for computer graphics. "Material.hpp" involves some material definitions used in the simulation. "Infrared_render.json" contains camera view configuration, image path import, etc. "Environment.hpp" involves environment loading, etc. The specific reference and inclusion relationships are as follows Figure 5 As indicated by the arrow in the middle.

[0053] Figure 5 The execution relationship between modules is shown in detail. The core calculation is infrared radiation calculation. The calculated infrared radiation value will be stored in the specified buffer. These buffers are used to save the intermediate calculation results and the final infrared radiation image data. The data stored in the buffer can be read and processed by the subsequent image rendering and display process with the help of library functions such as GLFWindow and ImGUI to generate high-quality infrared remote sensing images. S3 implements the complete process of infrared remote sensing scene ray tracing, from light generation, propagation, interaction to the final infrared radiation value calculation and storage, ensuring the accuracy and high-quality generation of infrared remote sensing images.

[0054] S4: Introduce image denoising algorithm and progressive rendering function based on deep learning, weighted average the current frame image and the previous rendering result, and realize the host computer interface with the help of ImGUI library functions. Users can choose the specific functions to be executed and the specific rendering band configuration of infrared remote sensing images according to actual needs. The present invention provides flexible scene loading options and supports the generation of different infrared remote sensing simulation images under various conditions, which enhances the application flexibility of the system.

[0055] The above is only a preferred embodiment of the present invention and does not limit the present invention in any form. Although the present invention has been disclosed as a preferred embodiment as above, it is not used to limit the present invention. Any technician familiar with this profession can make some changes or modify the technical contents disclosed above into equivalent embodiments without departing from the scope of the technical solution of the present invention. However, any simple modification, equivalent replacement and improvement made to the above embodiments without departing from the content of the technical solution of the present invention, based on the technical essence of the present invention, within the spirit and principles of the present invention, still fall within the protection scope of the technical solution of the present invention.

Claims

1. A heterogeneous hardware ray tracing acceleration method for space-based infrared remote sensing imaging simulation, characterized in that: The steps of the heterogeneous hardware ray tracing acceleration method for space-based infrared remote sensing imaging simulation include: Step 1: Build a space-based infrared remote sensing scene database; Step 2: Parse the scene database, configure the hardware ray tracing pipeline on the CPU side, and complete scene loading; Step 3: On the GPU side, perform the core infrared ray tracing calculation tasks according to the configured hardware ray tracing pipeline to complete the scene configuration; Step 4: Render and process the image, and generate infrared remote sensing images under different conditions according to the scene loading and scene configuration according to actual needs.

2. The heterogeneous hardware ray tracing acceleration method for space-based infrared remote sensing imaging simulation according to claim 1, characterized in that: The steps of building the scene database in step 1 include: Step 1.1: constructing a space-based infrared remote sensing scene database, wherein the space-based infrared remote sensing scene database includes but is not limited to an earth elevation model and a three-dimensional geometric model of cloud top surface height; Step 1.2: using MODTRAN software to calculate the environmental infrared radiation transmission model, wherein the environmental infrared radiation transmission model includes but is not limited to an atmospheric radiation model, a direct sunlight model, and an atmospheric transmittance model; Step 1.3: The scene data of the space-based infrared remote sensing scene database and the environmental infrared radiation transmission model are stored in a scene file Infrared_Render.json that supports flexible loading and parameter modification, thereby completing the construction of the scene database.

3. The heterogeneous hardware ray tracing acceleration method for space-based infrared remote sensing imaging simulation according to claim 1, characterized in that: In step 2, the CPU is used for data loading, parsing and initialization configuration to create an acceleration structure for the ray tracing pipeline. The data loading, parsing and initialization configuration include loading infrared remote sensing scene data, parsing model geometry information, physical material information and environmental parameters.

4. The heterogeneous hardware ray tracing acceleration method for space-based infrared remote sensing imaging simulation according to claim 1, characterized in that: The steps for configuring the ray tracing pipeline on the CPU side in step 2 include: Step 2.1: Create a device context, create a shader module, shader instance, shader pipeline and bind SBT; Step 2.2: Analyze the environmental infrared radiation transmission model, build the acceleration structure Traversable, and complete the configuration of the ray tracing pipeline.

5. The heterogeneous hardware ray tracing acceleration method for space-based infrared remote sensing imaging simulation according to claim 1, characterized in that: In step 3, the GPU is used to execute the core infrared ray tracing computing tasks, which include but are not limited to ray generation, scene traversal and infrared radiation calculation to generate high-quality infrared remote sensing images.

6. The heterogeneous hardware ray tracing acceleration method for space-based infrared remote sensing imaging simulation according to claim 1, characterized in that: The steps to perform the core ray tracing calculation tasks in step 3 include: Step 3.1: Write a ".cu" file to implement the space-based infrared remote sensing ray tracing kernel algorithm on the GPU side, generate primary rays and define their propagation characteristics; Step 3.2: Write a CUDA function module, perform ray generation, miss processing and hit processing based on the CUDA function module, and use the parallel computing capability of the GPU to accelerate the recursive calculation in the ray tracing process; Step 3.3: Bind the scene data and shader instance to the space-based infrared remote sensing ray tracing kernel, accurately simulate the physical behavior of infrared radiation, and complete the core ray tracing calculation. The accurate simulation of the physical behavior of infrared radiation includes the calculation of infrared self-emitted radiation, reflected radiation, and projected radiation.

7. The heterogeneous hardware ray tracing acceleration method for space-based infrared remote sensing imaging simulation according to claim 1, characterized in that: The steps of image processing and rendering in step 4 include: Step 4.1: Introduce a deep learning-based image denoising algorithm and progressive rendering function to weighted average the current frame image with the previous rendering results; Step 4.2: Use the ImGUI library function to design and implement the host computer interface, so that users can select corresponding functions and configure the rendering bands of space-based infrared remote sensing images; Step 4.3: Based on the scene loading and scene configuration functions provided in steps 2 and 3, infrared remote sensing images under different conditions are generated according to the actual needs of the user.

Images