Real-time coupling method for infrared characteristics of participating media based on graphic rendering engine

Abstract

The embodiment of the invention relates to the technical field of infrared characteristic coupling, in particular to a graphic rendering engine-based participating medium infrared characteristic real-time coupling method. Comprising the following steps: determining the radiation brightness of a target object collected by a camera according to the radiation brightness of the target object and the transmittance and the path radiation of a first space medium; decoupling the participating medium from each simulation object according to the radiation brightness of the participating medium, the transmittance of the participating medium, the transmittance of the second space medium and the path radiation, so as to enable the influence of the participating medium on the whole scene and the influence of the space medium on the participating medium to be equivalent to the radiation characteristics of the participating medium; obtaining the equivalent characteristic of the participating medium; and rasterization is performed on the scene based on an image rendering engine, so that the equivalent characteristic of the participating medium and the radiation characteristic of the target object are mixed in the same pixel, and scene infrared characteristic simulation is obtained. According to the method provided by the invention, real-time simulation of the scene infrared characteristics with participating media can be realized.

Description

Technical Field

[0001] The embodiment of the present invention relates to the field of infrared characteristic coupling technology, and in particular to a method for real-time coupling of infrared characteristics of a participating medium based on a graphics rendering engine. Background Art

[0002] In infrared scene simulation, the mutual coupling between the target and the environment, especially the influence of the transmission medium, needs to be considered. The transmission medium includes spatial media distributed throughout the space, such as the atmosphere, and semi-transparent participating media distributed in local space, such as jets, smoke screens, clouds, and smoke plumes. When simulating the infrared characteristics of a scene, in addition to calculating the radiation of the object itself, it is necessary to consider the coupling of the transmission characteristics of the spatial medium on the path between the camera and the object, and the coupling of the transmission of the participating medium on the path. The existence of participating media will divide the detection path into multiple segments, making the transmission characteristics from the object to the camera a function related to the position of the participating medium.

[0003] In the real-time simulation of infrared scenes, due to the complexity of the calculation elements, it is often necessary to use a graphics rendering engine to achieve real-time coupled simulation of the scene's infrared characteristics. When using a graphics rendering engine for simulation, it is first necessary to calculate the infrared characteristics of all objects in the scene separately, and then calculate the mixing or occlusion between objects at different distances of the same pixel in the scene based on the camera projection, and finally output the final image according to a certain mixing method. When calculating the infrared characteristics of the object separately in the first step, the infrared transmission characteristics between the object and the camera need to be coupled to the radiation characteristics of the object. If there is a participating medium in the scene, it is necessary to determine whether there is a participating medium on the path between each vertex of the object and the camera, the position and thickness of the path through the participating medium, and the transmission characteristics of the participating medium. This makes two or more objects that were originally simulated separately associated together, making the scene infrared characteristics simulation process very complicated and unable to meet the performance requirements of real-time simulation. Summary of the invention

[0004] The embodiments of the present invention provide a method, device, electronic device and storage medium for real-time coupling of infrared characteristics of participating media based on a graphics rendering engine, which can realize real-time simulation of infrared characteristics of a scene with participating media.

[0005] In a first aspect, an embodiment of the present invention provides a method for real-time coupling of infrared characteristics of a participating medium based on a graphics rendering engine, comprising:

[0006] Determine the radiance of the target object captured by the camera when no participating medium is included according to the radiance of the target object, the transmittance of the first spatial medium and the path radiation; wherein the first spatial medium is the spatial medium between the target object and the camera;

[0007] According to the radiation brightness of the participating medium, the transmittance of the participating medium, the transmittance of the second spatial medium and the process radiation, the participating medium is decoupled from each simulation object, so that the influence of the participating medium on the entire scene and the influence of the spatial medium on the participating medium are equivalent to the radiation characteristics of the participating medium itself, and the equivalent characteristics of the participating medium are obtained; wherein the second spatial medium is the spatial medium between the participating medium and the camera;

[0008] The scene is rasterized based on an image rendering engine, so that the equivalent characteristics of the participating medium and the radiation characteristics of the target object are mixed with the same pixel to obtain a simulation of the infrared characteristics of the scene.

[0009] In a second aspect, an embodiment of the present invention further provides a device for real-time coupling of infrared characteristics of a participating medium based on a graphics rendering engine, for implementing any of the methods described in the above claims, the device comprising:

[0010] A calculation unit, configured to determine the radiance of the target object captured by the camera when no participating medium is included, based on the radiance of the target object, the transmittance of the first spatial medium, and the path radiation; wherein the first spatial medium is the spatial medium between the target object and the camera;

[0011] An equivalent unit is used to decouple the participating medium from each simulation object according to the radiation brightness of the participating medium, the transmittance of the participating medium, the transmittance of the second spatial medium and the process radiation, so that the influence of the participating medium on the entire scene and the influence of the spatial medium on the participating medium are equivalent to the radiation characteristics of the participating medium itself, and the equivalent characteristics of the participating medium are obtained; wherein the second spatial medium is the spatial medium between the participating medium and the camera;

[0012] The rendering unit is used to rasterize the scene based on the image rendering engine, so that the equivalent characteristics of the participating medium and the radiation characteristics of the target object are mixed with the same pixel to obtain the infrared characteristic simulation of the scene.

[0013] In a third aspect, an embodiment of the present invention further provides an electronic device, including a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the method described in any embodiment of this specification is implemented.

[0014] In a fourth aspect, an embodiment of the present invention further provides a computer-readable storage medium having a computer program stored thereon, which, when executed in a computer, enables the computer to execute the method described in any embodiment of this specification.

[0015] The embodiment of the present invention provides a method, device, electronic device and storage medium for real-time coupling of infrared characteristics of participating media based on a graphics rendering engine. In order to realize real-time simulation of scenes in a graphics rendering engine, we hope to decouple the simulation objects (objects and participating media, participating media and participating media) in the first stage of simulation and perform separate simulations, that is, to equate the mutual influence of participating media and other objects in the scene to their own radiation characteristics, and use the equivalent characteristics as new targets for graphics rendering, without considering the occlusion relationship between participating media and other targets again, and complete the simulation by mixing the same pixels with independent radiation characteristics through a graphics rendering engine. The present invention proposes a method for real-time simulation of infrared characteristics of participating media based on a graphics rendering engine when simulating infrared characteristics of scenes with participating media, which solves the problem that the close coupling of transmission characteristics between participating media and simulation objects in the process of real-time simulation of infrared characteristics based on a graphics rendering engine leads to a serious decline in scene simulation performance, and realizes real-time simulation of infrared characteristics of scenes with participating media. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0017] Figure 1 It is a schematic diagram of the spatial relationship between a participatory medium and a scene object provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0018] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments in the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0019] As mentioned above, in the previous real-time simulation of scene infrared characteristics, the influence of spatial medium (atmosphere) transmission on participating medium and the influence of participating medium on other objects are often considered, but the influence of participating medium on spatial medium segmentation is ignored; or it is assumed that participating medium will have an impact on all spatial media, the influence of participating medium is coupled to spatial medium, and the same medium as spatial medium is used to couple to object radiation. The above two methods of participating medium simulation are unreasonable or inaccurate.

[0020] The specific implementation of the above concept is described below.

[0021] Please refer to Figure 1 The embodiment of the present invention provides a method for real-time coupling of infrared characteristics of a participating medium based on a graphics rendering engine, the method comprising:

[0022] Determine the radiance of the target object captured by the camera when no participating medium is included according to the radiance of the target object, the transmittance of the first spatial medium and the path radiation; wherein the first spatial medium is the spatial medium between the target object and the camera;

[0023] According to the radiation brightness of the participating medium, the transmittance of the participating medium, the transmittance of the second spatial medium and the process radiation, the participating medium is decoupled from each simulation object, so that the influence of the participating medium on the entire scene and the influence of the spatial medium on the participating medium are equivalent to the radiation characteristics of the participating medium itself, and the equivalent characteristics of the participating medium are obtained; wherein the second spatial medium is the spatial medium between the participating medium and the camera;

[0024] The scene is rasterized based on an image rendering engine, so that the equivalent characteristics of the participating medium and the radiation characteristics of the target object are mixed with the same pixel to obtain a simulation of the infrared characteristics of the scene.

[0025] In order to realize the real-time simulation of the scene in the graphics rendering engine, we hope to decouple the simulation objects (objects and participating media, participating media and participating media) in the first stage of simulation and perform separate simulations, that is, to equate the mutual influence between the participating media and other objects in the scene to their own radiation characteristics, and use the equivalent characteristics as new targets for graphics rendering, without considering the occlusion relationship between the participating media and other targets again. After the independent radiation characteristics are mixed with the same pixel through the graphics rendering engine, the simulation can be completed. The present invention proposes a method for real-time simulation of infrared characteristics of participating media based on a graphics rendering engine when simulating the infrared characteristics of a scene with participating media, which solves the problem that the close coupling of the transmission characteristics between the participating media and the simulation objects in the real-time simulation of infrared characteristics based on the graphics rendering engine leads to a serious decline in the scene simulation performance, and realizes the real-time simulation of infrared characteristics of scenes with participating media.

[0026] In some embodiments of the present invention, the scene includes a plurality of participating media and a plurality of target objects.

[0027] In some embodiments of the present invention, the radiant brightness of the target object captured by the camera is calculated by the following formula:

[0028] Qτ2+R2

[0029] Among them, Q is the radiation brightness of the target object, τ2 and R2 are the transmittance and path radiation of the first space medium respectively.

[0030] In some embodiments of the present invention, the equivalent characteristics of the participating medium include equivalent transmittance and equivalent radiance.

[0031] In some embodiments of the present invention, the equivalent transmittance is still the transmittance of the participating medium, and the equivalent radiant brightness is as follows:

[0032] R Δ (1-τ y )+τ Δ R y

[0033] Among them, τ Δ and R Δ are the transmittance and path radiation of the second space medium, τ y and R y are the transmittance and path radiation of the participating media respectively.

[0034] In this embodiment, the expression of equivalent radiance is obtained as follows:

[0035] We assume that in the simulation scene, there is an object (assumed to be a target or background) at point A, the camera is looking at the object at point C, there is a space medium (assumed to be a cloud) at point B between AC, and the space is filled with space medium (assumed to be the atmosphere). The radiation brightness of the object is Q, the transmittance and path radiation from the object to the space medium (the first section of the atmosphere) at the participating medium (cloud) are τ1 and R1 respectively, the transmittance and path radiation from the object to the space medium at the camera (the entire atmosphere, ignoring the influence of the participating medium) are R2 and R2 respectively, and the transmittance and path radiation from the participating medium (cloud) to the space medium at the camera (the second section of the atmosphere) are τ Δ and R Δ ,like Figure 1 As shown;

[0036] It is easy to conclude that τ1 and R1, τ2 and R2, τ Δ and R Δ The following relationship exists:

[0037] (Qτ1+R1)τ Δ +R Δ =Qτ2+R2

[0038] It follows that: Δ =τ2 / τ1, R Δ =R2-R1τ2 / τ1.

[0039] Assume that the overall transmittance and path radiance of the participating medium in the camera viewing direction are τ y and R y , then the radiant brightness at the camera entrance pupil can be calculated, the process is as follows:

[0040] The radiation brightness of the object after being transmitted through the first space medium to point B is: Qτ1+R1

[0041] The radiation brightness after participating medium coupling is: (Qτ1+R1)τ y +R y

[0042] After being transmitted through the second space medium, the radiant brightness at the camera entrance pupil is: [(Qτ1+R1)τ y +R y ]τ Δ +R Δ

[0043] The radiation brightness at the camera entrance pupil is derived as follows:

[0044] [(Qτ1+R1)τ y +R y ]τ Δ +R Δ

[0045] =[(Qτ1+R1)τ y +R y ]τ2 / τ1+R2-R1τ2 / τ1

[0046] =(Qτ2+R2)τ y -R2τ y +R1τ y τ2 / τ1+R y τ2 / τ1+R2-R1τ2 / τ1

[0047] =(Qτ2+R2)τ y +(R2-R1τ2 / τ1)(1-τ y )+R y τ2 / τ1

[0048] =(Qτ2+R2)τ y +RΔ (1-τ y )+τ Δ R y

[0049] From the above deduction, we can draw the following conclusion:

[0050] The participating medium transmission coupling scenario in the scenario is equivalent to the simple superposition of the following two scenarios:

[0051] (1) Scenarios when there is no participating medium (coupled space medium transmission);

[0052] (2) A medium with only equivalent participating media (with equivalent transmittance τ d and equivalent radiation R d ) scene.

[0053] The coupled scene radiance for an object in the first scene without participating media is Qτ2+R2.

[0054] The equivalent participatory medium for the second scenario is: τ d =τ y ,R d =R Δ (1-τ y )+τ Δ R y The transmittance of the equivalent participating medium is the same as the transmittance of the original participating medium. The radiation value of the equivalent participating medium is only related to the original participating medium properties (transmittance, path radiation) and the spatial transmission characteristics between the participating medium and the camera, and has nothing to do with other objects in the scene.

[0055] In this way, we can simulate the scene containing participatory media equivalent to a scene without participatory media and several separate equivalent participatory media, and then simulate each participatory medium and scene separately, and then synthesize them into one scene after simulation. There is no need to consider the occlusion relationship between the participatory medium and other objects or background in the scene, or between the participatory media, thereby achieving the decoupling of the target (or background) and the participatory medium simulation.

[0056] In some embodiments of the present invention, the same-pixel mixing is achieved by a mixing function, and the mixing function is as follows:

[0057] C src +τ src C dst

[0058] Among them, τ src , C src are the transmittance and radiance of the target object or the participating medium that is closer to the camera in the scene, respectively, and Cdst is the radiance of the target object or the participating medium that is farther away from the camera.

[0059] It should be noted that the object that the camera is close to can be the target object or the participating medium.

[0060] In some embodiments of the present invention, the participatory medium includes flames, smoke screens, clouds and plumes, and the spatial medium includes the atmosphere.

[0061] An embodiment of the present invention provides a device for real-time coupling of infrared characteristics of a participatory medium based on a graphics rendering engine. The device embodiment can be implemented by software, or by hardware or a combination of software and hardware. From a hardware perspective, a hardware architecture diagram of an electronic device in which a device for real-time coupling of infrared characteristics of a participatory medium based on a graphics rendering engine is provided in an embodiment of the present invention. In addition to a processor, memory, network interface, and non-volatile memory, the electronic device in the embodiment where the device is located can generally include other hardware, such as a forwarding chip responsible for processing messages, etc. Taking software implementation as an example, as a device in a logical sense, it is formed by the CPU of the electronic device in which it is located reading the corresponding computer program in the non-volatile memory into the memory for execution. A device for real-time coupling of infrared characteristics of a participatory medium based on a graphics rendering engine provided in this embodiment includes:

[0062] A calculation unit, configured to determine the radiance of the target object captured by the camera when no participating medium is included, based on the radiance of the target object, the transmittance of the first spatial medium, and the path radiation; wherein the first spatial medium is the spatial medium between the target object and the camera;

[0063] An equivalent unit is used to decouple the participating medium from each simulation object according to the radiation brightness of the participating medium, the transmittance of the participating medium, the transmittance of the second spatial medium and the process radiation, so that the influence of the participating medium on the entire scene and the influence of the spatial medium on the participating medium are equivalent to the radiation characteristics of the participating medium itself, and the equivalent characteristics of the participating medium are obtained; wherein the second spatial medium is the spatial medium between the participating medium and the camera;

[0064] The rendering unit is used to rasterize the scene based on the image rendering engine, so that the equivalent characteristics of the participating medium and the radiation characteristics of the target object are mixed with the same pixel to obtain the infrared characteristic simulation of the scene.

[0065] It is to be understood that the structure illustrated in the embodiment of the present invention does not constitute a specific limitation on a real-time coupling device for infrared characteristics of a participating medium based on a graphics rendering engine. In other embodiments of the present invention, a real-time coupling device for infrared characteristics of a participating medium based on a graphics rendering engine may include more or fewer components than shown in the figure, or combine some components, or split some components, or arrange the components differently. The components shown in the figure may be implemented in hardware, software, or a combination of software and hardware.

[0066] The information interaction, execution process and other contents between the modules in the above-mentioned device are based on the same concept as the embodiment of the method of the present invention. For the specific contents, please refer to the description in the embodiment of the method of the present invention, and no further description is given here.

[0067] An embodiment of the present invention further provides an electronic device, including a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, a method for real-time coupling of infrared characteristics of a participating medium based on a graphics rendering engine in any embodiment of the present invention is implemented.

[0068] An embodiment of the present invention further provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the processor executes a method for real-time coupling of infrared characteristics of a participatory medium based on a graphics rendering engine in any embodiment of the present invention.

[0069] Specifically, a system or device equipped with a storage medium can be provided, on which software program code that implements the functions of any of the above-mentioned embodiments is stored, and a computer (or CPU or MPU) of the system or device can be enabled to read and execute the program code stored in the storage medium.

[0070] In this case, the program code itself read from the storage medium can realize the function of any one of the above-mentioned embodiments, and thus the program code and the storage medium storing the program code constitute a part of the present invention.

[0071] The storage medium embodiments for providing the program code include a floppy disk, a hard disk, a magneto-optical disk, an optical disk (such as CD-ROM, CD-R, CD-RW, DVD-ROM, DVD-RAM, DVD-RW, DVD+RW), a magnetic tape, a non-volatile memory card, and a ROM. Alternatively, the program code can be downloaded from a server computer by a communication network.

[0072] In addition, it should be clear that the functions of any of the above embodiments can be implemented not only by executing the program code read by the computer, but also by enabling an operating system operating on the computer to complete part or all of the actual operations based on instructions from the program code.

[0073] In addition, it can be understood that the program code read from the storage medium is written to a memory provided in an expansion board inserted into the computer or to a memory provided in an expansion module connected to the computer, and then based on the instructions of the program code, a CPU installed on the expansion board or expansion module is enabled to perform part or all of the actual operations, thereby realizing the functions of any of the above-mentioned embodiments.

[0074] It should be noted that, in this article, relational terms such as first and second are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the statement "comprise a ..." do not exclude the presence of other identical factors in the process, method, article or device including the elements.

[0075] A person of ordinary skill in the art can understand that all or part of the steps of implementing the above method embodiments can be completed by hardware related to program instructions, and the aforementioned program can be stored in a computer-readable storage medium. When the program is executed, it executes the steps of the above method embodiments; and the aforementioned storage medium includes: ROM, RAM, magnetic disk or optical disk, etc., various media that can store program codes.

[0076] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A real-time coupling method of infrared characteristics of participating media based on a graphics rendering engine, characterized in that: include: Determine the radiance of the target object captured by the camera when no participating medium is included according to the radiance of the target object, the transmittance of the first spatial medium and the path radiation; wherein the first spatial medium is the spatial medium between the target object and the camera; According to the radiation brightness of the participating medium, the transmittance of the participating medium, the transmittance of the second spatial medium and the process radiation, the participating medium is decoupled from each simulation object, so that the influence of the participating medium on the entire scene and the influence of the spatial medium on the participating medium are equivalent to the radiation characteristics of the participating medium itself, and the equivalent characteristics of the participating medium are obtained; wherein the second spatial medium is the spatial medium between the participating medium and the camera; The scene is rasterized based on an image rendering engine, so that the equivalent characteristics of the participating medium and the radiation characteristics of the target object are mixed with the same pixel to obtain a simulation of the infrared characteristics of the scene.

2. The method according to claim 1, characterized in that The scene includes a plurality of participating media and a plurality of target objects.

3. The method according to claim 1, characterized in that The radiance of the target object collected by the camera is calculated by the following formula: Qτ2+R2 Among them, Q is the radiation brightness of the target object, τ2 and R2 are the transmittance and path radiation of the first space medium respectively.

4. The method according to claim 1, characterized in that: The equivalent characteristics of the participating medium include equivalent transmittance and equivalent radiance.

5. The method according to claim 4, characterized in that The equivalent transmittance is still the transmittance of the participating medium, and the equivalent radiant brightness is as follows: R Δ (1-t y )+τ Δ R y Among them, τ Δ and R Δ are the transmittance and path radiation of the second space medium, τ y and R y are the transmittance and path radiation of the participating media respectively.

6. The method according to claim 1, characterized in that The same pixel mixing is achieved by a mixing function, and the mixing function is as follows: C src +t src C dst Among them, τ src , C src are the transmittance and radiance of the target object or the participating medium that is closer to the camera in the scene, respectively, and C dst is the radiance of the target object or the participating medium that is farther away from the camera.

7. The method according to claim 1, characterized in that The participating media include flames, smoke screens, clouds and plumes, and the spatial medium includes the atmosphere.

8. A real-time coupling device for infrared characteristics of participating media based on a graphics rendering engine, characterized in that: For implementing the method described in any one of claims 1 to 7, the device comprises: A calculation unit, configured to determine the radiance of the target object captured by the camera when no participating medium is included, based on the radiance of the target object, the transmittance of the first spatial medium, and the path radiation; wherein the first spatial medium is the spatial medium between the target object and the camera; An equivalent unit is used to decouple the participating medium from each simulation object according to the radiation brightness of the participating medium, the transmittance of the participating medium, the transmittance of the second spatial medium and the process radiation, so that the influence of the participating medium on the entire scene and the influence of the spatial medium on the participating medium are equivalent to the radiation characteristics of the participating medium itself, and the equivalent characteristics of the participating medium are obtained; wherein the second spatial medium is the spatial medium between the participating medium and the camera; The rendering unit is used to rasterize the scene based on the image rendering engine, so that the equivalent characteristics of the participating medium and the radiation characteristics of the target object are mixed with the same pixel to obtain the infrared characteristic simulation of the scene.

9. An electronic device, comprising a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the method according to any one of claims 1 to 7 is implemented.

10. A computer-readable storage medium having a computer program stored thereon, which, when executed in a computer, causes the computer to execute the method according to any one of claims 1 to 7.

Images