A complex scene infrared image data regeneration method, device and storage medium

Abstract

The present invention discloses a method, device and storage medium for regenerating infrared image data of complex scenes, which relates to the field of image processing and is used to improve the infrared computing efficiency of complex scenes. The present invention obtains an intrinsic radiation library by performing infrared simulation on a target to be measured, matches the infrared radiation results according to the posture relationship between the detector and the target to be measured in the constructed complex scene, replaces the target to be measured in the bulletin board, and then uses a ray tracing method to calculate the infrared image data of the target to be measured. In the ray tracing process, the present invention does not need to track and calculate a large number of volume grids, which can greatly improve the computing efficiency.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and in particular to a method, a device and a storage medium for regenerating infrared image data of a complex scene. Background Art

[0002] Infrared image data regeneration is a process of fusing the target infrared radiation results with complex scenes and simulating the infrared radiation brightness.

[0003] In the infrared simulation calculation of complex scenes, the target, sky background and ground background are all indispensable parts. However, since the target may have high-temperature fluid areas such as tail flames or smoke plumes, before performing infrared simulation calculations on complex scenes, it is necessary to perform thermal solution on the flow field area of ​​the target and then integrate the target into the complex infrared scene.

[0004] When infrared simulation calculations are performed on complex scenes based on the ray tracing method, the existing technology of infrared simulation calculations of complex scenes represented by the paper "Aircraft Infrared Image Simulation Based on Ray Tracing" published by Chen Yitao proposes that when the light intersects with the target flow field area, the infrared radiation brightness is calculated by stepping through the grids one by one. During the process of light travel, if it intersects with the target wall, its emissivity, reflectivity and radiation brightness need to be calculated according to the wall material and temperature, and the direction of light tracing needs to be changed according to its reflection characteristics, and then the tracing continues until there is no intersection with any model in the direction of light travel. Although this method can effectively simulate and calculate the infrared radiation brightness of complex scenes within the detection range of the detector, the amount of calculation depends not only on the number of rays, but also on the number of target grids, and the calculation efficiency is low. If a large number of working condition simulation calculations are performed, it will take a lot of time. Summary of the invention

[0005] The object of the present invention is to provide a method, device and storage medium for regenerating infrared image data of complex scenes to improve the efficiency of infrared calculation of complex scenes in order to address all or part of the above-mentioned problems.

[0006] The technical solution adopted by the present invention is as follows:

[0007] A method for regenerating infrared image data of a complex scene, comprising:

[0008] Establish a three-dimensional scene including the target to be measured and the detector;

[0009] Calculating the position and posture of the target to be measured in the three-dimensional scene relative to the detector;

[0010] According to the posture of the target to be measured, the infrared radiation result of the corresponding posture is matched from the intrinsic radiation library; the intrinsic radiation library calculates the intrinsic radiation of the target to be measured from multiple angles;

[0011] Using the infrared radiation results matched from the intrinsic radiation library, replacing the target to be measured in the three-dimensional scene in the form of a bulletin board;

[0012] The ray tracing method is used to calculate the infrared image data of the target to be measured.

[0013] Optionally, the method for generating the intrinsic radiation library includes:

[0014] Configure the angle parameters and distance parameters of the simulation calculation;

[0015] According to the flow field data of the target to be measured, the infrared radiation results of detecting the target to be measured at various configured angles and distances are calculated respectively;

[0016] The infrared radiation results of the target to be measured detected from various angles are summarized to generate an intrinsic radiation library.

[0017] Optionally, the configured angle parameters of the simulation calculation include detection angles selected at multiple pitch angles and equal azimuth angle intervals with the target to be detected as the sphere center.

[0018] Optionally, the step of establishing a three-dimensional scene including the target to be detected and the detector includes:

[0019] Place the target to be measured in the three-dimensional scene and set the position and posture of the target to be measured;

[0020] Place the detector in the 3D scene and set its position and pose.

[0021] Optionally, the calculating the position and posture of the target to be detected in the three-dimensional scene relative to the detector includes:

[0022] Establish the detector coordinate system;

[0023] According to the poses of the target and the detector in the three-dimensional scene coordinate system, the pose of the target in the detector coordinate system is calculated.

[0024] Optionally, replacing the target to be measured in the three-dimensional scene in the bulletin board with the infrared radiation result matched from the intrinsic radiation library includes:

[0025] Calculate the image size of the infrared radiation result according to the number of pixels of the infrared radiation result matched from the intrinsic radiation library, the detection distance and the detection angle set during the simulation calculation;

[0026] According to the coordinates of the target to be measured in the three-dimensional scene, the coordinates of the four corner points of the bulletin board element are calculated, and according to the coordinates of the origin of the target to be measured corresponding to the infrared radiation result, the origin of the target to be measured in the bulletin board element is compared, and the infrared radiation result is bound to the bulletin board element in the form of texture.

[0027] Optionally, the method of calculating the infrared image data of the target to be measured by using a ray tracing method includes:

[0028] When tracing rays, for each ray, we have:

[0029] Get the intersection point between the light and the billboard pixel and calculate the corresponding texture coordinates;

[0030] Calculating the pixel position of the current intersection point in the infrared radiation result according to the texture coordinates and the image size of the infrared radiation result, and obtaining the infrared radiation brightness value and transmittance of the pixel according to the infrared radiation result;

[0031] When the transmittance is 0, the tracking is stopped, and the infrared radiation brightness value of the corresponding pixel of the detector is calculated according to the infrared radiation brightness value of the current pixel; when the transmittance is not 0, the forward tracking is continued, and the infrared radiation brightness value of the corresponding pixel of the detector is calculated according to the infrared radiation brightness value of the current pixel.

[0032] Optionally, the method for calculating the infrared radiation brightness value of the corresponding pixel of the detector according to the infrared radiation brightness value of the current pixel is:

[0033] ,

[0034] in, is the infrared radiation brightness value received by the detector; The target to be measured is in pixels s Infrared radiation brightness value; The target to be measured is in pixels s The transmittance of is the atmospheric transmittance between the target to be measured and the detector; The atmospheric radiation between the target to be measured and the detector; It is the atmospheric radiation behind the target to be measured along the direction of the light.

[0035] The present invention also provides a storage medium storing a computer program, and running the computer program can execute the above-mentioned complex scene infrared image data regeneration method.

[0036] The present invention also provides a device, which includes a processor and the above-mentioned storage medium, and the processor is used to run the computer program in the storage medium.

[0037] In summary, due to the adoption of the above technical solution, the beneficial effects of the present invention are:

[0038] The present invention replaces the original target model in the three-dimensional scene when simulating the target with the infrared radiation result calculated and output by the infrared simulation of the target to be measured, so that the calculated target infrared radiation result can be integrated into the complex scene, thereby realizing the regeneration of infrared image data of the complex scene. The present invention can ensure that the target and other elements in the complex scene maintain the original spatial relative position relationship, as well as the normal superposition of atmospheric transmission. In the process of ray tracing, there is no need to track and calculate a large number of volume grids, which can greatly improve the calculation efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] The present invention will now be described by way of example with reference to the accompanying drawings, in which:

[0040] Figure 1 The figure is a flow chart of a complex scene infrared image data regeneration method in one embodiment.

[0041] Figure 2 It is a schematic diagram of an embodiment of configuring angle parameters for simulation calculation. DETAILED DESCRIPTION

[0042] All features disclosed in this specification, or steps in all methods or processes disclosed, except mutually exclusive features and / or steps, can be combined in any manner.

[0043] Any feature disclosed in this specification (including any additional claims and abstract), unless otherwise stated, may be replaced by other alternative features that are equivalent or have similar purposes. That is, unless otherwise stated, each feature is only an example of a series of equivalent or similar features.

[0044] Example 1

[0045] The embodiment of the present application proposes a complex scene infrared image data regeneration method, which is performed based on the infrared simulation results of the target to be measured. Figure 1 As shown, the method includes the following processes:

[0046] S1. Construct an intrinsic radiation library of the target to be measured.

[0047] The purpose of this step is to construct an intrinsic radiation library that calculates the intrinsic radiation of the target to be measured from multiple angles based on the infrared simulation of the target to be measured, so as to be directly matched and used when performing infrared simulation on the target to be measured.

[0048] In some embodiments, this step includes the following sub-steps:

[0049] S1.1. Configure the angle parameters and distance parameters for simulation calculation.

[0050] In some feasible implementations, the configured angle parameters of the simulation calculation include detection angles selected at multiple pitch angles and equal azimuth angle intervals with the target to be detected as the sphere center.

[0051] like Figure 2 As shown, the configured angle parameters can be:

[0052] With the target to be detected as the sphere center, the pitch angles are 90 degrees, 45 degrees, 0 degrees, -45 degrees and -90 degrees, and the azimuth angles are -180 degrees (or 0 degrees), -135 degrees, -90 degrees, -45 degrees, 0 degrees, 45 degrees, 90 degrees, and 135 degrees. Since the angles are configured from the spherical coordinate system, all azimuth angles are in the same position at the 90-degree and -90-degree pitch angles. Of course, more or fewer detection angles can be configured on this basis, or they can be partially or completely replaced with other detection angles, for example, the pitch angle and / or azimuth angle can be configured to have an angle difference of only 5 degrees to increase the density of detection points.

[0053] The distance parameter is set to the configured distance, that is, the detection distance configured under each configured angle parameter. Figure 2 In the corresponding embodiment, for each configured angle parameter, the distance between the detector and the target to be detected (ie, the detection distance) is equal.

[0054] S1.2. According to the flow field data of the target to be measured, the infrared radiation results of the target to be measured at each configured angle are calculated respectively. Usually, the calculated infrared radiation results include infrared radiation brightness value and transmittance. In the case of the flow field data of the target to be measured and the position of the detector, the infrared radiation results detected at different detection angles can be calculated through infrared simulation.

[0055] S1.3. Summarize the infrared radiation results of the target to be measured detected from various angles to generate an intrinsic radiation library.

[0056] In practical applications, if an intrinsic radiation library already exists, the above step S1 can be omitted.

[0057] S2. Establish a three-dimensional scene including the target to be detected and the detector.

[0058] The three-dimensional scene to be constructed is the simulated complex scene. The target to be measured in the above embodiment may be the same target entity as the target to be measured here.

[0059] In some embodiments, a method for establishing a three-dimensional scene includes:

[0060] S2.1. Place the target to be measured in the three-dimensional scene and set the position and posture (i.e., position and attitude) of the target to be measured. In addition, in the simulated complex scene, the sky background and the ground background are also included. Therefore, in the three-dimensional scene, models such as buildings, clouds, and trees can also be placed. Similarly, the position and posture of these model entities need to be set.

[0061] S2.2, place the detector in the three-dimensional scene and set the detector's position and posture. For the detector, its posture is its orientation.

[0062] S3. Calculate the position and posture of the target to be measured in the three-dimensional scene relative to the detector.

[0063] The purpose of calculating the position of the target to be measured relative to the detector is to match the corresponding infrared radiation result from the intrinsic radiation library. In some embodiments, this step includes the following sub-steps:

[0064] S3.1. Establish the detector coordinate system. This is usually constructed based on the position and posture (i.e., orientation) of the detector in the three-dimensional scene coordinate system to facilitate the subsequent coordinate conversion.

[0065] S3.2. Calculate the position and posture of the target to be measured in the detector coordinate system according to the position and posture of the target to be measured and the detector in the three-dimensional scene coordinate system. The coordinate conversion of the target to be measured between different coordinate systems can be completed by existing technology, and the specific conversion process is not described here.

[0066] S4. Match the infrared radiation result of the corresponding posture from the intrinsic radiation library according to the posture of the target to be measured.

[0067] The intrinsic radiation library has been introduced above, which contains the infrared radiation results detected by the target under multiple detection angles. According to the angle between the target and the detector, the infrared radiation results corresponding to the corresponding detection angle can be matched.

[0068] S5. Using the infrared radiation results matched from the intrinsic radiation library, the target to be measured in the three-dimensional scene is replaced in the form of a bulletin board.

[0069] In the embodiment of the present application, the traditional grid-by-grid stepping tracing method for calculating infrared radiation brightness is abandoned, and instead, the billboard technology is used to perform ray tracing in the form of texture mapping.

[0070] In some embodiments, step S5 includes the following sub-steps:

[0071] S5.1. According to the number of pixels (width and height )、The detection distance set during simulation calculation ( ) and the detection angle, calculate the image size of the infrared radiation result. The image size calculated here is the actual size of the infrared radiation result image, which is the size of the texture map mixed in the billboard pixel.

[0072] S5.2. Calculate the coordinates of the four corner points of the billboard element according to the coordinates of the target to be measured in the three-dimensional scene. In some possible implementations, the calculation method includes:

[0073] (1) Obtain the coordinates of the target to be measured in the three-dimensional scene and the coordinates of the detector , calculate the distance between the target and the detector:

[0074] .

[0075] (2) According to the detector's field of view (including horizontal field of view and vertical field of view ) and the detection distance configured during simulation calculation ( ), calculate the spatial position coordinates (P1, P2, P3, P4) in the detector coordinate system corresponding to the four corner points of the infrared radiation result image of the target to be measured during simulation calculation:

[0076] ,

[0077] The coordinates of the four corner points are: , , , .

[0078] (3) According to the distance dts between the target to be measured and the detector in the three-dimensional scene, the coordinates of the four corner points of the infrared radiation result of the target to be measured in the detector coordinate system are calculated, that is, the coordinates of the four corner points of the bulletin board: , , , .

[0079] (4) Based on the coordinate transformation relationship between the detector coordinate system and the three-dimensional scene coordinate system, the coordinates of the four corner points of the billboard in the three-dimensional scene coordinate system are calculated.

[0080] According to the coordinates of the origin of the target to be measured corresponding to the infrared radiation result, the origin of the target to be measured in the billboard pixel is compared, and the infrared radiation result is bound to the billboard pixel in a texture manner, so that each point on the billboard corresponds to each pixel of the infrared radiation result image. That is, the infrared radiation result image replaces the target to be measured in the billboard pixel according to its actual size, and the origin of the target to be measured corresponding to the infrared radiation result image coincides with the origin of the target to be measured in the billboard pixel.

[0081] The mapping method of infrared radiation result texture and billboard is shown in Table 1.

[0082] Table 1 Mapping method of infrared radiation result texture and billboard

[0083]

[0084] Intersection with bulletin board Texture coordinates After binding the infrared radiation result image to the four corners of the billboard surface element, the mapping calculation is performed. The method is:

[0085] .

[0086] S6. Calculate the infrared image data of the target to be measured using a ray tracing method.

[0087] By binding the infrared radiation results as texture maps in the bulletin board and then using ray tracing methods for infrared simulation, there is no need to track and calculate each grid, which greatly reduces the amount of simulation calculations and improves the efficiency of infrared simulation calculations.

[0088] In some embodiments, step S6 includes the following sub-steps:

[0089] When tracing rays, for each ray, we have:

[0090] S6.1. Obtain the intersection point between the light ray and the billboard pixel, and calculate the corresponding texture coordinates.

[0091] S6.2. Calculate the pixel position of the current intersection in the infrared radiation result (image) according to the texture coordinates and the image size of the infrared radiation result, and obtain the infrared radiation brightness value and transmittance of the pixel according to the infrared radiation result.

[0092] S6.3. Stop tracking when the transmittance is 0, and calculate the infrared radiation brightness value of the corresponding pixel of the detector based on the infrared radiation brightness value of the current pixel; continue forward tracking when the transmittance is not 0, and calculate the infrared radiation brightness value of the corresponding pixel of the detector based on the infrared radiation brightness value of the current pixel.

[0093] Taking into account the influence of the atmosphere, in some feasible implementations, the method for calculating the infrared radiation brightness value of the corresponding pixel of the detector according to the infrared radiation brightness value of the current pixel is:

[0094] ,

[0095] in, is the infrared radiation brightness value received by the detector; The target to be measured is in pixels s Infrared radiation brightness value; The target to be measured is in pixels s The transmittance of is the atmospheric transmittance between the target to be measured and the detector; The atmospheric radiation between the target to be measured and the detector; It is the atmospheric radiation behind the target to be measured along the direction of the light.

[0096] According to the embodiments of the present application, the target infrared radiation results are integrated into the complex scene to achieve the effect of regenerating infrared image data of the complex scene. This method can take into account the spatial position of each element in the complex scene, and can also superimpose the atmospheric transmission data into the scene. At the same time, combined with the bulletin board technology, in the process of ray tracing, there is no need to track and calculate a large number of volume grids, which can greatly improve the calculation efficiency. This method can not only take advantage of the third-party software to perform infrared simulation on the target to be measured, and reproduce complex scenes more realistically, but also improve the efficiency of infrared simulation calculation, and then more efficiently simulate and generate higher quality samples, and form a sample library, which better assists the training and testing of target recognition algorithms. This method has obvious advantages in sample library generation and has broad application prospects.

[0097] The present invention is not limited to the above-mentioned specific embodiments, but extends to any new features or any new combination disclosed in this specification, as well as any new method or process steps or any new combination disclosed.

Claims

1. A method for regenerating infrared image data of complex scenes, characterized in that: include: Establish a three-dimensional scene including the target to be measured and the detector; Calculating the position and posture of the target to be measured in the three-dimensional scene relative to the detector; According to the posture of the target to be measured, the infrared radiation result of the corresponding posture is matched from the intrinsic radiation library; The intrinsic radiation library is obtained by calculating the intrinsic radiation of the target to be measured from multiple angles; The infrared radiation result matched from the intrinsic radiation library is used to replace the target to be measured in the three-dimensional scene in the form of a bulletin board, including: calculating the image size of the infrared radiation result according to the number of pixels of the infrared radiation result matched from the intrinsic radiation library, the detection distance and the detection angle set during the simulation calculation; calculating the coordinates of the four corner points of the billboard surface element according to the coordinates of the target to be measured in the three-dimensional scene, and comparing the coordinates of the origin of the target to be measured in the billboard surface element according to the origin coordinates of the target to be measured corresponding to the infrared radiation result to the origin of the target to be measured in the billboard surface element, and binding the infrared radiation result to the billboard surface element in the form of a texture; A ray tracing method is used to calculate infrared image data of a target to be measured, including: during ray tracing, for each ray, obtaining the intersection of the ray and the face element of a bulletin board and calculating the corresponding texture coordinates; calculating the pixel position of the current intersection in the infrared radiation result according to the texture coordinates and the image size of the infrared radiation result, and obtaining the infrared radiation brightness value and transmittance of the pixel according to the infrared radiation result; stopping tracing when the transmittance is 0, and calculating the infrared radiation brightness value of the corresponding pixel of the detector according to the infrared radiation brightness value of the current pixel; and continuing forward tracing when the transmittance is not 0, and calculating the infrared radiation brightness value of the corresponding pixel of the detector according to the infrared radiation brightness value of the current pixel.

2. The complex scene infrared image data regeneration method according to claim 1, characterized in that: The method for generating the intrinsic radiation library comprises: Configure the angle parameters and distance parameters of the simulation calculation; According to the flow field data of the target to be measured, the infrared radiation results of detecting the target to be measured at various configured angles and distances are calculated respectively; The infrared radiation results of the target to be measured detected from various angles are summarized to generate an intrinsic radiation library.

3. The complex scene infrared image data regeneration method according to claim 2, characterized in that: The configured angle parameters of the simulation calculation include detection angles selected at multiple pitch angles and equal azimuth angle intervals with the target to be detected as the sphere center.

4. The complex scene infrared image data regeneration method according to claim 1, characterized in that: The step of establishing a three-dimensional scene including the target to be detected and the detector includes: Place the target to be measured in the three-dimensional scene and set the position and posture of the target to be measured; Place the detector in the 3D scene and set its position and pose.

5. The complex scene infrared image data regeneration method according to claim 1, characterized in that: The calculating the position and posture of the target to be measured in the three-dimensional scene relative to the detector includes: Establish the detector coordinate system; According to the poses of the target and the detector in the three-dimensional scene coordinate system, the pose of the target in the detector coordinate system is calculated.

6. The complex scene infrared image data regeneration method according to claim 1, characterized in that: The method for calculating the infrared radiation brightness value of the corresponding pixel of the detector according to the infrared radiation brightness value of the current pixel is: ， in, is the infrared radiation brightness value received by the detector; The target to be measured is in pixels s Infrared radiation brightness value; The target to be measured is in pixels s The transmittance of is the atmospheric transmittance between the target to be measured and the detector; The atmospheric radiation between the target to be measured and the detector; It is the atmospheric radiation behind the target to be measured along the direction of the light.

7. A storage medium storing a computer program, characterized in that: Running the computer program can execute the complex scene infrared image data regeneration method as described in any one of claims 1-6.

8. A complex scene infrared image data regeneration device, characterized in that: It comprises a processor and a storage medium as claimed in claim 7, wherein the processor is used to run a computer program in the storage medium to execute a complex scene infrared image data regeneration method as claimed in any one of claims 1-6.

Images