Infrared simulation data and label automatic generation method based on virtual engine

Abstract

The invention relates to an infrared simulation data and label automatic generation method based on a virtual engine, and belongs to the field of data generation. Comprising the steps that a scene where data need to be acquired is set up in a Blender, 3D modeling is carried out, and multiple frames of animation are set to be used for obtaining data at different angles and in different scenes; modeling an object thermal radiation value received by the sensor by using a Planck black body radiation law, and calculating to obtain a thermal radiation self-emission value of a required material in an infrared band in a scene; based on the thermal radiation self-emission value, an Infrared-Group node combination is set for each material, and the infrared radiation effect of an object is generated through Infrared-Group control; correcting infrared radiation by arranging a proximity sensor; rendering through a ray tracing technology to obtain infrared data; nodes are built in the virtual engine to generate labels suitable for scene segmentation, target detection and depth estimation. Through simple mathematical modeling and innovative application of a virtual engine, high-efficiency generation of vivid and high-diversity infrared simulation data is successfully realized.

Description

Technical Field

[0001] The invention belongs to the field of data generation, and in particular relates to a method for automatically generating infrared simulation data and labels based on a virtual engine. Background Art

[0002] In recent years, deep learning tasks based on infrared data have been widely used in security, aerospace and other fields. However, the high cost and technical difficulty of obtaining real infrared data have made virtual data gradually become an important data source for infrared deep learning model training.

[0003] At present, the methods for obtaining infrared virtual data are mainly divided into three categories: modeling conversion technology based on grayscale images to infrared images, infrared data generation methods based on generative models, and infrared data collection based on virtual environments built by simulation software. The first type of modeling conversion-based methods maps visible light data to infrared images through mathematical models. Although images with high physical consistency can be generated, they rely on complex physical models and have high computational costs. Due to the lack of accuracy in material classification and sensor parameter simulation in the scene, it is difficult to capture the complex infrared radiation characteristics in the real world. The second type of method is based on generative models. The current commonly used method is style transfer, which converts visible light images into infrared images. This method has advantages in generating edge information, but usually ignores the underlying physical laws, resulting in the generated images may lack the physical characteristics of real infrared scenes. In addition, style transfer is highly dependent on paired visible light-infrared datasets, which are extremely scarce, so directly generating infrared data becomes a feasible method. However, this direct generation method still requires a large number of infrared datasets to train the network, which is in conflict with the current situation of limited infrared data resources. The last method collects infrared data by building specific scenes through simulation software, which significantly improves the diversity of data and shows good applicability in downstream tasks. However, this method has extremely high requirements on the quality of the generated data, and relies on the mathematical modeling of infrared radiation and the ability of simulation software to represent infrared characteristics. For example, the thermal radiation of actual objects is usually uneven, while some simulation methods may present a uniform radiation effect, resulting in differences from the actual situation. But if this problem can be overcome, the problem of small amount of infrared data and insufficient data richness can be fundamentally alleviated.

[0004] In summary, when the amount of infrared data is limited, using simulation software to build scenes and generate rich data sets is currently an ideal solution. However, existing simulation methods have significant limitations: one method directly uses the material temperature as the infrared radiation value of the object, which makes the generated infrared image similar to a segmentation map and lacks texture details; another method directly assigns the thermal emissivity of the material, resulting in a stiff image with insufficient physical meaning. These problems affect the fidelity of the simulation data.

[0005] Therefore, existing infrared simulation methods still face challenges in terms of stability, physical authenticity, and computational cost. In order to solve these problems, a more realistic, efficient, and physically consistent infrared simulation data generation technology is urgently needed to provide reliable data support for the training and application of deep learning models. In this context, the method based on Blender simulation software shows potential. Through its flexible physical modeling capabilities, it is expected to balance physical authenticity and computational efficiency in infrared data generation. Summary of the invention

[0006] The technical problems to be solved by the present invention are:

[0007] In order to solve the above-mentioned problems arising in the infrared data simulation process, the present invention provides a method for automatically generating infrared simulation data and labels based on a virtual engine, which provides a more reliable data basis for the training and application of deep learning models.

[0008] In order to solve the above technical problems, the technical solution adopted by the present invention is:

[0009] A method for automatically generating infrared simulation data and labels based on a virtual engine, characterized in that the steps are as follows:

[0010] Step 1: Build the scene for which data needs to be obtained in Blender, perform 3D modeling, and set up multi-frame animation to obtain data from different angles and scenes;

[0011] Step 2: Use Planck's blackbody radiation law to model the thermal radiation value of the object received at the sensor, and calculate the thermal radiation self-emission value of the required material in the infrared band in the scene;

[0012] Step 3: Set the Infrared-Group node combination for each material based on the thermal radiation self-emission value, and use the Infrared-Group to control the infrared radiation effect of the generated object; correct the infrared radiation by setting the proximity sensor; and render the infrared data through ray tracing technology;

[0013] Step 4: Build nodes in the Unreal Engine to generate labels suitable for scene segmentation, object detection, and depth estimation.

[0014] A further technical solution of the present invention: In step 2, Planck's blackbody radiation law is used to model the thermal radiation value of the object received at the sensor, and the modeling formula is as follows:

[0015]

[0016] Where L is the thermal radiation emission value, T is the material temperature, h is Planck's constant, c is the speed of light, λ is the wavelength, k is the Boltzmann constant, εavg is the average emissivity of the material.

[0017] A further technical solution of the present invention: In step 2, the self-emission value of the thermal radiation of the required material in the infrared band under the calculated scenario is specifically:

[0018] The long-wave infrared band is divided into multiple bands, and the thermal radiation emission values in multiple bands are calculated respectively.

[0019] A further technical solution of the present invention: The proximity sensor described in step 3 adopts the following formula:

[0020]

[0021] d(X,X s )=||X-X s ||

[0022] where L0 represents the thermal radiation value at point X, d represents the distance from point X s to point X, X s is any point other than point X, R represents the range affected by the heat source at point X, and α1 and α2 respectively represent attenuation factors;

[0023] When d(X,Xs)<R, the distance between the sensor and the heat source is relatively close, and the radiation intensity decays slowly with the increase of the distance. This situation is described by a smaller attenuation factor to reflect the sensitivity of the sensor to radiation within a short distance; when d(X,Xs)>R, the distance exceeds the detection range, and the radiation intensity decays rapidly. This situation is described by a larger attenuation factor to simulate the significant decrease in the sensor's perception of radiation at a long distance.

[0024] A further technical solution of the present invention: For each material, an Infrared-Group node combination is set based on the self-emission value of thermal radiation, including:

[0025] The calculated thermal radiation emission value is normalized, with the lowest value of the thermal radiation emission value in the scenario set to 0 and the highest value set to 1;

[0026] The normalized thermal radiation emission value is read into the layer weight through a script, and the layer weight determines the self-emission value of the material by controlling the color bar.

[0027] A further technical solution of the present invention: The infrared data is rendered through ray tracing technology, specifically:

[0028] L0(X,ω0,λ,t)=L(X,ω0,λ,t)+∫ Ω f r (X,ω i ,ω0,λ,t)L i(X,ω i ,λ,t)(ω i ·n)dω i

[0029] Among them, L0(X,ω0,λ,t) on the left side of the equal sign represents the light emitted from point x to the sensor ω0 direction, the first term L(X,ω0,λ,t) on the right side of the equal sign represents the self-luminescence of point x, and the second term ∫ Ω f r (X,ω i ,ω0,λ,t)L i (X,ω i ,λ,t)(ω i ·n)dω i It represents the sum of all incident light received by point x and reflected to the direction of ω0.

[0030] A further technical solution of the present invention is as follows: Building nodes in the virtual engine to generate labels suitable for scene segmentation, target detection, and depth estimation is specifically as follows:

[0031] After rendering is completed, the virtual engine automatically creates a rendering layer node in the node workspace, opens the relevant output channels of the rendering layer node for different labels, builds a new functional node in the synthesis workspace to obtain the corresponding physical properties, connects the relevant output channels of the rendering layer node with the functional node, and renders the output to obtain the label data.

[0032] A method for verifying the performance of generated simulated infrared data and tags, characterized by comprising:

[0033] Build pre-trained deep learning models;

[0034] Pair the generated simulated infrared data with the corresponding labels and input them into the pre-trained deep learning model for training;

[0035] Obtain real infrared data, input the real infrared data into the trained deep learning model to output the corresponding label, and compare the output corresponding label with the real label to verify the performance.

[0036] The deep learning model adopts the Yolov8 target detection model.

[0037] A computer system, characterized in that it comprises: one or more processors, and a computer-readable storage medium for storing one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors implement the above-mentioned method.

[0038] The beneficial effects of the present invention are:

[0039] The present invention provides a method for automatically generating infrared simulation data and labels based on a virtual engine. The method proposes a way to express the characteristics of infrared data in Blender, that is, by building an Infrared-Group node group to achieve the effect of an infrared correction equation. This node group corrects the infrared radiation data received by the sensor so that the generated data is closer to the actual situation, thereby improving the physical authenticity of the simulation data. The method is based on relatively simple mathematical modeling to achieve more realistic and physically meaningful infrared data simulation using a virtual engine. By quickly and massively collecting data and its corresponding various labels in a scripted manner, the diversity of infrared data can be effectively expanded, the problem of scarcity of infrared data can be solved, thereby improving the performance of deep learning models and providing a wider range of application possibilities in diverse scenarios.

[0040] The method of the present invention solves the problems that arise in the existing infrared data collection process. Through simple mathematical modeling and innovative use of virtual engines, it successfully achieves efficient generation of realistic and diverse infrared simulation data, providing a more reliable data foundation for deep learning models. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] The drawings are only for the purpose of illustrating particular embodiments and are not to be considered limiting of the present invention. Like reference symbols denote like components throughout the drawings.

[0042] Figure 1 It is the overall framework of the present invention.

[0043] Figure 2 This is the Infrared-Group structure diagram.

[0044] Figure 3 The following is a comparison of the effects of using and not using the Infrared-Group structure: (a) not using the Infrared-Group structure; (b) using the Infrared-Group structure; (c) real infrared data samples.

[0045] Figure 4 It is a structural diagram of generating segmentation Mask labels in an example of the present invention.

[0046] Figure 5 It is a structural diagram of generating a depth map label according to an example of the present invention.

[0047] Figure 6 It is a partial schematic diagram of infrared simulation data collected by the present invention.

[0048] Figure 7These are three types of label schematic diagrams collected by the present invention: (a) a schematic diagram of a segmentation mask in an example of the present invention; (b) a schematic diagram of a bounding box in an example of the present invention; (c) a schematic diagram of a depth map in an example of the present invention.

[0049] Figure 8 It is a schematic diagram of the result of target detection of the example of the present invention: (a) F-Tank is used as a training set to detect simple data; (b) B-Tank is used as a training set to detect simple data; (c) is used as a training set to detect complex data; (d) B-Tank is used as a training set to detect complex data. DETAILED DESCRIPTION

[0050] In order to make the purpose, technical solutions and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention. In addition, the technical features involved in the various embodiments of the present invention described below can be combined with each other as long as they do not conflict with each other.

[0051] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the terms used in this way can be interchangeable where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0052] The present invention provides a method for automatically generating infrared simulation data and labels based on the Blender virtual engine, and uses downstream tasks to prove the effectiveness of simulation data in improving the performance of deep learning models. The method includes the following steps:

[0053] Step 1: Build the scene for which data needs to be obtained in Blender, perform 3D modeling, and set up multi-frame animation to obtain data from different angles and scenes.

[0054] Step 2: Use Planck's blackbody radiation law to model the thermal radiation value of the object received at the sensor, and calculate the thermal radiation self-emission value of the required material in the infrared band in the scene;

[0055] Step 3, set the infrared correction effect, set the Infrared-Group node combination for each material, use Infrared-Group to control the infrared radiation effect of the generated object, make the infrared effect more realistic and explainable, and render the infrared data through ray tracing technology. At the same time, use Brush to select special areas (areas with higher temperatures) and add Proximity effects to control the influence of special areas on the adjacent areas of the surface. It is used to simulate the effect of the infrared correction equation. In this method, it is called a neighbor sensor.

[0056] Step 4: Build nodes in the virtual engine to generate labels suitable for scene segmentation, object detection, and depth estimation.

[0057] Step 5: Use the collected simulated infrared data and labels in combination with real data to perform model training for downstream tasks, proving that the infrared simulation data collected by this method has a significant effect in improving the performance of deep learning model tasks.

[0058] In step 1, 3D modeling is performed according to the required task scenario, and animation is produced in the scenario according to the task to obtain simulation data with richer scenarios and larger quantities.

[0059] In step 2, Planck's blackbody radiation law is used to calculate the thermal radiation emission value of the material, and the heat radiated by the object is quantified into the thermal radiation emission value. The modeling formula is as follows:

[0060]

[0061] Where L is the thermal radiation emission value [W / m 2 / sr], T is the material temperature [K], h is Planck's constant, c is the speed of light, λ is the wavelength [μm], 7-13μm is selected as the long-wave infrared band, k is the Boltzmann constant, ε avg It is the average emissivity of the material, a value between 0 and 1, which relates the radiation of real objects to the radiation of black bodies. The thermal radiation emission values ​​of materials at different temperatures in the long-wave infrared band can be obtained through Planck's blackbody radiation equation.

[0062] In the calculation process of step 2, the band 7-13 μm is divided into 3n, (n∈{1,2,...18}) bands, and the thermal radiation emission values ​​in these bands are calculated respectively.

[0063] The infrared correction equation used in step 3 is:

[0064]

[0065] d(X,X s )=||XX s ||

[0066] Among them, L0 represents the thermal radiation value at point X, d represents the distance from point X s to point X, and X s is any point other than point X, R represents the range affected by the heat source at point X, and α1 and α2 respectively represent the attenuation factors;

[0067] This formula is used to correct the radiation intensity detected by the sensor to reflect the influence of the distance between the sensor and the heat source on radiation attenuation. Specifically, the following two situations are considered:

[0068] 1. When d(X, X_s) < R, the distance between the sensor and the heat source is relatively close, and the radiation intensity decays slowly as the distance increases. This situation is described by a smaller attenuation factor to reflect the high sensitivity of the sensor to radiation within a short distance.

[0069] 2. When d(X, X_s) > R, the sensor is far from the heat source, and the radiation intensity decays rapidly. This situation uses a larger attenuation factor to simulate the significant decrease in the sensor's perception of radiation at a long distance. For example, in the area adjacent to the high radiation rate area, the radiation value does not drop suddenly but shows a slow decreasing trend. This method not only accurately reflects the spatial distribution of thermal radiation but also simulates the gradual change effect of thermal radiation between different materials, thereby improving the realism and physical consistency of the simulation image.

[0070] Subsequently, an Infrared_Group node group is added to each material. This node group is used to normalize the thermal radiation brightness in the scene, set the self-emission value of the pixel point with the lowest thermal radiation energy to 0, and the highest to 1. Then, the thermal radiation emission value calculated in step 1 is read into the layer weights, and these weights control the self-emission value of the material through the color bar. Finally, infrared data is obtained through ray tracing rendering, and the formula for ray tracing is as follows:

[0071] L0(X, ω0, λ, t) = L(X, ω0, λ, t) + ∫ Ω f r (X, ω i , ω0, λ, t)L i (X, ω i , λ, t)(ω i ·n)dω i

[0072] Among them, L0(X, ω0, λ, t) on the left side of the equal sign represents the light emitted from point x in the direction of the sensor ω0, the first term L(X, ω0, λ, t) on the right side of the equal sign represents the self-luminescence of point x, and the second term ∫ Ω f r (X, ω i , ω0, λ, t)L i(X,ω i ,λ,t)(ω i ·n)dω i It represents the sum of all incident light received by point x and reflected to the direction of ω0.

[0073] Step 4: After the rendering in step 3 is completed, the virtual engine automatically creates a rendering layer node in the node workspace, opens the relevant output channels of the rendering layer node for different labels, builds a new functional node in the synthesis workspace to obtain the corresponding physical properties, connects the relevant output channels of the rendering layer node with the functional node, and renders the output to obtain the label data.

[0074] Step 5: Perform model training for downstream tasks to verify whether simulated infrared data can improve performance in deep learning tasks. The specific steps are as follows:

[0075] Step 5.1: Select appropriate deep learning models and real datasets for specific downstream tasks.

[0076] Step 5.2: Pair the generated simulated infrared data with the corresponding labels to construct a data set for model training. In this process, the experimental data and control data are reasonably set, and conclusions are drawn through comparative experiments.

[0077] The present invention provides a method for generating infrared simulation data based on a virtual engine, which solves the problems that arise in the existing infrared data collection process. Through simple mathematical modeling and innovative use of virtual engines, the efficient generation of realistic and diverse infrared simulation data is successfully achieved, providing a more reliable data foundation for deep learning models.

[0078] In order to enable those skilled in the art to better understand the present invention, the present invention is described in detail below in conjunction with specific embodiments.

[0079] Example 1

[0080] like Figure 1 As shown, the present invention provides a method for automatically generating infrared simulation data and labels based on a virtual engine, and the steps are as follows:

[0081] Step 1: Use the virtual engine Blender to build a 3D scene and set the animation.

[0082] This example simulates a tank driving in a mountainous terrain. The scene contains two basic elements: mountain scene and tank. A total of 1000 frames of animation are set in the example, from the 1st frame to the 1000th frame, to complete a tank driving animation.

[0083] Step 2: Preset the materials that will appear in the scene and calculate the thermal radiation emission value of the materials. Figure 7The street scene in the example (a) contains 4 objects: 1) Tank: including materials such as Al2O3 and paint; 2) Mountain: including materials such as grass, soil, and rocks; 3) Wooden bridge: including materials such as wood and paint; 4) Lake. The thermal radiation emission values ​​of these materials in the 7-13μm band are calculated using Planck's blackbody radiation law.

[0084]

[0085] Taking the material Al2O3 as an example, the temperature of the material in this scene is set to 20 degrees Celsius, that is, T = 293.15K, and the band is divided into 54 segments. Each segment is calculated separately, and the partial results of the thermal radiation emission value of aluminum in 54 bands are shown in Table 1:

[0086] Table 1 Thermal radiation values ​​of Al2O3 material in 54 bands in this example

[0087] Band 7-7.11 7.11-7.22 7.22-7.33 ... 12.61-12.72 12.72-12.83 12.83-12.94 Emissivity 0.8111 0.9761 0.9689 ... 0.9500 0.9738 0.9565 Emitted Value 0.1304 0.1303 0.1284 ... 0.0499 0.0487 0.0480

[0088] Step 3: Set the infrared effect of the scene. First, use the script to build the Infrared-Group node group. The node group structure is as follows: Figure 2 As shown; secondly, the emission values ​​of the material obtained in step 2 in 54 bands are averaged every 3 times to obtain 18 emission values, which are read into the layer weight through the script. The layer weight will generate 18 color controllers, making the material have a unique luminous style. This step simulates the physical behavior of infrared objects radiating heat in the real world; secondly, by refining the material, the luminous part of the object can be controlled more accurately; by proximity to the sensor, the surrounding area of ​​the brighter luminous part (the part with higher temperature) will show a slow luminous decrease (heat decrease) instead of a sudden decrease. Finally, the infrared data is rendered, and the effect is as follows Figure 6 shown.

[0089] Figure 3 The difference in simulation effect between using and not using the Infrared-Group node group is shown in Figure 3(a). Infrared-Group is not used, and the material emission value is directly assigned to the material's self-luminous property. It can be seen that the luminous intensity of the entire Tank is very uniform, which cannot well represent the physical meaning of infrared. Figure 3(b) shows the effect of using Infrared-Group. It can be seen that the thermal radiation emission value of the background object is low (low brightness), and the thermal radiation emission value of the tank is high (high brightness). In addition, the emission value of the part of the tank close to the engine is higher (brighter), and the emission value at the edge is lower (darker).

[0090] Step 4: In this example, nodes are built to generate labels for scene segmentation, target detection, and depth estimation. The specific examples are:

[0091] The method to obtain the Mask label based on object segmentation is as follows: Create a unique ID for the objects used in the scene built in step 1, such as Figure 7 As shown in (a), in this example, 1) Tank, 2) Mountain, 3) Wooden Bridge, and 4) Lake are matched one by one. Figure 4 As shown, the following operations are completed using the script: Open the rendering layer object ID output channel, add the Normalize Node, Combine Color Node, and Composite Node. Connect the rendering layer material ID output to the Normalize Node input to map the rendering layer output pixel value to between 0 and 1. Connect the Normalize Node output to the input of the Hue parameter in the Combine Color Node, and adjust the saturation (S) and brightness (V) parameters in the Combine Color Node between 0 and 1 to obtain the ideal output color effect. In this example, the S and V values ​​in the Combine Color Node are 0.82 and 0.90 respectively. The Mask label obtained based on object segmentation is shown below. Figure 7 As shown in (a), the same objects in the label will be assigned the same color block, while different objects have different color blocks to distinguish their materials.

[0092] The Mask labeling method based on material segmentation is as follows: In the example, it is divided into 1) Al2O3, paint, etc. 2) grass, soil, rock, etc. 3) wood, 4) water. Figure 4 As shown, open the material ID channel of the rendering layer and create the same node as in the above object segmentation Mask label step for output. The resulting Mask label effect is that objects with different ID numbers are assigned different colors to distinguish the objects.

[0093] Generate the Bounding Box label for target detection, obtain the category and coordinates of the object of interest and output them in YOLO format, that is, output the coordinates of the upper left corner and lower right corner of the Bounding Box. In this example, only the Tank class is concerned, and the output YOLO format label is:

[0094] Tank <371> <330> <1188> <907>

[0095] Visualize the labels onto the infrared data map, such as Figure 7 (b) as shown.

[0096] Generate depth map labels: Figure 5Open the render layer depth output channel as shown, add the Normalize Node, Despeckle Node, Anti-Aliasing Node, Composite Node Node). Connect the render layer depth channel output to the normalization node so that the render layer output pixel value is mapped between 0 and 1. Connect the normalization node output to the denoising node input. There are three adjustable parameters in the denoising node, threshold, neighbor term, and coefficient, which are used to adjust the denoising effect. The threshold parameter is used to control the high / low complexity pixels of the image, and the neighbor term is used to control the range of neighboring pixels that match the pixel. In this example, the threshold is 0.2, the neighbor term is 0.5, and the coefficient is 1. Connect the denoising node output to the anti-aliasing node input. The denoising node is used to eliminate distortion artifacts that appear around jagged edges. The anti-aliasing node effect is controlled by three parameters, threshold, contrast limit, and corner rounding. The threshold is used to control the edge detection sensitivity of the entire image; the contrast limit is used to control the contrast level to be considered when detecting edges; the corner rounding is used to help preserve the original shape. The higher the value, the better the corners will be preserved, that is, similar to the original image. In this example, the threshold is 0.7, the contrast limit is 0.2, and the corner rounding is 0.6. Figure 7 (c) shows the depth estimation map in a street scene.

[0097] Step 5: In this example, target detection is used as a downstream task to verify the effectiveness of the simulated infrared dataset. The specific steps are as follows:

[0098] Step 5.1: In this example, the Yolov8 target detection architecture is used to detect the Tank class.

[0099] Step 5.2: The data set used for target detection is the simulated infrared data B-Tank collected by Blender in this example and the Tank data used for comparison, including 1) the real measured infrared Tank data R-Tank such as Figure 3 (c) as shown; 2) the simulation data F-Tank used for training, as shown Figure 3(a) As shown. Due to the high cost of collecting infrared tank target data, a large number of combat tests are required to obtain sufficient data. Therefore, the R-Tank data set has only 157 frames, while the F-Tank has a total of 429 frames. The collection method is to simply assign the material emission value to the self-luminous attribute. The data scene includes common soil, vegetation, rocks, water, and other environments. In this example, Blender obtains a total of 20,000 frames of B-Tank, which are divided into training set and validation set in a ratio of 8:2. The F-Tank data is also divided into training set and validation set in a ratio of 8:2. All real infrared data are used as test sets. Our goal is to prove that when the number of simulated infrared data is large enough and the effect is realistic enough, a good performance migration effect can be achieved on the real data set.

[0100] The large amount of simulated infrared data collected in steps 4 and 5 is used for target detection training, and then evaluated on real Tank data. The target detection experiment design is shown in Table 2, and the experimental results are shown in Table 2. From the target detection results, it can be seen that as the amount of Blender simulated data increases, the accuracy of the detection results on the real data gradually increases. Visualization Figure 8 As shown in the figure, it can be seen that the addition of Blender data greatly improves the performance of the object detection network.

[0101] Table 2 Target detection experiment design

[0102]

[0103] Table 3. Target detection experimental results

[0104]

[0105] The following conclusions can be drawn from the results in Table 3: As the amount of Blender simulation data increases, the Recall of experimental group 2 (Blender_10000) and experimental group 3 (Blender_20000) increases to 0.433 and 0.526, respectively. This shows that large-scale Blender simulation data improves the recall rate of the model in the target detection task, especially when the amount of data reaches 20,000, the Recall nearly doubles, proving the significant impact of data scale on model performance. In complex scenes (36 samples under conditions such as complex backgrounds or occlusions), the Recall of the control group HW_429 is 0.0811, which is lower than the overall Recall, indicating that complex scenes increase the recognition difficulty of the model. As the training data increases to 10,000 and 20,000, the Recall of experimental group 5 and experimental group 6 increases to 0.359 and 0.470, respectively. This shows that in complex scenes, the model's ability to recognize targets improves significantly as the size of the Blender dataset increases, especially when the data volume reaches 20,000, the Recall is close to 0.5, indicating that the model is more robust under complex conditions.

[0106] The above description is only a specific implementation mode of the present invention, but the protection scope of the present invention is not limited thereto. Any technician familiar with the technical field can easily think of various equivalent modifications or substitutions within the technical scope disclosed in the present invention, and these modifications or substitutions should be included in the protection scope of the present invention.

Claims

1. A method for automatically generating infrared simulation data and labels based on a virtual engine, characterized in that: The steps are as follows: Step 1: Build the scene for which data needs to be obtained in Blender, perform 3D modeling, and set up multi-frame animations to obtain data at different angles and in different scenarios; Step 2: Model the thermal radiation value of the object received at the sensor using Planck's blackbody radiation law, and calculate the thermal radiation self-emission value of the required materials in the infrared band in the scene; Step 3: Set up Infrared-Group node combinations for each material based on the thermal radiation self-emission value, and use Infrared-Group to control the generation of the infrared radiation effect of the object; correct the infrared radiation through the set proximity sensor; render to obtain infrared data through ray tracing technology; Step 4: Build nodes in the virtual engine to generate labels suitable for scene segmentation, object detection, and depth estimation.

2. The method for automatically generating infrared simulation data and labels based on a virtual engine according to claim 1, characterized in that: In Step 2, the thermal radiation value of the object received at the sensor is modeled using Planck's blackbody radiation law, and the modeling formula is as follows: Where L is the thermal radiation emission value, T is the material temperature, h is Planck's constant, c is the speed of light, λ is the wavelength, k is the Boltzmann constant, ε avg is the average emissivity of the material.

3. The method for automatically generating infrared simulation data and labels based on a virtual engine according to claim 2, characterized in that: In Step 2, the thermal radiation self-emission value of the required materials in the infrared band in the scene is calculated. Specifically: the long-wave infrared band is divided into multiple bands, and the thermal radiation emission values in multiple bands are calculated separately.

4. The method for automatically generating infrared simulation data and labels based on a virtual engine according to claim 1, characterized in that: The proximity sensor described in Step 3 uses the following formula: d(X,X s )=||X-X s || Among them, L0 represents the thermal radiation value at point X, d represents X s The distance from point to point X, X s is any point except point X, R represents the range of heat source influence at point X, α1 and α2 represent attenuation factors respectively; When d(X,Xs)<R, the distance between the sensor and the heat source is relatively close, and the radiation intensity decays slowly with the increase of the distance. This situation is described by a smaller attenuation factor to reflect the sensitivity of the sensor to radiation at close range; when d(X,Xs)>R, the distance exceeds the detection range, and the radiation intensity decays rapidly. This situation is described by a larger attenuation factor to simulate the significant decrease in the sensor's perception of radiation at long distances.

5. The method for automatically generating infrared simulation data and labels based on a virtual engine according to claim 1, characterized in that: Setting up Infrared-Group node combinations for each material based on the thermal radiation self-emission value includes: Normalize the calculated thermal radiation emission value, set the lowest value of the thermal radiation emission value in the scene to 0, and the highest value to 1; Read the normalized thermal radiation emission value into the layer weight through a script, and the layer weight will determine the self-emission value of the material by controlling the color bar.

6. The method for automatically generating infrared simulation data and labels based on a virtual engine according to claim 1, characterized in that: Rendering to obtain infrared data through ray tracing technology specifically means: L0(X,ω0,λ,t)=L(X,ω0,λ,t)+∫ Ω f r (X,ω i ,ω0,λ,t)L i (X,ω i ,λ,t)(ω i ·n)dω i Among them, L0(X,ω0,λ,t) on the left side of the equal sign represents the light emitted from point x to the sensor ω0 direction, the first term L(X,ω0,λ,t) on the right side of the equal sign represents the self-luminescence of point x, and the second term ∫ Ω f r (X,ω i ,ω0,λ,t)L i (X,ω i ,λ,t)(ω i ·n)dω i It represents the sum of all incident light received by point x and reflected to the direction of ω0.

7. The method for automatically generating infrared simulation data and labels based on a virtual engine according to claim 1, characterized in that: Building nodes in the virtual engine to generate labels suitable for scene segmentation, object detection, and depth estimation specifically means: After rendering, the virtual engine automatically creates a render layer node in the node workspace. For different labels, open the relevant output channels of the render layer node, and build new functional nodes in the composite workspace to obtain the corresponding physical properties, and connect the relevant output channels of the render layer node to the functional nodes to render and output the label data.

8. A method for verifying the performance of a simulated infrared data and a tag generated based on the method described in any one of claims 1 to 7, characterized in that: Including: Construct a pre-trained deep learning model; Pair the generated simulated infrared data with the corresponding labels, and input them into the pre-trained deep learning model for training; Obtain real infrared data, input the real infrared data into the trained deep learning model to output the corresponding labels, and compare the output corresponding labels with the real labels to verify the performance.

9. A method for verifying the performance of simulated infrared data and tags according to claim 8, characterized in that: The deep learning model uses the Yolov8 object detection model.

10. A computer system, characterized in that Including: One or more processors, and a computer-readable storage medium for storing one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors implement the method described in any one of claims 1-9.

Images