Infrared simulation data and label automatic generation method based on virtual engine

Abstract

The present invention relates to a method for automatically generating infrared simulation data and labels based on a virtual engine, and belongs to the field of data generation. The method comprises: building a scene in Blender where data needs to be obtained, performing 3D modeling, and setting a multi-frame animation to obtain data at different angles and in different scenes; using Planck's blackbody radiation law to model the thermal radiation value of the object received at the sensor, and calculating the thermal radiation self-emission value of the required material in the scene in the infrared band; setting an Infrared-Group node combination for each material based on the thermal radiation self-emission value, and using Infrared-Group to control the infrared radiation effect of the generated object; correcting the infrared radiation by setting a proximity sensor; obtaining infrared data by rendering through ray tracing technology; and building nodes in the virtual engine to generate labels suitable for scene segmentation, target detection, and depth estimation. Through simple mathematical modeling and the innovative use of the virtual engine, the efficient generation of realistic and diverse infrared simulation data is successfully achieved.

Description

TECHNICAL FIELD

[0001] The application belongs to the field of data generation, and particularly relates to an infrared simulation data and label automatic generation method based on a virtual engine. BACKGROUND

[0002] In recent years, deep learning tasks based on infrared data have been widely applied in the fields of security and aerospace. However, the cost of obtaining real infrared data is high and the technical difficulty is great, so virtual data has gradually become an important data source for training infrared deep learning models.

[0003] Currently, the methods for obtaining infrared virtual data mainly fall into three categories: modeling conversion technology based on grayscale images to infrared images, infrared data generation methods based on generative models, and collecting infrared data by constructing virtual environments based on simulation software. The first method based on modeling conversion maps visible light data to infrared images through mathematical models. Although this method can generate images with high physical consistency, it relies on complex physical models and has high computational cost. Due to the lack of accuracy in material classification and sensor parameter simulation in the scene, it is difficult to capture the complex infrared radiation characteristics in the real world. The second method is based on generative models. The current common way is style transfer, which converts visible light images to infrared images. This method has an advantage in generating edge information, but it usually ignores the underlying physical laws, resulting in generated images that may lack the physical characteristics of real infrared scenes. In addition, style transfer is highly dependent on paired visible light-infrared datasets, and such paired data is extremely scarce, so directly generating infrared data becomes a viable way. However, this direct generation method still requires a large amount of infrared dataset to train the network, which contradicts the current situation of limited infrared data resources. The third method collects infrared data by constructing specific scenes through simulation software, significantly improving the diversity of the data and showing good applicability in downstream tasks. However, this method has high requirements for the quality of generated data and relies on the performance of infrared radiation mathematical modeling and simulation software in representing infrared characteristics. For example, the thermal radiation of actual objects is usually not uniform, while some simulation methods may present uniform radiation effects, leading to differences from the real situation. However, if this problem can be overcome, the problem of insufficient infrared data quantity and richness can be fundamentally alleviated.

[0004] In summary, in the case of limited infrared data, using simulation software to construct scenes to generate rich datasets is currently the ideal solution. However, existing simulation methods have significant limitations: one method directly uses material temperature as the infrared radiation value of the object, which makes the generated infrared images similar to segmentation maps and lacks texture details; another method directly assigns the thermal emissivity of materials, resulting in images that appear rigid and lack physical meaning. These problems all affect the realism of the simulation data.

[0005] Therefore, the existing infrared simulation method still faces challenges in stability, physical authenticity and computational cost. In order to solve these problems, there is an urgent need for an infrared simulation data generation technology that is more realistic, efficient and physically consistent, providing reliable data support for the training and application of deep learning models. In this context, the method based on Blender simulation software shows potential, as it can balance physical authenticity and computational efficiency in infrared data generation through its flexible physical modeling capabilities. SUMMARY

[0006] The technical problem to be solved by the present application is:

[0007] To solve the above problems in the process of infrared data simulation, the present application provides a method for automatically generating infrared simulation data and labels based on a virtual engine, providing a more reliable data foundation for the training and application of deep learning models.

[0008] To solve the above technical problems, the technical solution adopted by the present application is:

[0009] A method for automatically generating infrared simulation data and labels based on a virtual engine, characterized by the following steps:

[0010] Step 1: Build the scene in Blender where data needs to be obtained, perform 3D modeling, and set up multiple frames of animation to obtain data at different angles and in different scenes.

[0011] Step 2: Model the thermal radiation value of the object received at the sensor using Planck's blackbody radiation law, and calculate the thermal radiation self-emission value of the required material in the infrared band under the scene.

[0012] Step 3: Set up an Infrared-Group node combination for each material based on the thermal radiation self-emission value, use Infrared-Group to control the generation of infrared radiation effects for objects, correct infrared radiation by setting up nearby sensors, and render infrared data through ray tracing technology.

[0013] Step 4: Build nodes in the virtual engine to generate labels suitable for scene segmentation, object detection and depth estimation.

[0014] Further technical solutions of the present application: In step 2, the thermal radiation value of the object received at the sensor is modeled using Planck's blackbody radiation law, and the modeling formula is as follows:

[0015]

[0016] Where L is the thermal radiation emission value, T is the material temperature, h is the Planck constant, c is the speed of light, λ is the wavelength, k is the Boltzmann constant, and ε is the emissivity.avg is the average emissivity of the material.

[0017] A further technical solution of the present invention is: in step 2, the thermal radiation self-emission value of the required material in the infrared band under the scene is calculated, specifically:

[0018] The long-wave infrared band is divided into multiple bands, and the thermal radiation emission values ​​in multiple bands are calculated respectively.

[0019] A further technical solution of the present invention: The proximity sensor in step 3 adopts the following formula:

[0020]

[0021] d(X,X s )=||XX s ||

[0022] Among them, L0 represents the thermal radiation value at point X, d represents the s The distance from point to point X, X s is any point except point X, R represents the range of heat source influence at point X, α1 and α2 represent attenuation factors respectively;

[0023] When d(X,Xs) <R时,传感器和热源之间的距离较近,辐射强度随距离增加而衰减较慢,这种情况通过较小的衰减因子来描述,以反映近距离内传感器对辐射的敏感度;当d(X,Xs)> When the distance exceeds the detection range, the radiation intensity decays faster. This situation is described by a larger attenuation factor, simulating a significant decrease in the sensor's perception of radiation at long distances.

[0024] A further technical solution of the present invention is that the step of setting an Infrared-Group node combination for each material based on the thermal radiation self-emission value includes:

[0025] Normalize the calculated thermal radiation emission values, setting the lowest thermal radiation emission value in the scene to 0 and the highest thermal radiation emission value to 1;

[0026] The normalized thermal radiation emission value is read into the layer weight through the script, and the layer weight determines the material's self-emission value by controlling the color bar.

[0027] A further technical solution of the present invention is: the infrared data is obtained by rendering using ray tracing technology, specifically:

[0028] L0(X,ω0,λ,t)=L(X,ω0,λ,t)+∫ Ω f r (X,ω i ,ω0,λ,t)L i(X,ω i ,λ,t)(ω i ·n)dω i

[0029] Wherein, the left side of the equal sign L0(X,ω0,λ,t) represents the light emitted by the x point to the sensor ω0 direction, the first term L(X,ω0,λ,t) on the right side of the equal sign represents the self-luminous of x point, the second term ∫ Ω f r (X,ω i ,ω0,λ,t)L i (X,ω i ,λ,t)(ω i ·n)dω i Represents the sum of all incident light reflected to the outgoing light of ω0 direction received by x point.

[0030] The further technical scheme of the present application: the node built in the virtual engine generates labels suitable for scene segmentation, target detection and depth estimation, specifically:

[0031] After rendering, the virtual engine automatically creates a rendering layer node in the node workspace, opens the relevant output channels of the rendering layer node for different labels, builds new function nodes in the composition workspace to obtain corresponding physical properties, connects the rendering layer node relevant output channels with the function nodes, and obtains label data through rendering output.

[0032] A performance verification method of generated simulation infrared data and labels, characterized in that it comprises:

[0033] Construct a pre-trained deep learning model;

[0034] Pair the generated simulation infrared data with the corresponding labels, input the pre-trained deep learning model for training;

[0035] Obtain real infrared data, input the real infrared data into the trained deep learning model to output corresponding labels, compare the output corresponding labels with the real labels, and verify the performance.

[0036] The deep learning model adopts a Yolov8 target detection model.

[0037] A computer system, characterized by comprising: one or more processors, a computer readable storage medium for storing one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors implement the above method.

[0038] The beneficial effects of the present application are:

[0039] The application provides an infrared simulation data and label automatic generation method based on a virtual engine, which proposes a way to express infrared data characteristics in Blender, that is, an Infrared-Group node group is built to realize the effect of an infrared correction equation. The node group corrects the infrared radiation data received by the sensor, so that the generated data is closer to the real situation, thereby improving the physical authenticity of the simulation data. The method is based on relatively simple mathematical modeling, and realizes simulation of more realistic and physically meaningful infrared data by using a virtual engine. The method quickly and massively collects data and corresponding labels in a scripted way, effectively realizes the expansion of the diversity of infrared data, solves the problem of the scarcity of infrared data, thereby improves the performance of the deep learning model, and provides more extensive application possibilities in diversified scenarios.

[0040] The method solves the problems in the existing infrared data acquisition process. Through simple mathematical modeling and innovative use of a virtual engine, the method successfully realizes efficient generation of realistic and diverse infrared simulation data, and provides a more reliable data basis for a deep learning model. BRIEF DESCRIPTION OF DRAWINGS

[0041] The accompanying drawings are included to provide a further understanding of the application and are incorporated in and constitute a part of this specification, illustrate embodiments of the application and together with the description serve to explain the principles of the application. In the drawings:

[0042] Figure 1 is the overall framework of the application.

[0043] Figure 2 is an Infrared-Group structure diagram.

[0044] Figure 3 is an effect comparison diagram of whether to use the Infrared-Group structure: (a) without using the Infrared-Group structure; (b) using the Infrared-Group structure; (c) a real infrared data sample.

[0045] Figure 4 is a structure diagram of the generated segmentation mask label of the example of the application.

[0046] Figure 5 is a structure diagram of the generated depth map label of the example of the application.

[0047] Figure 6 is a part of the infrared simulation data collected by the application.

[0048] Figure 7are three kinds of label indication drawings collected by the present application: (a) a segmentation Mask indication drawing in the examples of the present application; (b) a Bounding Box indication drawing in the examples of the present application; (c) a depth map indication drawing in the examples of the present application.

[0049] Figure 8 are the results of target detection in the examples of the present application: (a) F-Tank as a training set to detect simple data; (b) B-Tank as a training set to detect simple data; (c) as a training set to detect complex data; (d) B-Tank as a training set to detect complex data. DETAILED DESCRIPTION

[0050] In order to make the purpose, technical scheme and advantages of the present application clearer, the present application is further described in detail below in combination with the drawings and examples. It should be understood that the specific examples described herein are only used to explain the present application and do not limit the present application. In addition, the technical features involved in each embodiment of the present application described below can be combined with each other as long as they do not conflict with each other.

[0051] It should be noted that the terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and do not necessarily describe a specific order or sequence. It should be understood that the terms used in this way can be interchanged as appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "include" and "have" and any variations thereof are intended to cover non-exclusive inclusion, for example, a process, method, system, product or device including a series of steps or units does not necessarily limit to those steps or units clearly listed, but can include other steps or units not clearly listed or inherent to these processes, methods, products or devices.

[0052] The present application provides an infrared simulation data and label automatic generation method based on Blender virtual engine, and uses downstream tasks to prove the effectiveness of simulation data on the performance improvement of deep learning model. The method comprises the following steps:

[0053] Step 1: Build the scene where the data needs to be obtained in Blender, perform 3D modeling, and set up multiple frames of animation to facilitate obtaining data at different angles and in different scenes.

[0054] Step 2: Model the object thermal radiation value received at the sensor using the Planck blackbody radiation law, and calculate the thermal radiation self-emission value of the required material in the infrared band under the scene;

[0055] Step 3, set up the infrared correction effect, set up Infrared-Group node combination for each material, use Infrared-Group to control the generation of object infrared radiation effect, so that the infrared effect is more realistic and has explainability, and render the infrared data through ray tracing technology. At the same time, use Brush to select special areas (areas with higher temperature) to add Proximity effect to control the influence of special areas on adjacent areas of the surface. Used to simulate the effect of infrared correction equation. In this method, it is called proximity sensor.

[0056] Step 4, build nodes in virtual engine to generate labels suitable for scene segmentation, target detection and depth estimation.

[0057] Step 5, use the collected simulated infrared data and labels to train the model for downstream tasks combined with real data, which proves that the infrared simulation data collected by this method has significant effect on improving the performance of deep learning model tasks.

[0058] Step 1, according to the required task scene, 3D modeling is carried out, and according to the task, animation is made in the scene to obtain more abundant and larger amount of simulation data.

[0059] In step 2, the material's heat radiation emission value is calculated by using Planck blackbody radiation law, which quantifies the heat radiation of the object into heat radiation emission value, and the modeling formula is as follows:

[0060]

[0061] Where L is the heat radiation emission value [W / m 2 / sr], T is the material temperature [K], h is the Planck constant, c is the speed of light, λ is the wavelength [μm], 7-13 μm is selected as the long-wave infrared band, k is the Boltzmann constant, ε avg is the average emissivity of the material, a value between 0 and 1, which relates the radiation of a real object to that of a blackbody. Through the Planck blackbody radiation equation, the heat radiation emission value of different temperature materials in the long-wave infrared band can be obtained

[0062] In the calculation process of step 2, the band 7-13 μm is divided into 3n, (n∈{1,2,...18}) bands, and the heat radiation emission values in these bands are calculated respectively.

[0063] The infrared correction equation used in step 3 is:

[0064]

[0065] d(X,X s )=||X-X s ||

[0066] where L0 represents the thermal radiation value at point X, d represents the distance from point X to point X_s, X_s represents the heat source, R represents the range of heat source influence at point X, and a1, a2 represent the attenuation factors, respectively. s s where L0 represents the thermal radiation value at point X, d represents the distance from point X to point X_s, X_s represents the heat source, R represents the range of heat source influence at point X, and a1, a2 represent the attenuation factors, respectively.

[0067] This formula is used to correct the radiation intensity detected by the sensor to reflect the influence of the distance between the sensor and the heat source on the radiation attenuation. Specifically, the following two cases are considered:

[0068] 1. When d(X, X_s) < R, the sensor is close to the heat source, and the radiation intensity decreases slowly with the increase of distance. This case is described by a small attenuation factor to reflect the high sensitivity of the sensor to radiation in close proximity.

[0069] 2. When d(X, X_s) > R, the sensor is far from the heat source, and the radiation intensity attenuates rapidly. This case uses a larger attenuation factor to simulate the significant decrease in sensor perception of radiation at a distance. For example, the area adjacent to a high-emissivity region does not experience a sudden drop in radiation value, but rather a slow decreasing trend. This method not only accurately reflects the spatial distribution of thermal radiation, but also simulates the gradual transition of thermal radiation between different materials, thereby improving the realism and physical consistency of the simulation image.

[0070] Subsequently, an Infrared_Group node group is added for each material. This node group is used to normalize the thermal radiation brightness in the scene, setting the spontaneous emission value of the pixel point with the lowest thermal radiation energy to 0 and the highest to 1. Then, the thermal radiation emission value calculated in step 1 is read into the layer weight, and these weights control the spontaneous emission value of the material through a color bar. Finally, the infrared data is obtained by ray tracing rendering, and the formula for ray tracing is as follows:

[0071] L0(X, ω0, λ, t) = L(X, ω0, λ, t) + ∫ Ω f r (X, ω i , ω0, λ, t) L i (X, ω i , λ, t) (ω i · n) dω i

[0072] where L0(X, ω0, λ, t) on the left side of the equation represents the light emitted by point X in the direction of sensor ω0, and the first term L(X, ω0, λ, t) on the right side of the equation represents the self-light of point X, and the second term ∫ Ω f r (X, ω i , ω0, λ, t) L i ​(X,ω i ,λ,t)(ω i ·n)dω i represents the sum of all incident light received at point x reflected to the direction of outgoing light ω0.

[0073] Step 4, after the rendering of step 3 is completed, the virtual engine automatically creates a rendering layer node in the node workspace, opens the relevant output channels of the rendering layer node for different labels, and builds new function nodes in the composition workspace to obtain the corresponding physical properties, and connects the rendering layer node relevant output channel with the function node, and the rendering output obtains the label data.

[0074] Step 5, model training of downstream tasks is carried out to verify whether the simulated infrared data can improve the performance in deep learning tasks, and the specific steps are as follows:

[0075] Step 5.1: for specific downstream tasks, select the applicable deep learning model and real data set.

[0076] Step 5.2: pair the generated simulated infrared data with the corresponding labels to build a data set for model training. In this process, set up experimental data and control data reasonably, and draw conclusions through comparison experiments.

[0077] The present application provides an infrared simulation data generation method based on virtual engine, which solves the problems in the existing infrared data acquisition process. Through simple mathematical modeling and innovative use of virtual engine, the efficient generation of realistic and diverse infrared simulation data is successfully realized, providing a more reliable data basis for deep learning models.

[0078] In order to make those skilled in the art better understand the present application, the present application will be described in detail below with specific examples.

[0079] Example 1

[0080] As Figure 1 shown, the present application provides a kind of infrared simulation data and label automatic generation method based on virtual engine, steps are as follows:

[0081] Step 1, use virtual engine Blender to build 3D scene, set animation.

[0082] This example simulates that the tank drives in mountainous terrain, and the scene includes mountainous terrain, Tank two basic elements. In the example, a total of 1000 frames of animation are set, from the 1st frame to the 1000th frame, the animation of a Tank driving is completed.

[0083] Step 2, preset the material that will appear in the scene, and calculate the thermal radiation emission value of the material. As Figure 7(a) The street scene in the example shown contains four objects: 1) Tank: including Al2O3, paint and other materials; 2) Mountain: including grass, soil, rock and other materials; 3) Wooden bridge: including wood, paint and other materials; 4) Lake water. The thermal radiation emission values of these materials in the 7-13 μm wave band are calculated using the Planck blackbody radiation law.

[0084]

[0085] Taking the material Al2O3 as an example, the temperature of the material in the scene is set to 20 degrees Celsius, i.e. T = 293.15 K, and the wave band is divided into 54 segments, and the thermal radiation emission values of aluminum in the 54 wave bands are calculated as shown in Table 1:

[0086] Table 1 Thermal radiation values of Al2O3 material in 54 wave bands in this example

[0087] Waveband 7-7.11 7.11-7.22 7.22-7.33 ... 12.61-12.72 12.72-12.83 12.83-12.94 Emissivity 0.8111 0.9761 0.9689 ... 0.9500 0.9738 0.9565 Emission value 0.1304 0.1303 0.1284 ... 0.0499 0.0487 0.0480

[0088] Step 3, set the infrared effect of the scene. First, build an Infrared-Group node group through a script, and the node group structure is as shown in Figure 2 Secondly, the emission values of the materials in the 54 wave bands obtained in step 2 are averaged every 3 to obtain 18 emission values, which are read into the layer weight through a script. The layer weight generates 18 color controllers, so that the material has a unique glow pattern, which simulates the physical behavior of infrared objects radiating heat in the real world. By refining the material, the light-emitting part of the object can be controlled more accurately. Through the proximity sensor, the surrounding of the part that glows brighter (has a higher temperature) will show a slow decrease in glow (decrease in heat), rather than a sudden drop. Finally, the infrared data is rendered, and the effect is as shown in Figure 6

[0089] Figure 3 The simulation effect difference brought by whether to use the Infrared-Group node group is shown. 3(a) is not using Infrared-Group, directly assigning the material emission value to the material self-emission attribute. It can be seen that the glow intensity of the entire Tank is very uniform, and it cannot well represent the physical meaning of infrared. 3(b) is the effect of using Infrared-Group. It can be seen that the background object has a low thermal radiation emission value (low brightness), and the tank has a high thermal radiation emission value (high brightness). In addition, the part of the tank near the engine has a higher emission value (brighter), and the edge has a lower emission value (darker)

[0090] Step 4, in this example, the node is built to generate labels suitable for scene segmentation, target detection and depth estimation. The specific example is:​

[0091] The method for obtaining Mask labels based on object segmentation is as follows: a unique ID is created for each object used in the scene built in step 1, as shown in Figure 7 (a). In this example, the correspondence is as follows: 1) Tank, 2) Mountain, 3) Wooden Bridge, and 4) Lake Water. As shown in Figure 4 The following operations are completed using scripts: opening the rendering layer object ID output channel, adding a normalization node (Normalize Node), a combine color node (Combine Color Node), and a composite node (Composite Node). The rendering layer material ID output is connected to the input of the normalization node, so that the rendering layer output pixel value is mapped to the range of 0 to 1. The output of the normalization node is connected to the input of the hue parameter in the combine color node, and the desired output color effect is obtained by adjusting the two 0-1 parameters of saturation (S) and brightness (V) in the combine color node. In this example, the values of S and V in the combine color node are 0.82 and 0.90, respectively. The Mask label based on object segmentation obtained is shown in Figure 7 (a). In this label, the same object is assigned the same color block, and different objects are assigned different color blocks for material differentiation.

[0092] The method for obtaining Mask labels based on material segmentation is as follows: in this example, the materials are divided into 1) Al2O3, paint, etc., 2) grass, soil, and rock, 3) wood, and 4) water. As shown in Figure 4 the rendering layer material ID channel is opened, and the same nodes as in the above step of obtaining Mask labels based on object segmentation are created for output. The obtained Mask label effect is that objects with different ID numbers are assigned to different colors for object differentiation.

[0093] Bounding Box labels for object detection are generated, the class and coordinates of the object of interest are obtained, and the YOLO format is output, i.e., the left upper corner and right lower corner coordinates of the Bounding Box frame are output. In this example, only the Tank class is considered, and the output YOLO format label is:

[0094] Tank<371><330><1188><907>

[0095] The labels are visualized on the infrared data graph, as shown in Figure 7 (b).

[0096] Depth map labels are generated: as shown in Figure 5The opening render layer depth output channel is shown, and a normalize node, a despeckle node, an anti-aliasing node, and a composite node are added. The render layer depth channel output is connected to the normalize node to map the render layer output pixel value to 0 to 1. The normalize node output is connected to the input of the despeckle node. The despeckle node has three adjustable parameters, threshold, neighborhood, and coefficient, which are used to adjust the despeckle effect. The threshold parameter is used to control the high / low complexity of the pixels in the image, and the neighborhood is used to control the range of adjacent pixels matched with the pixel. In this example, the threshold is 0.2, the neighborhood is 0.5, and the coefficient is 1. The despeckle node output is connected to the input of the anti-aliasing node, which is used to eliminate distortion artifacts around the jagged edges. The anti-aliasing node effect is controlled by three parameters, threshold, contrast limit, and corner roundness. The threshold is used to control the edge detection sensitivity of the entire image; the contrast limit is used to control the contrast level to be considered when detecting edges; and the corner roundness is used to help preserve the original shape, and the higher the value, the better the angle will be preserved, i.e. similar to the original image. In the example, the threshold is 0.7, the contrast limit is 0.2, and the corner roundness is 0.6. Figure 7 (c) shows the depth estimation map in a street scene.

[0097] Step 5. In this example, target detection is used as a downstream task to verify the effectiveness of the simulated infrared dataset. The specific steps are as follows:

[0098] Step 5.1. In this example, the Yolov8 target detection architecture is used to detect Tank class.

[0099] Step 5.2. The dataset for target detection is the simulated infrared data B-Tank collected by Blender in this example and the Tank data used for comparison, which includes 1) the real measured infrared Tank data R-Tank as shown in Figure 3 (c); 2) the simulated data F-Tank used for training, as shown in Figure 3(a) shown. Due to the high cost of collecting infrared tank target data, a large number of battle tests are needed to obtain sufficient data. Therefore, the R-Tank dataset has only 157 frames, while the F-Tank has a total of 429 frames. The data collection method is to simply assign the material emission value to the self-luminous attribute, and the data scene includes common soil, vegetation, rock, water, and other environments. In this example, Blender obtains a total of 20,000 frames of B-Tank, which is divided into training and validation sets according to an 8:2 ratio. The F-Tank data is also divided into training and validation sets according to an 8:2 ratio, and the real infrared data is all used as the test set. Our goal is to prove that when the number of simulated infrared data is large enough and the effect is realistic enough, good performance transfer effect can be achieved on the real data set.

[0100] The large amount of simulated infrared data collected in steps 4 and 5 is used for target detection training, and then evaluated on real Tank data. The target detection experiment design is shown in Table 2, and the experimental results are shown in Table 2. From the target detection results, it can be seen that as the number of Blender simulation data increases, the detection accuracy on real data gradually increases. Visualization is shown in Figure 8 It can be seen that the addition of Blender data greatly improves the performance of the target detection network.

[0101] Table 2 Target detection experiment design

[0102]

[0103] Table 3 Target detection experiment results

[0104]

[0105] From the results of Table 3, the following conclusions can be drawn: Experiment group 2 (Blender_10000) and experiment group 3 (Blender_20000) increase the recall to 0.433 and 0.526, respectively, as the Blender simulation data volume increases. This shows that large-scale Blender simulation data improves the recall rate of the model in the target detection task. Especially when the data volume reaches 20000, the recall nearly doubles, proving the significant impact of data size on model performance. In complex scenarios (36 samples under complex background or occlusion conditions), the recall of the control group HW_429 is 0.0811, which is lower than the overall recall, indicating that complex scenarios increase the difficulty of model recognition. As the training data increases to 10000 and 20000, the recall of experiment group 5 and experiment group 6 increases to 0.359 and 0.470, respectively. This shows that in complex scenarios, the model's ability to identify targets improves significantly as the Blender dataset size expands. Especially when the data volume reaches 20000, the recall is close to 0.5, indicating that the model performs more robustly under complex conditions.

[0106] The above merely illustrates the specific embodiments of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art can easily think of various equivalent modifications or replacements within the scope of the present application, and these modifications or replacements shall be encompassed within the protection scope of the present application.

Claims

1. A method for automatically generating infrared simulation data and labels based on a virtual engine, characterized in that: Here are the steps: Step 1: Build the scene for which data needs to be obtained in Blender, perform 3D modeling, and set up multi-frame animation to obtain data from different angles and scenes; Step 2: Use Planck's blackbody radiation law to model the thermal radiation value of the object received at the sensor and calculate the thermal radiation self-emission value of the required material in the infrared band in the scene; Step 3: Set an Infrared-Group node combination for each material based on the thermal radiation self-emission value, and use the Infrared-Group to control the infrared radiation effect of the generated object; correct the infrared radiation by setting a proximity sensor; and render the infrared data using ray tracing technology; The proximity sensor described in step 3 uses the following formula: in, Indicates the thermal radiation value at point X, d indicates The distance from point X, For any point other than point X, R express X The range of heat source influence at the point, are the attenuation factors, is the wavelength, Indicates the sensor, Indicates the thermal radiation correction value; When d(X, Xs) < R, the distance between the sensor and the heat source is close, and the radiation intensity decays slowly with increasing distance. This situation is described by a smaller attenuation factor, reflecting the sensor's sensitivity to radiation at close distances. When d(X, Xs) > R, the distance exceeds the detection range, and the radiation intensity decays faster. This situation is described by a larger attenuation factor, simulating a significant decrease in the sensor's radiation perception at long distances. The Infrared-Group node combination is set for each material based on the thermal radiation self-emission value, including: Normalize the calculated thermal radiation emission values, setting the lowest thermal radiation emission value in the scene to 0 and the highest thermal radiation emission value to 1; The normalized thermal radiation emission value is read into the layer weight through the script. The layer weight determines the self-emission value of the material by controlling the color bar; Step 4: Build nodes in Unreal Engine to generate labels suitable for scene segmentation, object detection, and depth estimation.

2. The method for automatically generating infrared simulation data and labels based on a virtual engine according to claim 1, characterized in that: In step 2, Planck's blackbody radiation law is used to model the thermal radiation value of the object received at the sensor. The modeling formula is as follows: Where L is the thermal radiation emission value, T is the material temperature, h is Planck's constant, c is the speed of light, k is the Boltzmann constant, is the average emissivity of the material.

3. The method for automatically generating infrared simulation data and labels based on a virtual engine according to claim 2, characterized in that: In step 2, the thermal radiation self-emission value of the required material in the infrared band in the scene is calculated, specifically: The long-wave infrared band is divided into multiple bands, and the thermal radiation emission values ​​in multiple bands are calculated respectively.

4. The method for automatically generating infrared simulation data and labels based on a virtual engine according to claim 1, characterized in that: The infrared data is obtained by rendering through ray tracing technology, specifically: Among them, the left side of the equal sign Indicates the x-point sensor Directionally emitted light, the first term on the right side of the equal sign Represents the self-luminescence of point x, the second term Indicates that all incident light received by point x is reflected to The sum of all outgoing light in all directions.

5. The method for automatically generating infrared simulation data and labels based on a virtual engine according to claim 1, characterized in that: The nodes are built in the virtual engine to generate labels suitable for scene segmentation, target detection, and depth estimation, specifically: After rendering is completed, the virtual engine automatically creates a rendering layer node in the node workspace, opens the relevant output channels of the rendering layer node for different labels, and builds a new functional node in the synthesis workspace to obtain the corresponding physical properties, and connects the relevant output channels of the rendering layer node with the functional node, and renders the output to obtain the label data.

6. A method for verifying the performance of a tag and simulated infrared data generated by the method according to any one of claims 1 to 5, characterized in that: include: Build pre-trained deep learning models; Pair the generated simulated infrared data with the corresponding labels and input them into the pre-trained deep learning model for training; Obtain real infrared data, input the real infrared data into the trained deep learning model to output the corresponding label, and compare the output corresponding label with the real label to verify the performance.

7. The method for verifying the performance of simulated infrared data and tags according to claim 6, characterized in that: The deep learning model adopts the Yolov8 target detection model.

8. A computer system, characterized in that include: One or more processors, a computer-readable storage medium for storing one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors implement the method according to any one of claims 1 to 7.

Images