An infrared image generation method, device, electronic device and storage medium

Abstract

An infrared image generation method, apparatus, electronic device, and storage medium provided by an embodiment of the present application are applied to the field of information technology. It can calculate multiple collision positions of light rays emitted from a preset starting point and the infrared radiation intensity corresponding to each collision position with respect to a target object; for each pixel point of the target object on the imaging plane, calculate the infrared radiation intensity of each pixel point according to the multiple collision positions and the infrared radiation intensity corresponding to each collision position; generate an infrared image of the target object according to the calculated infrared radiation intensity of each pixel point. Through the method of the embodiment of the present application, the infrared radiation intensity of each pixel point can be calculated by directly using the collision positions and the infrared radiation intensity corresponding to each collision position, and the infrared image of the target object can be generated, thereby improving the rendering effect during infrared image generation.

Description

Technical Field

[0001] This application relates to the field of information technology, and particularly to an infrared image generation method, apparatus, electronic device, and storage medium. Background Art

[0002] Currently, infrared image simulation methods have been widely used. Through infrared image simulation, according to the principles of graphics, the image effects captured by a real camera can be simulated by a computer. It can not only obtain experimental data that are difficult to achieve under some natural conditions and special experimental conditions, but also reduce the experimental cost.

[0003] However, if the rasterization method is used in the rendering technology to simulate infrared images, that is, the color of each pixel in the image is calculated through the GPU fragment shader, the rendering effect is often poor. Summary of the Invention

[0004] The purpose of the embodiments of this application is to provide an infrared image generation method, apparatus, electronic device, and storage medium to improve the rendering effect of infrared images during the infrared image generation process. The specific technical solutions are as follows:

[0005] In the first aspect of the embodiments of this application, an infrared image generation method is provided, including:

[0006] Calculating the collision positions of the light rays emitted from a preset starting point and a target object, and the infrared radiation intensity corresponding to each collision position;

[0007] For each pixel point of the target object on the imaging plane, calculating the infrared radiation intensity of each pixel point according to the multiple collision positions and the infrared radiation intensity corresponding to each collision position;

[0008] Generating an infrared image of the target object according to the calculated infrared radiation intensity of each pixel point.

[0009] Optionally, the calculating the collision positions of the light rays emitted from a preset starting point and a target object, and the infrared radiation intensity corresponding to each collision position includes:

[0010] Calculating the first collision position of the light rays emitted from a preset starting point and the target object and the corresponding first infrared radiation intensity according to the preset scene parameters and material parameters of the target object;

[0011] Calculating the collision position of the light rays emitted from the Nth collision point and the target object and the corresponding (N + 1)th infrared radiation intensity according to the preset scene parameters and the preset material parameters of the target object, where N is greater than or equal to 1.

[0012] Optionally, calculating the first collision position and the corresponding first infrared radiation intensity of the light emitted from the preset starting point and the target object according to the preset scene parameters and material parameters of the target object includes:

[0013] According to the preset scene parameters and material parameters of the target object, calculate the first collision position of the light emitted from the preset starting point and the target object, and the self-infrared radiation intensity, the first external light source direct irradiation infrared radiation intensity, the first external light source indirect irradiation infrared radiation intensity, the first surrounding object indirect irradiation infrared radiation intensity, and the first attenuation rate corresponding to the first collision position;

[0014] Calculate the first infrared radiation intensity according to the self-infrared radiation intensity corresponding to the first collision position, the first external light source direct irradiation infrared radiation intensity, the first external light source indirect irradiation infrared radiation intensity, the first surrounding object indirect irradiation infrared radiation intensity, and the first attenuation rate.

[0015] Optionally, calculating the (N + 1)-th collision position and the corresponding (N + 1)-th infrared radiation intensity of the light emitted from the N-th collision point and the target object according to the preset scene parameters and the preset material parameters of the target object includes:

[0016] According to the preset material parameters of the target object, calculate the (N + 1)-th collision position of the light emitted from the N-th collision point and the target object, and the self-infrared radiation intensity, the second external light source direct irradiation infrared radiation intensity, the second external light source indirect irradiation infrared radiation intensity, the second surrounding object indirect irradiation infrared radiation intensity, and the second attenuation rate corresponding to the (N + 1)-th collision position;

[0017] Calculate the (N + 1)-th infrared radiation intensity according to the self-infrared radiation intensity corresponding to the (N + 1)-th collision position, the second external light source direct irradiation infrared radiation intensity, the second external light source indirect irradiation infrared radiation intensity, the second surrounding object indirect irradiation infrared radiation intensity, and the second attenuation rate.

[0018] Optionally, for each pixel point of the target object on the imaging plane, calculating the infrared radiation intensity of each pixel point according to the multiple collision positions and the infrared radiation intensity corresponding to each collision position includes:

[0019] For any pixel point of the target object on the imaging plane, identify the target collision position corresponding to the pixel point among the multiple collision positions, and the target infrared radiation intensity corresponding to the target collision position;

[0020] Superimpose the target infrared radiation intensities to obtain the infrared radiation intensity of the pixel point.

[0021] Optionally, after generating the infrared image of the target object based on the calculated infrared radiation intensity of each pixel point, the method further includes:

[0022] Identifying a target area corresponding to the first collision position of the target object in the imaging plane;

[0023] Calibrating the infrared image of the target object according to the position of the target area to obtain a calibrated infrared image.

[0024] Optionally, after generating the infrared image of the target object based on the calculated infrared radiation intensity of each pixel point, the method further includes:

[0025] Inputting the infrared image of the target object into a target recognition model to be trained to obtain a recognition result of the target object;

[0026] Updating the parameters of the target recognition model to be trained according to the recognition result, and returning to execute the step of inputting the infrared image of the target object into the target recognition model to be trained to obtain the recognition result of the target object until a preset termination condition is met, and obtaining a trained target recognition model.

[0027] The above trained target recognition model can be applied to infrared recognition scenarios, such as pedestrian recognition and vehicle recognition in scenarios with relatively dim natural light (rainy days, nights), to improve the accuracy of infrared image recognition.

[0028] In a second aspect of the embodiments of the present application, an infrared image generation device is provided, including:

[0029] A collision position calculation module, configured to calculate multiple collision positions of light rays emitted from a preset starting point and the target object and the infrared radiation intensity corresponding to each collision position;

[0030] A radiation intensity calculation module, configured to calculate the infrared radiation intensity of each pixel point on the imaging plane of the target object according to the multiple collision positions and the infrared radiation intensity corresponding to each collision position;

[0031] An infrared image generation module, configured to generate the infrared image of the target object according to the calculated infrared radiation intensity of each pixel point.

[0032] Optionally, the collision position calculation module includes:

[0033] A first collision calculation sub-module, configured to calculate a first collision position of light rays emitted from a preset starting point and the target object and the corresponding first infrared radiation intensity according to the preset scene parameters and material parameters of the target object;

[0034] The Nth collision calculation sub-module is used to calculate the light emitted from the Nth collision point, the (N + 1)th collision position of the target object, and the corresponding (N + 1)th infrared radiation intensity according to the preset scene parameters and the preset material parameters of the target object, where N is greater than or equal to 1.

[0035] Optionally, the first collision calculation sub-module includes:

[0036] The first parameter calculation unit is used to calculate the light emitted from the preset starting point, the first collision position of the target object, and the corresponding self-infrared radiation intensity, the first external light source direct irradiation infrared radiation intensity, the first external light source indirect irradiation infrared radiation intensity, the first surrounding object indirect irradiation infrared radiation intensity, and the first attenuation rate according to the preset scene parameters and material parameters of the target object;

[0037] The first radiation intensity calculation unit is used to calculate the first infrared radiation intensity according to the self-infrared radiation intensity corresponding to the first collision position, the first external light source direct irradiation infrared radiation intensity, the first external light source indirect irradiation infrared radiation intensity, the first surrounding object indirect irradiation infrared radiation intensity, and the first attenuation rate.

[0038] Optionally, the Nth collision calculation sub-module includes:

[0039] The Nth parameter calculation unit is used to calculate the light emitted from the Nth collision point, the (N + 1)th collision position of the target object, and the corresponding self-infrared radiation intensity, the second external light source direct irradiation infrared radiation intensity, the second external light source indirect irradiation infrared radiation intensity, the second surrounding object indirect irradiation infrared radiation intensity, and the second attenuation rate according to the preset material parameters of the target object;

[0040] The Nth radiation intensity calculation unit is used to calculate the (N + 1)th infrared radiation intensity according to the self-infrared radiation intensity corresponding to the (N + 1)th collision position, the second external light source direct irradiation infrared radiation intensity, the second external light source indirect irradiation infrared radiation intensity, the second surrounding object indirect irradiation infrared radiation intensity, and the second attenuation rate.

[0041] Optionally, the radiation intensity calculation module includes:

[0042] The pixel recognition sub-module is used to identify the target collision position corresponding to any pixel point of the target object on the imaging plane and the target infrared radiation intensity corresponding to the target collision position;

[0043] The radiation intensity superposition sub-module is used to superpose the target infrared radiation intensity to obtain the infrared radiation intensity of the pixel point.

[0044] Optionally, the device further includes:

[0045] An area recognition module, configured to recognize a target area corresponding to the first collision position of the target object in the imaging plane;

[0046] An image calibration module, configured to calibrate the infrared image of the target object according to the position of the target area to obtain a calibrated infrared image.

[0047] Optionally, the device further includes:

[0048] An image input module, configured to input the infrared image of the target object into a target recognition model to be trained, and obtain a recognition result of the target object;

[0049] A model training module, configured to update parameters of the target recognition model to be trained according to the recognition result, and return to continue executing the step of inputting the infrared image of the target object into the target recognition model to be trained to obtain the recognition result of the target object, until a preset termination condition is met, and obtain a trained target recognition model.

[0050] The above trained target recognition model can be applied to an infrared recognition scenario, such as pedestrian recognition and vehicle recognition in a scenario with relatively dim natural light (rainy days, nights), to improve the accuracy of infrared image recognition.

[0051] On the other hand, an embodiment of the present application provides an electronic device, including a processor, a communication interface, a memory, and a communication bus. Among them, the processor, the communication interface, and the memory complete communication with each other through the communication bus;

[0052] The memory is used to store a computer program;

[0053] The processor is configured to implement any of the above infrared image generation methods when executing the program stored on the memory.

[0054] On the other hand, an embodiment of the present application provides a computer-readable storage medium, in which a computer program is stored, and the computer program implements any of the above infrared image generation methods when being executed by a processor.

[0055] On the other hand, an embodiment of the present application further provides a computer program product including instructions, which when running on a computer, causes the computer to execute any of the above infrared image generation methods.

[0056] Advantages of the embodiments of the present application:

[0057] An infrared image generation method, apparatus, electronic device, and storage medium provided by an embodiment of the present application can calculate multiple collision positions of light rays emitted from a preset starting point and a target object, and the infrared radiation intensity corresponding to each collision position; for each pixel point of the target object on the imaging plane, calculate the infrared radiation intensity of each pixel point according to the multiple collision positions and the infrared radiation intensity corresponding to each collision position; generate an infrared image of the target object according to the calculated infrared radiation intensity of each pixel point. Through the method of the embodiment of the present application, the infrared radiation intensity of each pixel point can be calculated by directly using the collision position and the infrared radiation intensity corresponding to each collision position, and the infrared image of the target object can be generated, thereby improving the rendering effect when generating the infrared image.

[0058] Of course, implementing any product or method of the present application does not necessarily require achieving all the above-mentioned advantages simultaneously. BRIEF DESCRIPTION OF THE DRAWINGS

[0059] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings required for use in the description of the embodiments or the prior art. Obviously, the following drawings are only some embodiments of the present application, and those of ordinary skill in the art can also obtain other embodiments based on these drawings.

[0060] Figure 1 It is a schematic flowchart of an infrared image generation method provided by an embodiment of the present application;

[0061] Figure 2 It is a schematic flowchart of calculating the collision position and the corresponding infrared radiation intensity provided by an embodiment of the present application;

[0062] Figure 3 It is an example diagram of an infrared image generation method provided by an embodiment of the present application;

[0063] Figure 4 It is another example diagram of an infrared image generation method provided by an embodiment of the present application;

[0064] Figure 5 It is a schematic flowchart of calculating the infrared radiation intensity of each pixel point provided by an embodiment of the present application;

[0065] Figure 6 It is a schematic flowchart of generating a calibrated infrared image provided by an embodiment of the present application;

[0066] Figure 7a It is a schematic diagram of the generated calibrated infrared image provided by an embodiment of the present application;

[0067] Figure 7bA flowchart of a model training method provided by an embodiment of the present application;

[0068] Figure 8 Another example diagram of an infrared image generation method provided by an embodiment of the present application;

[0069] Figure 9 A structural schematic diagram of an infrared image generation device provided by an embodiment of the present application;

[0070] Figure 10 Another structural schematic diagram of an infrared image generation device provided by an embodiment of the present application;

[0071] Figure 11 A structural schematic diagram of an electronic device provided by an embodiment of the present application. Detailed implementation manners

[0072] Next, the technical solutions in the embodiments of the present application will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art based on the present application belong to the scope of protection of the present application.

[0073] Image simulation: A technology that, based on the principles of graphics, simulates the image effects captured by a real camera through a computer.

[0074] Automated calibration: A technology that automatically frames an object of interest by outputting specific markers during image rendering.

[0075] In the first aspect of the embodiments of the present application, first, an infrared image generation method is provided, including:

[0076] Calculate the collision positions of the light rays emitted from the preset starting point and the target object, and the infrared radiation intensity corresponding to each collision position;

[0077] For each pixel point of the target object on the imaging plane, calculate the infrared radiation intensity of each pixel point according to the multiple collision positions and the infrared radiation intensity corresponding to each collision position;

[0078] Generate an infrared image of the target object according to the calculated infrared radiation intensity of each pixel point.

[0079] It can be seen that through the method of the embodiments of the present application, the infrared radiation intensity of each pixel point can be calculated by directly using the collision positions and the infrared radiation intensity corresponding to each collision position, so as to generate the infrared image of the target object, thereby improving the rendering effect during infrared image generation.

[0080] Specifically, refer to Figure 1 ,Figure 1 A schematic flowchart of an infrared image generation method provided by an embodiment of the present application, including:

[0081] Step S11, calculate the multiple collision positions of the light rays emitted from a preset starting point and the target object, and the infrared radiation intensity corresponding to each collision position.

[0082] The preset starting point in the embodiment of the present application can be a certain spatial position set in advance. For example, a virtual scene can be preset in advance, and a certain position in the virtual scene can be selected as the preset starting point. Specifically, a virtual camera can be set, and the light rays emitted from the preset starting point can be identified through the parameters of the virtual camera. For example, the set parameters can include the internal parameters and external parameters of the virtual camera. Among them, the virtual camera can be a mathematical simplified model of a real camera. A virtual camera object can be established through the internal parameters of the virtual camera. The essence of the internal parameters is a three-dimensional to two-dimensional mapping function. The external parameters refer to the position and orientation of the virtual camera in the world space. When the virtual scene is preset in advance, the three-dimensional virtual scene can be mapped into a two-dimensional plane image through the camera object, just like a real camera. In addition, different cameras can be simulated by setting different camera internal parameters, such as a perspective projection camera with the effect of objects being smaller in the distance and larger up close, an orthographic projection camera without the effect of objects being smaller in the distance and larger up close, a fish-eye camera with a 180° viewing angle, etc.

[0083] Among them, to calculate the multiple collision positions of the light rays emitted from the preset starting point and the target object and the infrared radiation intensity corresponding to each collision position, the multiple collision positions of the multiple light rays emitted from the preset starting point and the target object can be calculated. Moreover, for each light ray, it can collide with the target object one or more times. For example, when a certain light ray collides with the target object, it is emitted onto an object in the virtual scene, then reflected by the object onto the target object and collides again, or repeats this action to collide again. Specifically, to calculate the multiple collision positions of the light rays emitted from the preset starting point and the target object and the infrared radiation intensity corresponding to each collision position, the infrared radiance can be calculated using ray tracing. For the collision position, the position of the collision point can be calculated based on the spatial position where the preset starting point is located, the angle of the light ray emitted from the preset starting point, and the spatial position of the target object. When multiple collisions occur, the position of the next collision and the infrared radiation intensity can be calculated respectively with the position of the previous collision point as the starting position. For example, when tracking 100 light rays emitted by a virtual camera towards the target object, 80 light rays collided with the target object once, 3 light rays collided with the target object three times, and 15 light rays collided with the target object twice. Then, a total of 80 + 15 + 15 + 3 + 3 + 3 = 119 collisions occurred. When calculating the multiple collision positions of the light rays emitted from the preset starting point and the target object and the infrared radiation intensity corresponding to each collision position, for each of the 119 collisions, the collision position and the infrared radiation intensity of each collision are required.

[0084] The method of the embodiments of the present application is applied to an intelligent terminal and can be implemented through the intelligent terminal. Specifically, the intelligent terminal can be a computer, a mobile phone, a server, etc.

[0085] Step S12: For each pixel point of the target object on the imaging plane, calculate the infrared radiation intensity of each pixel point according to the multiple collision positions and the infrared radiation intensity corresponding to each collision position.

[0086] Among them, to calculate the infrared radiation intensity of each pixel point according to the multiple collision positions and the infrared radiation intensity corresponding to each collision position, for each pixel point on the target object, the sum of the infrared radiation intensities of the collisions where the collision position is the pixel position among the multiple collision positions can be calculated. For example, among the 119 collisions of the target object, the collision positions of 3 collisions are the same pixel point. Then, the infrared radiation intensity for this pixel point is the sum of the infrared radiation intensities of these three collisions.

[0087] Step S13: Generate an infrared image of the target object according to the calculated infrared radiation intensity of each pixel point.

[0088] Among them, according to the calculated infrared radiation intensity of each pixel point, an infrared image of the target object can be generated. An infrared radiance map can be generated according to the calculated infrared radiation intensity of each pixel point. For example, different infrared radiation intensity values can be represented by different colors to generate an infrared radiance map. In the actual use process, various infrared imaging maps can also be generated according to specific user requirements, such as thermal imaging maps, infrared night vision maps, etc.

[0089] It can be seen that through the method of the embodiment of the present application, the infrared radiation intensity of each pixel point can be calculated based on the direct collision position and the infrared radiation intensity corresponding to each collision position, so as to generate the infrared image of the target object, thereby improving the rendering effect when generating the infrared image.

[0090] Optionally, referring to Figure 2 , step S11 of calculating the collision positions of the light rays emitted from the preset starting point and the target object and the infrared radiation intensity corresponding to each collision position includes:

[0091] Step S111, calculate the first collision position of the light rays emitted from the preset starting point and the target object and the corresponding first infrared radiation intensity according to the preset scene parameters and material parameters of the target object;

[0092] Step S112, calculate the collision position of the light rays emitted from the Nth collision point and the target object and the corresponding (N + 1)th infrared radiation intensity according to the preset scene parameters and the preset material parameters of the target object, where N is greater than or equal to 1.

[0093] Optionally, calculating the first collision position of the light rays emitted from the preset starting point and the target object and the corresponding first infrared radiation intensity according to the preset scene parameters and material parameters of the target object includes: calculating the first collision position of the light rays emitted from the preset starting point and the target object, and the self-infrared radiation intensity, the direct irradiation infrared radiation intensity of the first external light source, the indirect irradiation infrared radiation intensity of the first external light source, the indirect irradiation infrared radiation intensity of the first surrounding object, and the first attenuation rate corresponding to the first collision position according to the preset scene parameters and material parameters of the target object; calculating the first infrared radiation intensity according to the self-infrared radiation intensity, the direct irradiation infrared radiation intensity of the first external light source, the indirect irradiation infrared radiation intensity of the first external light source, the indirect irradiation infrared radiation intensity of the first surrounding object, and the first attenuation rate corresponding to the first collision position.

[0094] Optionally, according to the preset scene parameters and the preset material parameters of the target object, calculate the light emitted from the Nth collision point, the (N + 1)th collision position of the target object, and the corresponding (N + 1)th infrared radiation intensity, including: according to the preset material parameters of the target object, calculate the light emitted from the Nth collision point, the (N + 1)th collision position of the target object, and the self-infrared radiation intensity, the direct irradiation infrared radiation intensity of the second external light source, the indirect irradiation infrared radiation intensity of the second external light source, the indirect irradiation infrared radiation intensity of the second surrounding object, and the second attenuation rate corresponding to the (N + 1)th collision position; according to the self-infrared radiation intensity, the direct irradiation infrared radiation intensity of the second external light source, the indirect irradiation infrared radiation intensity of the second external light source, the indirect irradiation infrared radiation intensity of the second surrounding object, and the second attenuation rate corresponding to the (N + 1)th collision position, calculate the (N + 1)th infrared radiation intensity.

[0095] In the embodiments of the present application, the preset material parameters of the target object can be preset. For example, each model object in the scene is given an infrared material in advance, such as a temperature distribution field, a bidirectional reflection distribution function and a bidirectional scattering distribution function corresponding to a specific infrared band, etc. The infrared material parameters are mainly used to calculate the infrared radiance. The environmental parameters can also be preset, such as the atmospheric transmittance, etc., which are also used for the calculation of the light radiation. When calculating the light emitted from the preset starting point, the multiple collision positions of the target object, and the infrared radiation intensity corresponding to each collision position, the light emitted from the preset starting point and the first collision position of the target object, and the self-infrared radiation intensity, the direct irradiation infrared radiation intensity of the first external light source, the indirect irradiation infrared radiation intensity of the first external light source, the indirect irradiation infrared radiation intensity of the first surrounding object, and the first attenuation rate corresponding to the first collision position can be calculated according to the preset parameters. For example, referring to Figure 3 , when a ray collides with an object in the scene, calculate the infrared radiance at the collision point through the formula:

[0096] IR rad =(IR(T)+IR direc t+IR indirect )*Attenuation(σ, d)

[0097] where IR(T) is the infrared radiation emitted by itself at the collision point, which is related to the temperature at the collision point, IR direct is the direct illumination of the infrared light source, IR indirect is the indirect illumination of the surrounding objects, and Attenuation(σ, d) is the attenuation rate, which is related to the infrared transmittance of the material or the atmospheric transmittance σ and the propagation distance d.

[0098] Specifically, according to the preset material parameters of the target object, the light emitted from the preset starting point, the first collision position of the target object, and the corresponding first infrared radiation intensity can be calculated using a ray tracing algorithm. For example, a ray tracing method with a virtual camera as the light starting point: starting from the virtual camera coordinates, emit rays in the direction of each pixel on the imaging plane to start ray tracing. When the ray collides with the object surface, calculate the infrared radiation at the collision point, and calculate the outgoing direction and energy attenuation of the new ray according to material properties such as the bidirectional reflectance distribution function or bidirectional scattering distribution function. Based on the new ray, search for a new collision point again, and so on until there is no collision or the number of collisions reaches the set upper limit. Finally, accumulate the radiation on the traced path to calculate the infrared light intensity corresponding to each pixel in this sampling. In actual use, it is also possible to take the average value after sampling multiple times, and at the same time, the ID (tag) value of the first collision needs to be saved. For example, see Figure 4 , after calculating the first collision point, starting from the first collision point, emit a ray in a new direction to continue searching for the second collision point. The new direction is calculated through the infrared material properties. Calculate the radiation of the second collision point, then starting from the second collision point, emit a ray in a new direction to calculate the radiation of the third collision point, and so on iteratively until the set number of collisions is reached or no collision point can be found. Accumulate the radiation calculated for each collision point, and thus complete one sampling. Take the average value of multiple samplings as the final infrared radiation.

[0099] It can be seen that through the method of the embodiment of the present application, according to the preset material parameters of the target object, the light emitted from the preset starting point, the first collision position of the target object, and the corresponding first infrared radiation intensity can be calculated. According to the preset scene parameters and the preset material parameters of the target object, the light emitted from the Nth collision point, the (N + 1)th collision position of the target object, and the corresponding (N + 1)th infrared radiation intensity can be calculated, so that multiple collision positions and the infrared radiation intensities corresponding to each collision position can be calculated.

[0100] Optionally, see Figure 5 , in step S12, for each pixel point of the target object on the imaging plane, according to multiple collision positions and the infrared radiation intensities corresponding to each collision position, calculate the infrared radiation intensity of each pixel point, including:

[0101] Step S121, for any pixel point of the target object on the imaging plane, identify the target collision position corresponding to this pixel point among multiple collision positions, and the target infrared radiation intensity corresponding to the target collision position;

[0102] Step S122, superimpose the target infrared radiation intensities to obtain the infrared radiation intensity of this pixel point.

[0103] Among them, for any pixel point of the target object on the imaging plane, the target collision position corresponding to this pixel point among multiple collision positions can be recognized, one or more target collision positions corresponding to this pixel point among the multiple collision positions of multiple collisions can be recognized, and the target infrared radiation intensity corresponding to each target collision position. For example, among 119 collisions of the target object, the collision positions of 3 collisions are all the same pixel point, then the collision positions of these three collisions are the target collision positions corresponding to this pixel point, and the infrared radiation intensity of each collision among these three collisions is the target infrared radiation intensity of this pixel point. The target infrared radiation intensities are superimposed to obtain the infrared radiation intensity of this pixel point. For example, among 119 collisions of the target object, the collision positions of 3 collisions are all the same pixel point, then the infrared radiation intensities of these 3 collisions can be summed to obtain the infrared radiation intensity of this pixel point.

[0104] It can be seen that through the method of the embodiments of the present application, for any pixel point of the target object on the imaging plane, the target collision position corresponding to this pixel point among multiple collision positions and the target infrared radiation intensity corresponding to the target collision position can be recognized, the target infrared radiation intensities are superimposed to obtain the infrared radiation intensity of this pixel point, so as to calculate the infrared radiation intensities of each pixel point.

[0105] Optionally, referring to Figure 6 , after generating the infrared image of the target object according to the calculated infrared radiation intensities of each pixel point, the above method further includes:

[0106] Step S61, recognizing the target area corresponding to the first collision position of the target object on the imaging plane;

[0107] Step S62, calibrating the infrared image of the target object according to the position of the target area to obtain the calibrated infrared image.

[0108] In the embodiments of the present application, a three-dimensional virtual scene can be built in advance. Specifically, 3ds Max (a three-dimensional mapping tool), Maya (another three-dimensional mapping tool), etc. can be used. A numerical ID or ID texture (that is, different parts of an object can be set with different IDs) can also be assigned to each object in the scene in advance, so that it can be associated with the pixels in the final imaging map during ray tracing according to the ID value to achieve automatic calibration. Among them, in the embodiments of the present application, when calculating the multiple collision positions of the rays emitted from the preset starting point and the target object and the infrared radiation intensity corresponding to each collision position, the ID value of the first collision can be recorded.

[0109] When identifying the target area corresponding to the first collision position of the target object in the imaging plane, it can be identified according to the recorded ID value of the first collision to obtain an ID map that matches the final imaging map. Then, the infrared image of the target object is calibrated according to the position of the target area, and then the pixel area occupied by any object of interest in the scene in the final imaging map is obtained and calibrated to obtain the calibrated infrared image. For example, see Figure 7a , each model object in the virtual scene has its own ID or ID texture. To generate an ID map that matches the final infrared simulation image, in the ray tracing method, the internal and external camera parameters consistent with those during rendering must be used. Then, starting from the camera, rays are emitted in the direction of each pixel on the imaging plane, and the ID value of the first collision point is recorded. As Figure 3 shown, an ID map that matches the infrared simulation map is obtained, and then the calibrated infrared image can be obtained by assigning different color values to each ID value. Figure 7a Different fillings in

[0110] represent different colors. During actual use, the infrared image of the target object generated by the infrared image generation method of the embodiment of the present application can be used for training the model. Optionally, the network model to be trained can be a target recognition model, a feature extraction model, etc. Specifically, when applied to the target recognition module, see Figure 7b , after generating the infrared image of the target object according to the calculated infrared radiation intensity of each pixel point, the above method further includes:

[0111] Step S71, input the infrared image of the target object into the target recognition model to be trained to obtain the recognition result of the target object;

[0112] Step S72, update the parameters of the target recognition model to be trained according to the recognition result, and return to the step of inputting the infrared image of the target object into the target recognition model to be trained to obtain the recognition result of the target object, and continue to execute until the preset termination condition is met, and a trained target recognition model is obtained.

[0113] To illustrate the method of the embodiment of the present application, the following is described in conjunction with specific embodiments. See Figure 8 , including:

[0114] Build a three-dimensional virtual scene. The three-dimensional virtual scene can be built using off-the-shelf commercial software, such as 3ds Max, Maya, etc.

[0115] Set the object IDs and infrared materials in the scene. Assign a numerical ID or ID texture (i.e., different parts of a model can be set with different IDs) to each model object in the scene. This ID value can be associated with the pixels in the final rendered image during subsequent ray tracing to achieve an automated calibration mechanism. Assign infrared materials to each model object in the scene, such as temperature distribution fields, bidirectional reflectance distribution functions, and bidirectional scattering distribution functions corresponding to specific infrared bands. The infrared material parameters are mainly used to calculate infrared radiance.

[0116] Set environmental parameters, such as atmospheric transmittance, which is also used for light radiance calculation.

[0117] Set up a virtual camera, including internal and external parameters. A virtual camera is a mathematical simplified model of a real camera. By setting the internal parameters of the camera, a virtual camera object is established, which is essentially a three-dimensional to two-dimensional mapping function. The external parameters refer to the position and orientation of the virtual camera in the world space. This camera object can map the three-dimensional virtual scene into a two-dimensional planar image, just like a real camera. Additionally, by setting different internal parameters of the camera, different cameras can be simulated, such as a perspective projection camera with the effect of objects appearing smaller in the distance and larger up close, an orthographic projection camera without this effect, and a fisheye camera with a 180° field of view, etc.

[0118] Ray tracing for calculating infrared radiance. Use the algorithm framework of ray tracing to calculate the infrared radiance of each pixel on the final rendered image. Here is an example of a ray tracing method starting from the camera: Starting from the camera coordinates, emit rays in the direction of each pixel on the imaging plane to start ray tracing. When a ray collides with the object surface, calculate the infrared radiance at the collision point, and calculate the outgoing direction and energy attenuation of the new ray according to material properties such as the bidirectional reflectance distribution function or bidirectional scattering distribution function. Based on the new ray, search for new collision points again. Repeat this process until there are no more collisions or the number of collisions reaches the set upper limit. Finally, accumulate the radiance along the traced path to calculate the infrared light intensity corresponding to each pixel for this sampling. After sampling multiple times and taking the average, the ID value of the first collision needs to be saved.

[0119] Infrared imaging. Convert the infrared radiance map obtained from the previous step of ray tracing into an infrared imaging map according to the specific needs of the user, such as a thermal imaging map, an infrared night vision map, etc.

[0120] Automated calibration. According to the ID value saved during the first collision of ray tracing, an ID map that matches the final rendered image can be obtained. Based on this, the pixel area occupied by any object of interest in the scene in the final rendered image can be obtained.

[0121] See Figure 9 , Figure 9To illustrate a schematic structural diagram of the infrared image generation device provided by the embodiments of the present application, it includes: a scene construction module, a rendering module, and an automatic calibration module.

[0122] The scene construction module includes the construction of the entire three-dimensional virtual scene, the assignment of materials, the setting of environmental parameters, and the setting of virtual cameras.

[0123] The rendering module includes ray tracing to calculate infrared radiance and infrared imaging.

[0124] The automatic calibration module is for the automatic calibration of the region of interest in the final imaging map.

[0125] In the second aspect of the embodiments of the present application, an infrared image generation device is provided. Refer to Figure 10 , including:

[0126] The collision position calculation module 1001 is used to calculate multiple collision positions of the light rays emitted from the preset starting point and the target object, and the infrared radiation intensity corresponding to each collision position.

[0127] The radiation intensity calculation module 1002 is used to calculate the infrared radiation intensity of each pixel point on the imaging plane of the target object according to multiple collision positions and the infrared radiation intensity corresponding to each collision position.

[0128] The infrared image generation module 1003 is used to generate an infrared image of the target object according to the calculated infrared radiation intensity of each pixel point.

[0129] Optionally, the collision position calculation module 1001 includes:

[0130] The first collision calculation sub-module is used to calculate the first collision position of the light rays emitted from the preset starting point and the target object, and the corresponding first infrared radiation intensity according to the preset scene parameters and material parameters of the target object.

[0131] The Nth collision calculation sub-module is used to calculate the (N + 1)th collision position of the light rays emitted from the Nth collision point and the target object, and the corresponding (N + 1)th infrared radiation intensity according to the preset scene parameters and the preset material parameters of the target object, where N is greater than or equal to 1.

[0132] Optionally, the first collision calculation sub-module includes:

[0133] The first parameter calculation unit is used to calculate the first collision position of the light rays emitted from the preset starting point and the target object, the self-infrared radiation intensity corresponding to the first collision position, the direct irradiation infrared radiation intensity of the first external light source, the indirect irradiation infrared radiation intensity of the first external light source, the indirect irradiation infrared radiation intensity of the first surrounding object, and the first attenuation rate according to the preset scene parameters and material parameters of the target object.

[0134] The first radiation intensity calculation unit is configured to calculate the first infrared radiation intensity according to the self-infrared radiation intensity corresponding to the first collision position, the direct irradiation infrared radiation intensity of the first external light source, the indirect irradiation infrared radiation intensity of the first external light source, the indirect irradiation infrared radiation intensity of the first surrounding object, and the first attenuation rate.

[0135] Optionally, the Nth collision calculation sub-module includes:

[0136] The Nth parameter calculation unit is configured to calculate the light ray emitted from the Nth collision point and the (N + 1)th collision position of the target object, and the self-infrared radiation intensity, the direct irradiation infrared radiation intensity of the second external light source, the indirect irradiation infrared radiation intensity of the second external light source, the indirect irradiation infrared radiation intensity of the second surrounding object, and the second attenuation rate corresponding to the (N + 1)th collision position according to the preset material parameters of the target object;

[0137] The Nth radiation intensity calculation unit is configured to calculate the (N + 1)th infrared radiation intensity according to the self-infrared radiation intensity, the direct irradiation infrared radiation intensity of the second external light source, the indirect irradiation infrared radiation intensity of the second external light source, the indirect irradiation infrared radiation intensity of the second surrounding object, and the second attenuation rate corresponding to the (N + 1)th collision position.

[0138] Optionally, the radiation intensity calculation module 1002 includes:

[0139] The pixel recognition sub-module is configured to identify, for any pixel point of the target object on the imaging plane, the target collision position corresponding to the pixel point among multiple collision positions, and the target infrared radiation intensity corresponding to the target collision position;

[0140] The radiation intensity superposition sub-module is configured to superpose the target infrared radiation intensities to obtain the infrared radiation intensity of the pixel point.

[0141] Optionally, the above device further includes:

[0142] The area recognition module is configured to recognize the target area corresponding to the first collision position of the target object in the imaging plane;

[0143] The image calibration module is configured to calibrate the infrared image of the target object according to the position of the target area to obtain the calibrated infrared image.

[0144] Optionally, the device further includes:

[0145] The image input module is configured to input the infrared image of the target object into the target recognition model to be trained to obtain the recognition result of the target object;

[0146] The model training module is used to update the parameters of the target recognition model to be trained according to the recognition results, and return to continue the steps of inputting the infrared image of the target object into the target recognition model to be trained to obtain the recognition results of the target object until the preset termination condition is met, and a trained target recognition model is obtained.

[0147] Through the device of the embodiments of the present application, the infrared radiation intensity of each pixel point can be calculated through the direct collision position and the infrared radiation intensity corresponding to each collision position, so as to generate the infrared image of the target object, thereby improving the rendering effect when generating the infrared image.

[0148] The embodiments of the present application also provide an electronic device, such as Figure 11 shown, including a processor 1101, a communication interface 1102, a memory 1103, and a communication bus 1104. Among them, the processor 1101, the communication interface 1102, and the memory 1103 complete mutual communication through the communication bus 1104.

[0149] The memory 1103 is used to store computer programs.

[0150] When the processor 1101 is used to execute the program stored on the memory 1103, the following steps are implemented:

[0151] Calculate the light rays emitted from the preset starting point and the multiple collision positions of the target object and the infrared radiation intensity corresponding to each collision position.

[0152] For each pixel point of the target object on the imaging plane, calculate the infrared radiation intensity of each pixel point according to the multiple collision positions and the infrared radiation intensity corresponding to each collision position.

[0153] Generate an infrared image of the target object according to the calculated infrared radiation intensity of each pixel point.

[0154] The communication bus mentioned in the above electronic device may be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. This communication bus can be divided into an address bus, a data bus, a control bus, etc. For the sake of convenience of representation, only a thick line is used in the figure, but it does not mean that there is only one bus or one type of bus.

[0155] The communication interface is used for communication between the above electronic device and other devices.

[0156] The memory may include a Random Access Memory (RAM), or may also include a Non-Volatile Memory (NVM), such as at least one disk memory. Optionally, the memory may also be at least one storage device located away from the aforementioned processor.

[0157] The aforementioned processor may be a general-purpose processor, including a Central Processing Unit (CPU), a Network Processor (NP), etc.; it may also be a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field-Programmable Gate Array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components.

[0158] In another embodiment provided by the present application, there is also provided a computer-readable storage medium, in which a computer program is stored, and when the computer program is executed by a processor, the steps of any of the above infrared image generation methods are implemented.

[0159] In another embodiment provided by the present application, there is also provided a computer program product containing instructions, which when run on a computer, causes the computer to execute any of the infrared image generation methods in the above embodiments.

[0160] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the processes or functions described in the embodiments of the present application are generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from a website, computer, server, or data center to another website, computer, server, or data center by wire (such as coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (such as infrared, wireless, microwave, etc.). The computer-readable storage medium can be any available medium that the computer can access or a data storage device such as a server or data center that includes one or more integrated available media. The available medium can be a magnetic medium (such as a floppy disk, hard disk, magnetic tape), an optical medium (such as a DVD), or a semiconductor medium (such as a solid state disk (SSD)).

[0161] It should be noted that, in this document, relational terms such as first and second are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the term "comprising", "including" or any other variation thereof is intended to cover non-exclusive inclusion, so that a process, method, article or device comprising a series of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article or device. Without further limitation, an element defined by the statement "comprising a..." does not exclude the existence of additional identical elements in the process, method, article or device comprising the element.

[0162] Each embodiment in this specification is described in a related manner. The same or similar parts among the embodiments can be referred to each other, and the differences between each embodiment and other embodiments are emphasized. In particular, for the embodiments of the device, electronic device, storage medium, and computer program product, since they are basically similar to the method embodiments, the description is relatively simple, and the relevant parts can refer to the partial description of the method embodiments.

[0163] The above are only the preferred embodiments of the present application and are not intended to limit the protection scope of the present application. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principle of the present application are all included in the protection scope of the present application.

Claims

1. An infrared image generation method, characterized in that, Including: Calculate the light emitted from a preset starting point and the first collision position of the target object, and the corresponding self-infrared radiation intensity, the first external light source direct irradiation infrared radiation intensity, the first external light source indirect irradiation infrared radiation intensity, the first surrounding object indirect irradiation infrared radiation intensity, and the first attenuation rate according to the preset scene parameters and material parameters of the target object; Calculate the corresponding first infrared radiation intensity according to the self-infrared radiation intensity corresponding to the first collision position, the first external light source direct irradiation infrared radiation intensity, the first external light source indirect irradiation infrared radiation intensity, the first surrounding object indirect irradiation infrared radiation intensity, and the first attenuation rate; Calculate the light emitted from the Nth collision point and the (N + 1)th collision position of the target object and the corresponding (N + 1)th infrared radiation intensity according to the preset scene parameters and the preset material parameters of the target object, where N is greater than or equal to 1; For each pixel point of the target object on the imaging plane, calculate the infrared radiation intensity of each pixel point according to multiple collision positions and the infrared radiation intensity corresponding to each collision position; Generate the infrared image of the target object according to the calculated infrared radiation intensity of each pixel point; Identify the target area corresponding to the first collision position of the target object in the imaging plane; Calibrate the infrared image of the target object according to the position of the target area to obtain the calibrated infrared image.

2. The method according to claim 1, characterized in that The calculating the light emitted from the Nth collision point and the (N + 1)th collision position of the target object and the corresponding (N + 1)th infrared radiation intensity according to the preset scene parameters and the preset material parameters of the target object includes: Calculate the light emitted from the Nth collision point and the (N + 1)th collision position of the target object, and the corresponding self-infrared radiation intensity, the second external light source direct irradiation infrared radiation intensity, the second external light source indirect irradiation infrared radiation intensity, the second surrounding object indirect irradiation infrared radiation intensity, and the second attenuation rate according to the preset material parameters of the target object; Calculate the (N + 1)th infrared radiation intensity according to the self-infrared radiation intensity corresponding to the (N + 1)th collision position, the second external light source direct irradiation infrared radiation intensity, the second external light source indirect irradiation infrared radiation intensity, the second surrounding object indirect irradiation infrared radiation intensity, and the second attenuation rate.

3. The method according to claim 1, wherein The calculating the infrared radiation intensity of each pixel point according to the multiple collision positions and the infrared radiation intensity corresponding to each collision position for each pixel point of the target object on the imaging plane includes: For any pixel point of the target object on the imaging plane, identify the target collision position corresponding to the pixel point among the multiple collision positions and the target infrared radiation intensity corresponding to the target collision position; Superimpose the target infrared radiation intensity to obtain the infrared radiation intensity of the pixel point.

4. The method according to claim 1, wherein After generating the infrared image of the target object according to the calculated infrared radiation intensity of each pixel point, the method further includes: Input the infrared image of the target object into the target recognition model to be trained, and obtain the recognition result of the target object; Update the parameters of the target recognition model to be trained according to the recognition result, and return to the step of inputting the infrared image of the target object into the target recognition model to be trained to obtain the recognition result of the target object, and continue to execute until the preset termination condition is met, and obtain the trained target recognition model.

5. An infrared image generation device, characterized in that, It includes: A collision position calculation module, configured to calculate multiple collision positions of the light rays emitted from a preset starting point and the target object, and the infrared radiation intensity corresponding to each collision position; A radiation intensity calculation module, configured to calculate the infrared radiation intensity of each pixel point on the imaging plane of the target object according to the multiple collision positions and the infrared radiation intensity corresponding to each collision position; An infrared image generation module, configured to generate the infrared image of the target object according to the calculated infrared radiation intensity of each pixel point; The collision position calculation module includes: A first collision calculation sub-module, configured to calculate the first collision position of the light rays emitted from a preset starting point and the target object and the corresponding first infrared radiation intensity according to the preset scene parameters and material parameters of the target object; An Nth collision calculation sub-module, configured to calculate the (N + 1)th collision position of the light rays emitted from the Nth collision point and the target object and the corresponding (N + 1)th infrared radiation intensity according to the preset scene parameters and the preset material parameters of the target object, where N is greater than or equal to 1; The first collision calculation sub-module includes: A first parameter calculation unit, configured to calculate the first collision position of the light rays emitted from a preset starting point and the target object, and the self-infrared radiation intensity, the first external light source direct irradiation infrared radiation intensity, the first external light source indirect irradiation infrared radiation intensity, the first surrounding object indirect irradiation infrared radiation intensity, and the first attenuation rate corresponding to the first collision position according to the preset scene parameters and material parameters of the target object; A first radiation intensity calculation unit, configured to calculate the first infrared radiation intensity according to the self-infrared radiation intensity corresponding to the first collision position, the first external light source direct irradiation infrared radiation intensity, the first external light source indirect irradiation infrared radiation intensity, the first surrounding object indirect irradiation infrared radiation intensity, and the first attenuation rate; A region recognition module, configured to recognize the target region corresponding to the first collision position in the imaging plane of the target object; An image calibration module, configured to calibrate the infrared image of the target object according to the position of the target region to obtain the calibrated infrared image.

6. An electronic device, characterized in that, It includes a processor, a communication interface, a memory, and a communication bus. Among them, the processor, the communication interface, and the memory complete communication with each other through the communication bus; The memory is used to store computer programs; When the processor is used to execute the program stored on the memory, it implements the method steps described in any one of claims 1-4.

7. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program, and when the computer program is executed by the processor, it implements the method steps described in any one of claims 1-4.

Images