An infrared remote sensing image generation method, device and electronic equipment

Abstract

This invention relates to the field of image generation technology, providing a method, apparatus, and electronic device for generating infrared remote sensing images. The method involves constructing multiple temperature prediction functions based on a set of predicted temperature parameters for multiple types of ground features within a target image area at preset times, surface temperature values, and multi-spectral reflectance of the ground features. The method calculates the radiance values ​​for each type of ground feature at multiple times based on the multi-spectral reflectance, temperature prediction functions, a set of temperature parameters to be predicted at multiple times, a set of infrared radiation parameters, and a preset radiance value calculation function. Initial infrared remote sensing images at multiple times are obtained based on these radiance values. A training set for an infrared remote sensing image generation model is constructed using the original visible light remote sensing image and the initial infrared remote sensing images at multiple times. The infrared remote sensing image generation model is trained using the training set to obtain a trained target infrared remote sensing image generation model.

Description

Technical Field

[0001] This invention relates to the field of image generation technology, and in particular to an infrared remote sensing image generation method, apparatus, and electronic device. Background Technology

[0002] Currently, infrared remote sensing technology, compared to visible light remote sensing technology, has unique advantages in penetrating fog, clouds, and other atmospheric interference, and is therefore widely used in environmental monitoring, disaster early warning, agricultural monitoring, military reconnaissance, and many other fields. With the development of artificial intelligence technology, infrared remote sensing images have demonstrated significant advantages in target detection and recognition tasks using network models, leading to a growing demand for high-quality infrared remote sensing images.

[0003] Based on this, existing technologies can generate infrared remote sensing images using physical models and deep learning networks. Specifically, the physical model method generates infrared remote sensing images by simulating the physical phenomena in the actual infrared radiation transmission process, while the deep learning network generates infrared remote sensing images by automatically learning the complex mapping relationship between visible light images and infrared images.

[0004] However, with existing technologies, physical model methods are highly susceptible to external environmental factors, and the temperature prediction models in physical model methods are complex and time-consuming. Deep learning networks rely on a large amount of data and suffer from overfitting and poor model generalization ability, resulting in poor image quality and low efficiency in the generated infrared remote sensing images. Summary of the Invention

[0005] Therefore, it is necessary to provide an infrared remote sensing image generation method, apparatus, and electronic device to address the aforementioned technical problems.

[0006] In a first aspect, embodiments of the present invention provide an infrared remote sensing image generation method, the method comprising:

[0007] Based on the set of temperature prediction parameters for multiple types of land cover materials in the target shooting area at a preset time, the surface temperature value, and the multi-spectral reflectance of the land cover materials, multiple temperature prediction functions are constructed. The set of temperature prediction parameters includes at least: atmospheric temperature, atmospheric humidity, and wind speed for multiple types of land cover materials at a preset time. The surface temperature value and the multi-spectral reflectance of the land cover materials are obtained from the original visible light remote sensing image and the corresponding original infrared remote sensing image of the target shooting area taken at the preset time.

[0008] Based on the multi-spectral reflectance of the land cover materials corresponding to multiple categories of land cover materials, the temperature prediction function, the set of temperature parameters to be predicted at multiple times, the set of infrared radiation parameters, and the preset radiance value calculation function, the radiance values ​​of multiple categories of land cover materials at multiple times are calculated. The set of infrared radiation parameters includes at least: solar radiance value, sky radiance value, atmospheric transmittance, and atmospheric path radiance value.

[0009] Based on multiple radiance values, initial infrared remote sensing images at multiple times are acquired using sensor imaging technology;

[0010] Based on the original visible light remote sensing image and the initial infrared remote sensing images at multiple times, a data training set for constructing an infrared remote sensing image generation model is built.

[0011] An infrared remote sensing image generation model is trained using a training data set to obtain a trained target infrared remote sensing image generation model. The target infrared remote sensing image generation model is used to generate target infrared remote sensing images based on visible light remote sensing images. The infrared remote sensing image generation model includes a generator and a discriminator. The generator includes at least a downsampling layer, an intermediate upsampling layer, and a multi-head attention layer. The downsampling layer, intermediate layer, and upsampling layer contain hollow residual blocks. The downsampling layer and the upsampling layer are connected by a skip connection for multi-head attention feature extraction.

[0012] In one embodiment, the surface temperature value and the multispectral reflectance of the ground material are obtained based on the original visible light remote sensing image and the corresponding original infrared remote sensing image of the target area captured at a preset time, including:

[0013] Acquire the original visible light remote sensing image of the target shooting area at a preset time, and the original infrared remote sensing image corresponding to the original visible light remote sensing image;

[0014] The original visible light remote sensing image contains multiple categories of land cover materials, and different categories of land cover materials are labeled.

[0015] Based on the original visible light remote sensing image and the original infrared remote sensing image, the surface temperature values ​​of multiple types of land cover materials at a preset time and the initial multi-spectral reflectance curves of the land cover materials are obtained, wherein the initial multi-spectral reflectance curves of the land cover materials are the first infrared band.

[0016] Based on the initial multi-spectral reflectance curve of the ground material and the preset multi-spectral reflectance curves of multiple preset unknown categories of ground materials in the preset software spectral data, the multi-spectral reflectance of the ground material is determined, wherein the multi-spectral reflectance curve of the ground material is the second infrared band, and the second infrared band is greater than the first infrared band.

[0017] In one embodiment, determining the multispectral reflectance of the land cover material based on the initial multispectral reflectance curve of the land cover material and the preset multispectral reflectance curves of multiple preset unknown categories of land cover materials in the preset software spectral data includes:

[0018] The initial multispectral reflectance curve of the ground feature material is matched with the preset multispectral reflectance curves of multiple preset unknown categories of ground feature materials. The preset unknown category ground feature material corresponding to the preset multispectral reflectance curve with the highest matching degree is determined as the ground feature material. The preset multispectral reflectance curve with the highest matching degree is the target multispectral reflectance curve. The preset multispectral reflectance curve is the third infrared band, and the third infrared band is greater than the second infrared band.

[0019] The multispectral reflectance of the ground material is determined based on the target multispectral reflectance curve.

[0020] In one embodiment, the construction of multiple temperature prediction functions based on a set of temperature prediction parameters for multiple types of land cover materials included in the target shooting area at a preset time, surface temperature values, and multi-spectral reflectance of the land cover materials includes:

[0021] By using the least squares method, the set of temperature prediction parameters, the surface temperature value, and the multi-spectral reflectance of the ground material at a preset time are fitted to obtain multiple temperature prediction factors corresponding to each type of ground material.

[0022] Multiple temperature prediction functions are constructed based on the temperature prediction parameter sets corresponding to multiple categories of land cover materials, the multi-spectral reflectance of the land cover materials, and the multiple temperature prediction factors.

[0023] In one embodiment, the step of calculating the radiance values ​​of multiple types of land cover materials at multiple times based on the multi-spectral reflectance of the land cover materials corresponding to multiple categories, the temperature prediction function, the set of temperature parameters to be predicted at multiple times, the set of infrared radiation parameters, and the preset radiance value calculation function includes:

[0024] Based on the multi-spectral reflectance of the land cover materials corresponding to multiple categories of land cover materials, and the set of temperature parameters to be predicted at multiple times, the predicted temperature values ​​of the multiple categories of land cover materials at multiple times are obtained through the temperature prediction function.

[0025] Based on the predicted temperature values, infrared radiation parameter sets, and preset radiance value calculation functions of multiple types of land cover materials at multiple times, the radiance values ​​of multiple types of land cover materials at multiple times are calculated.

[0026] In one embodiment, the step of calculating the radiance values ​​of multiple types of land cover materials at multiple times based on predicted temperature values, infrared radiation parameter sets, and a preset radiance value calculation function at multiple times includes:

[0027] Based on the predicted temperature values ​​of multiple types of ground features at multiple times, the radiance of multiple blackbody objects was calculated using Planck's formula.

[0028] Based on the radiance of multiple blackbody objects, the set of infrared radiation parameters, and the preset radiance value calculation function, the radiance values ​​of multiple types of ground features at multiple times are calculated.

[0029] In one embodiment, the step of constructing a data training set for an infrared remote sensing image generation model based on the original visible light remote sensing image and initial infrared remote sensing images at multiple times includes:

[0030] The original visible light remote sensing image and the initial infrared remote sensing image corresponding to each time moment are determined to be a data training subset;

[0031] A data training set for the infrared remote sensing image generation model is constructed based on the data training subsets corresponding to multiple time points.

[0032] In one embodiment, multi-head attention feature extraction is performed between the downsampling layer and the upsampling layer via a skip connection, including:

[0033] Determine the target upsampling sub-layer corresponding to each downsampling sub-layer;

[0034] The downsampled feature maps output by each downsampled sub-layer are input to the target upsampled sub-layer through the multi-head attention layer.

[0035] Secondly, embodiments of the present invention provide an infrared remote sensing image generation apparatus, comprising:

[0036] The function construction module is used to construct multiple temperature prediction functions based on the set of temperature prediction parameters for multiple types of land cover materials contained in the target shooting area at a preset time, the surface temperature value, and the multi-spectral reflectance of the land cover materials. The set of temperature prediction parameters includes at least: atmospheric temperature, atmospheric humidity, and wind speed for multiple types of land cover materials at a preset time. The surface temperature value and the multi-spectral reflectance of the land cover materials are obtained based on the original visible light remote sensing image and the corresponding original infrared remote sensing image of the target shooting area captured at the preset time.

[0037] The radiance value acquisition module is used to calculate the radiance values ​​of multiple types of land cover materials at multiple times based on the multi-spectral reflectance of the land cover materials corresponding to multiple categories of land cover materials, the temperature prediction function, the set of temperature parameters to be predicted at multiple times, the set of infrared radiation parameters, and the preset radiance value calculation function. The set of infrared radiation parameters includes at least: solar radiance value, sky radiance value, atmospheric transmittance, and atmospheric path radiance value.

[0038] The initial infrared remote sensing image acquisition module is used to acquire initial infrared remote sensing images at multiple times based on multiple radiance values ​​using sensor imaging technology.

[0039] The training set acquisition module is used to construct a data training set for the infrared remote sensing image generation model based on the original visible light remote sensing image and initial infrared remote sensing images at multiple times.

[0040] The training module is used to train an infrared remote sensing image generation model using a training dataset to obtain a trained target infrared remote sensing image generation model. The target infrared remote sensing image generation model is used to generate target infrared remote sensing images based on visible light remote sensing images. The infrared remote sensing image generation model includes a generator and a discriminator. The generator includes at least a downsampling layer, an intermediate upsampling layer, and a multi-head attention layer. The downsampling layer, intermediate layer, and upsampling layer contain dilated residual blocks. The downsampling layer and the upsampling layer are connected via skip connections for multi-head attention feature extraction.

[0041] Thirdly, embodiments of the present invention provide an electronic device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps of the infrared remote sensing image generation method described in the first aspect.

[0042] The technical solution provided by the embodiments of the present invention has the following advantages compared with the prior art:

[0043] This invention provides an infrared remote sensing image generation method, apparatus, and electronic device. This method constructs multiple temperature prediction functions based on a set of temperature prediction parameters for multiple types of ground features within a target imaging area at preset times, surface temperature values, and multi-spectral reflectance of the ground features. Based on the multi-spectral reflectance of the ground features corresponding to multiple types of ground features, the temperature prediction functions, the set of temperature prediction parameters at multiple times, the set of infrared radiation parameters, and a preset radiance value calculation function, the radiance values ​​of multiple types of ground features at multiple times are calculated, thereby obtaining initial infrared remote sensing images at multiple times. This reduces the number of parameters required to obtain the initial infrared remote sensing images, improving computational efficiency. Furthermore, by utilizing the original visible light remote sensing image and the initial infrared remote sensing images at multiple times, a training set for training the infrared remote sensing image generation model is constructed, enhancing the training set and improving the image quality of the target infrared remote sensing images generated by the trained target infrared remote sensing image generation model. Attached Figure Description

[0044] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with the invention and, together with the description, serve to explain the principles of the invention.

[0045] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0046] Figure 1 A flowchart illustrating an infrared remote sensing image generation method provided in an embodiment of the present invention;

[0047] Figure 2 A schematic diagram of a network model for a generator provided in an embodiment of the present invention;

[0048] Figure 3 This is a schematic diagram of an infrared remote sensing image generation device provided in an embodiment of the present invention. Detailed Implementation

[0049] To better understand the above-mentioned objectives, features, and advantages of the present invention, the solutions of the present invention will be further described below. It should be noted that, unless otherwise specified, the embodiments of the present invention and the features thereof can be combined with each other.

[0050] Many specific details are set forth in the following description in order to provide a full understanding of the invention, but the invention may also be practiced in other ways different from those described herein; obviously, the embodiments in the specification are only some embodiments of the invention, and not all embodiments.

[0051] Currently, the main methods for acquiring infrared remote sensing images are physical modeling and deep learning network methods. Physical modeling methods generate infrared remote sensing images by simulating the physical phenomena involved in real infrared radiation transmission, while deep learning network methods generate infrared remote sensing images by automatically learning the complex mapping relationship between visible light images and infrared images.

[0052] However, with existing technologies, physical model methods are highly susceptible to external environmental factors, and the temperature prediction models in physical model methods are complex and time-consuming. Deep learning networks rely on a large amount of data and suffer from overfitting and poor model generalization ability, resulting in poor image quality and low efficiency in the generated infrared remote sensing images.

[0053] Therefore, this invention provides an infrared remote sensing image generation method. It constructs multiple temperature prediction functions based on a set of temperature prediction parameters for multiple types of land cover materials within the target image area at preset times, surface temperature values, and multi-spectral reflectance of the land cover materials. Based on the multi-spectral reflectance of the land cover materials corresponding to multiple types of land cover materials, the temperature prediction functions, the set of temperature prediction parameters at multiple times, the set of infrared radiation parameters, and a preset radiance value calculation function, the radiance values ​​of multiple types of land cover materials at multiple times are calculated, thereby obtaining initial infrared remote sensing images at multiple times. This reduces the number of parameters required to obtain the initial infrared remote sensing images, improving computational efficiency. Furthermore, by utilizing the original visible light remote sensing image and the initial infrared remote sensing images at multiple times, a training set for training the infrared remote sensing image generation model is constructed, enhancing the training set and improving the image quality of the target infrared remote sensing images generated by the trained target infrared remote sensing image generation model.

[0054] In one embodiment, such as Figure 1 As shown, Figure 1 This is a flowchart illustrating an infrared remote sensing image generation method provided in an embodiment of the present invention, specifically including the following steps:

[0055] S10: Based on the set of temperature prediction parameters for multiple types of land cover materials in the target shooting area at a preset time, the surface temperature value, and the multi-spectral reflectance of the land cover materials, construct multiple temperature prediction functions.

[0056] The target shooting area refers to the area corresponding to the target object being photographed, such as grassland or city, but is not limited thereto. This invention does not specifically limit the scope, and those skilled in the art can set it according to the actual situation.

[0057] The above-mentioned land features refer to solid objects on the ground surface, and the land feature material refers to the material corresponding to the solid object. The land feature material can be, for example, water, buildings, roads, soil and grassland, but is not limited to these. This invention does not specifically limit the scope, and those skilled in the art can set it according to the actual situation.

[0058] The aforementioned preset time refers to a specific time, such as 8 o'clock, but is not limited thereto. This invention is not specifically limited, and those skilled in the art can set it according to the actual situation.

[0059] The aforementioned set of temperature prediction parameters includes at least the atmospheric temperature, atmospheric humidity, and wind speed of multiple types of land cover materials at a preset time. Among these, the atmospheric temperature, atmospheric humidity, and wind speed can be obtained from publicly available meteorological data websites.

[0060] The aforementioned surface temperature values ​​and multi-spectral reflectance of ground features were obtained based on the original visible light remote sensing images and corresponding original infrared remote sensing images of the target area captured at a preset time.

[0061] Multispectral reflectance of ground features refers to the reflectance information of ground features in different spectral bands, enabling precise identification and analysis of ground features and reflecting their ability to reflect light. Multispectral reflectance of ground features can be determined by the ratio of the radiance in a specific spectral band to the solar radiance incident on that band.

[0062] Specifically, at a preset time, the set of temperature prediction parameters corresponding to multiple categories of land cover materials in the target shooting area is obtained, namely, the atmospheric temperature, atmospheric humidity, wind speed, surface temperature value and multi-spectral reflectance of multiple categories of land cover materials at the preset time. Based on the atmospheric temperature, atmospheric humidity, wind speed, surface temperature value and multi-spectral reflectance of multiple categories of land cover materials at the preset time, temperature prediction functions corresponding to multiple categories of land cover materials are constructed.

[0063] S11: Based on the multi-spectral reflectance of multiple types of land cover materials, temperature prediction function, temperature prediction parameter set at multiple times, infrared radiation parameter set, and preset radiance value calculation function, calculate the radiance values ​​of multiple types of land cover materials at multiple times.

[0064] The set of temperature parameters to be predicted refers to the parameter information used to obtain the predicted temperature values ​​at multiple times. The set of temperature parameters to be predicted includes at least the atmospheric temperature, atmospheric humidity, and wind speed of multiple types of land cover materials at multiple times.

[0065] The aforementioned set of infrared radiation parameters includes at least: solar radiance value, sky radiance value, atmospheric transmittance, and atmospheric path radiance value. These parameters can be calculated using preset software such as MODTRAN. Specifically, multiple images are acquired from Landsat 8 data, including imaging time, latitude and longitude of the shooting area, solar altitude angle, solar zenith angle, observation zenith angle, and band information. The solar radiance value, sky radiance value, atmospheric transmittance, and atmospheric path radiance value are then calculated using preset software such as MODTRAN. However, this is not a limitation, and the present invention is not specifically limited. Those skilled in the art can set these parameters according to actual conditions.

[0066] The aforementioned preset radiance value calculation function refers to the function defined to calculate the radiance value. This preset radiance value calculation function can be defined as follows:

[0067] L a =[L b ×α+(L sun +L sky )×ρ]×τ+L p

[0068] In the formula, L b L represents the blackbody radiance; α represents the emissivity of the ground material; L sun Solar radiation intensity received by multiple types of ground features; L sky ρ represents the ambient radiance received by various types of ground features; τ represents the reflectance of the ground feature material; L represents the atmospheric transmittance. p α represents atmospheric path radiance, where α and ρ are not specifically limited in this invention, and can be set by those skilled in the art according to actual conditions.

[0069] The aforementioned radiance value refers to the infrared radiation energy of an object, and infrared remote sensing images can be obtained based on the radiance value.

[0070] S12: Based on multiple radiance values, acquire initial infrared remote sensing images at multiple times using sensor imaging technology.

[0071] Specifically, after obtaining the radiance values ​​of multiple types of land cover materials at multiple times, sensor imaging technology acquires initial infrared remote sensing images at multiple times.

[0072] S13: Construct a data training set for the infrared remote sensing image generation model based on the original visible light remote sensing image and the initial infrared remote sensing images at multiple times.

[0073] The training set is used to train an initial infrared remote sensing image generation model, which is a neural network model such as an adversarial network.

[0074] Optionally, based on the above embodiments, in some embodiments of the present invention, one implementation of S13 may be:

[0075] S131: Determine the original visible light remote sensing image and the corresponding initial infrared remote sensing image at each time point as a data training subset.

[0076] S132: Construct the data training set for the infrared remote sensing image generation model based on the data training subsets corresponding to multiple time points.

[0077] Specifically, for the original visible light remote sensing image and the initial infrared remote sensing images corresponding to the original visible light remote sensing image at multiple times, a data training subset is determined by identifying an original visible light remote sensing image and the initial infrared remote sensing images corresponding to each time. Then, for an original visible light remote sensing image and the initial infrared remote sensing images corresponding to the original visible light remote sensing image at multiple times, the data training subsets corresponding to each time are obtained, thus obtaining the data training set used to train the infrared remote sensing image generation model.

[0078] Thus, this embodiment enhances the training set by constructing a data training set containing the original visible light remote sensing image and the initial infrared remote sensing image corresponding to the original visible light remote sensing image at multiple times, thereby improving the training effect of the infrared remote sensing image generation model and thus improving the image quality of the infrared remote sensing image obtained by the infrared remote sensing image generation model.

[0079] S14: Train the infrared remote sensing image generation model using the data training set to obtain the trained target infrared remote sensing image generation model.

[0080] The target infrared remote sensing image generation model is used to generate target infrared remote sensing images based on visible light remote sensing images. Specifically, the visible light remote sensing images are input into the trained target infrared remote sensing image generation model, and the target infrared remote sensing images are generated using the target infrared remote sensing images.

[0081] The aforementioned infrared remote sensing image generation model includes a generator and a discriminator. The generator includes at least a downsampling layer, an intermediate upsampling layer, and a multi-head attention layer. The downsampling layer, intermediate layer, and upsampling layer contain hollow residual blocks. The hollow residual blocks replace the residual blocks of the original infrared remote sensing image generation model, thereby enabling the target infrared remote sensing image generation model to better extract spatial information of remote sensing images at different scales, improve feature extraction capabilities, and thus improve the image quality of the generated infrared remote sensing images.

[0082] The downsampling layer and the upsampling layer are connected via skip connections for multi-head attention feature extraction.

[0083] Among them, multi-head attention features refer to feature extraction through multi-head attention layers. This method can superimpose deep semantic features and shallow features of the input image from bottom to top to obtain more sufficient feature information, thereby improving the image quality of the generated infrared remote sensing image.

[0084] Optionally, based on the above embodiments, in some embodiments of the present invention, such as... Figure 2 As shown, the downsampling layer 21 contains multiple downsampling sub-layers 211, and the upsampling layer 23 contains multiple upsampling sub-layers 233. It should be noted that the number of downsampling sub-layers and upsampling sub-layers is the same. One way to implement multi-head attention feature extraction between the downsampling layer and the upsampling layer through skip connections is as follows:

[0085] S131: Determine the target upsampling sub-layer corresponding to each downsampling sub-layer.

[0086] S132: Input the downsampled feature maps output by each downsampled sub-layer into the target upsampled sub-layer through a multi-head attention layer.

[0087] Specifically, for a downsampling layer containing multiple downsampling sub-layers, a corresponding target upsampling sub-layer is determined among the multiple upsampling sub-layers. The downsampling feature maps output by each downsampling sub-layer are then processed by a multi-head attention layer to extract attention features before being input into the target upsampling sub-layer.

[0088] For example, continue to refer to Figure 2 As shown, the downsampling sub-layers 211 contained in the downsampling layer 21 are determined to be the corresponding target upsampling sub-layers 233 in the multiple upsampling sub-layers 233 contained in the upsampling layer 23. The downsampling feature maps output by each downsampling sub-layer 211 are then processed by the multi-head attention layer 24 for attention feature extraction and input to the target upsampling sub-layer 233.

[0089] Thus, this embodiment uses a skip connection between the downsampling layer and the upsampling layer to perform multi-head attention feature extraction, enabling the target infrared remote sensing image generation model to better extract spatial information of remote sensing images at different scales, improving feature extraction capabilities, and thereby improving the image quality of the generated infrared remote sensing images.

[0090] Thus, the infrared remote sensing image generation method provided in this embodiment constructs multiple temperature prediction functions by using a set of temperature prediction parameters for multiple types of land cover materials included in the target imaging area at preset times, surface temperature values, and multi-spectral reflectance of the land cover materials. Based on the multi-spectral reflectance of the land cover materials corresponding to multiple types of land cover materials, the temperature prediction functions, the set of temperature prediction parameters at multiple times, the set of infrared radiation parameters, and the preset radiance value calculation function, the radiance values ​​of multiple types of land cover materials at multiple times are calculated, thereby obtaining initial infrared remote sensing images at multiple times. This reduces the parameters required to obtain the initial infrared remote sensing images, improves computational efficiency, and further utilizes the original visible light remote sensing images and the initial infrared remote sensing images at multiple times to construct a data training set for training the infrared remote sensing image generation model. This enhances the data training set and improves the image quality of the target infrared remote sensing images generated by the trained target infrared remote sensing image generation model.

[0091] Optionally, based on the above embodiments, in some embodiments of the present invention, the surface temperature value and the multi-spectral reflectance of the ground material are obtained from the original visible light remote sensing image and the corresponding original infrared remote sensing image of the target shooting area taken at a preset time. One possible method is as follows:

[0092] S201: Acquire the original visible light remote sensing image of the target shooting area at a preset time, and the original infrared remote sensing image corresponding to the original visible light remote sensing image.

[0093] Specifically, for the target shooting area, acquire the original visible light remote sensing image taken at a preset time, and the original infrared remote sensing image taken at the same angle at the preset time.

[0094] For example, it can be obtained from publicly available datasets such as the Landsat8 dataset, but is not limited thereto. The present invention is not specifically limited, and those skilled in the art can set it according to the actual situation.

[0095] S202: Determine the multiple categories of land cover materials contained in the original visible light remote sensing image, and label the different categories of land cover materials.

[0096] The labeling process refers to labeling multiple categories of land cover materials using preset numbers and characters. These preset numbers and characters represent multiple categories of land cover materials. For example, for an original visible light remote sensing image, if it is determined that there are five categories of land cover materials including water, buildings, roads, soil, and grassland, then water, buildings, roads, soil, and grassland are labeled as 1, 2, 3, 4, and 5, respectively. However, this invention is not limited to these categories and those skilled in the art can set them according to the actual situation.

[0097] Specifically, after obtaining the original visible light remote sensing image of the target area taken at a preset time, the multiple categories of ground features contained in the original visible light remote sensing image are determined, and the different categories of ground features are marked.

[0098] Optionally, based on the above embodiments, in some embodiments of the present invention, the original visible light remote sensing image is classified by a semi-supervised grayscale threshold segmentation method to determine the multiple categories of ground features contained in the original visible light remote sensing image.

[0099] S203: Based on the original visible light remote sensing image and the original infrared remote sensing image, obtain the surface temperature values ​​of multiple types of ground materials at a preset time and the initial multi-spectral reflectance curves of the ground materials.

[0100] The initial ground material multi-spectral reflectance curve is the first infrared band. For example, the first infrared band can be 1.57um–1.65um, but it is not limited thereto. This invention does not specifically limit it, and those skilled in the art can set it according to the actual situation.

[0101] Specifically, after obtaining the original visible light remote sensing image and the original infrared remote sensing image, the surface temperature value and multi-spectral reflectance curve of multiple types of land cover materials at a preset time are obtained based on the original visible light remote sensing image and the original infrared remote sensing image.

[0102] Optionally, based on the above embodiments, in some embodiments of the present invention, for the original visible light remote sensing images and the original infrared remote sensing images, the surface temperature values ​​of multiple types of ground materials at a preset time and the initial multi-spectral reflectance curves of the ground materials are obtained by inversion technology.

[0103] S204: Determine the multispectral reflectance of the ground cover material based on the initial multispectral reflectance curve of the ground cover material and the preset multispectral reflectance curves of multiple preset unknown categories of ground cover materials in the preset software spectral data.

[0104] In this context, the multi-spectral reflectance curve of the ground material is the second infrared band, which is larger than the first infrared band. For example, following the above embodiment, the first infrared band can be 1.57um–1.65um, and the second infrared band is 1um–3um. Since the second infrared band is larger than the first infrared band, the multi-spectral reflectance curve of the ground material corresponding to the second infrared band can provide more image information when acquiring infrared remote sensing images in the future, thereby improving the quality of the generated infrared remote sensing images.

[0105] The aforementioned multiple preset unknown category land feature materials are multiple unknown category land feature materials.

[0106] Optionally, based on the above embodiments, in some embodiments of the present invention, one implementation of S204 may be:

[0107] The initial multispectral reflectance curve of the ground feature material is matched with the preset multispectral reflectance curves of multiple preset unknown categories of ground features material. The preset unknown category ground feature material corresponding to the preset multispectral reflectance curve with the highest matching degree is determined as the ground feature material, and the preset multispectral reflectance curve with the highest matching degree is determined as the target multispectral reflectance curve.

[0108] The preset multi-spectral reflectance curve is the third infrared band, which is larger than the second infrared band. For example, following the above embodiment, the first infrared band can be 1.57um–1.65um, the second infrared band is 1um–3um, and the third infrared band is 1um–14um, but it is not limited to this. The present invention is not specifically limited, and those skilled in the art can set it according to the actual situation.

[0109] Specifically, the multispectral reflectance curve of the ground feature material is matched with the preset multispectral reflectance curves of multiple preset unknown categories of ground features, and the preset multispectral reflectance curve of the preset unknown category of ground features with the highest matching degree is determined as the target multispectral reflectance curve. In other words, the preset unknown category of ground features with the highest matching degree is the ground feature material currently being matched.

[0110] For example, assuming the current land cover material is water, the multispectral reflectance curve corresponding to the water body is curve 1. The preset multispectral reflectance curves corresponding to multiple preset unknown land cover materials in the preset software spectral data are curves 2, 3, 4, 5, 6...n. Furthermore, the third infrared bands corresponding to curves 2, 3, 4, 5, 6... are all greater than the first infrared band corresponding to curve 1. Curve 1 is matched with curves 2, 3, 4, 5, 6...n, and the curve with the highest matching degree is determined to be curve 4. Therefore, the preset unknown land cover material corresponding to curve 4 is considered to be water, and curve 4 corresponding to the water body is the target multispectral reflectance curve. However, this is not limited to this; the present invention is not specifically limited, and those skilled in the art can set it according to the actual situation.

[0111] Based on the target's multispectral reflectance curve, determine the multispectral reflectance of the ground material.

[0112] Specifically, after obtaining the target multispectral reflectance curve, the multispectral reflectance of the ground material is determined based on the target multispectral reflectance curve.

[0113] Optionally, based on the above embodiments, in some embodiments of the present invention, one way to determine the multispectral reflectance of ground material according to the target multispectral reflectance curve may be:

[0114] The multi-spectral reflectance curve of the target is sampled, and the average value of the multiple reflectance values ​​obtained from the sampling is calculated. The average value is the multi-spectral reflectance of the ground material.

[0115] Thus, the infrared remote sensing image generation method provided by the present invention matches the initial multispectral reflectance curve of the ground object material with the ground object material, obtains the preset multispectral reflectance curve of the preset unknown category ground object material with a high matching degree as the target multispectral reflectance curve, and further determines the multispectral reflectance of the ground object material based on the target multispectral reflectance curve. Since the second infrared band corresponding to the target multispectral reflectance curve is greater than the first infrared band of the initial ground object material multispectral reflectance curve, the obtained multispectral reflectance of the ground object material can provide more image information when acquiring infrared remote sensing images in the future, thereby improving the quality of the generated infrared remote sensing images.

[0116] Optionally, based on the above embodiments, in some embodiments of the present invention, one implementation of S10 may be:

[0117] S101: Using the least squares method, the temperature prediction parameter set, surface temperature value and multi-band reflectance of each type of land cover material at a preset time are fitted to obtain multiple temperature prediction factors corresponding to each type of land cover material.

[0118] S102: Construct multiple temperature prediction functions based on the temperature prediction parameter sets corresponding to multiple categories of ground materials, the multi-spectral reflectance of ground materials, and multiple temperature prediction factors.

[0119] Specifically, the set of temperature prediction parameters for each category of land cover material at a preset time is obtained, such as atmospheric temperature, atmospheric humidity, and wind speed for multiple categories of land cover materials at a preset time. The least squares method is used to fit the atmospheric temperature, atmospheric humidity, wind speed, surface temperature value, and multi-spectral reflectance of the land cover materials at the preset time, thereby obtaining multiple temperature prediction factors for each category of land cover material at the preset time. Based on the set of temperature prediction parameters corresponding to each category of land cover material, the multi-spectral reflectance of the land cover materials, and the multiple temperature prediction factors, a temperature prediction function corresponding to each category of land cover material is constructed.

[0120] Optionally, based on the above embodiments, in some embodiments of the present invention, the temperature prediction function can be defined by the following expression:

[0121] T a =a0+a1T air +a2H+a3V+a4ρ

[0122] Among them, T airH represents atmospheric temperature, V represents humidity, ρ represents wind speed, ρ represents shortwave reflectance of ground material, and a0, a1, a2, a3, and a4 represent temperature prediction factors. a0, a1, a2, a3, and a4 are obtained through fitting.

[0123] Thus, this embodiment constructs temperature prediction functions corresponding to each type of land cover material. With a small number of parameters and temperature prediction functions, it can obtain the predicted temperature of each type of land cover material at multiple times, thereby improving computational efficiency.

[0124] Optionally, based on the above embodiments, in some embodiments of the present invention, one implementation of S11 may be:

[0125] S111: Based on the multi-spectral reflectance of multiple types of land cover materials and the set of temperature parameters to be predicted at multiple times, the predicted temperature values ​​of multiple types of land cover materials at multiple times are obtained through the temperature prediction function.

[0126] S112: Based on the predicted temperature values, infrared radiation parameter sets, and preset radiance value calculation functions of multiple types of land cover materials at multiple times, the radiance values ​​of multiple types of land cover materials at multiple times are calculated.

[0127] Specifically, the multi-spectral reflectance of multiple land cover materials and a set of temperature prediction parameters at multiple times are obtained for each category of land cover material. These parameters include atmospheric temperature, humidity, and wind speed at multiple times for each category of land cover material. The multi-spectral reflectance, atmospheric temperature, humidity, and wind speed are then substituted into a temperature prediction function to obtain the predicted temperature values ​​for each category of land cover material at multiple times. After obtaining the predicted temperature values ​​for each category of land cover material at multiple times, the predicted temperature values ​​and the infrared radiation parameter set are substituted into a preset radiance value calculation function to calculate the radiance values ​​for each category of land cover material at multiple times.

[0128] Optionally, based on the above embodiments, in some embodiments of the present invention, one implementation of S112 may be:

[0129] Based on the predicted temperature values ​​of multiple types of ground features at multiple times, the radiance of multiple blackbody objects was calculated using Planck's formula.

[0130] Based on multiple sets of blackbody radiance and infrared radiation parameters, as well as a preset radiance value calculation function, the radiance values ​​of multiple types of ground features at multiple times are calculated.

[0131] Specifically, the predicted temperature values ​​of multiple types of land cover materials at multiple times are substituted into Planck's formula to calculate the blackbody radiance of each type of land cover material at multiple times. The multiple sets of blackbody radiance and infrared radiation parameters are then substituted into a preset radiance value calculation function to calculate the radiance values ​​of multiple types of land cover materials at multiple times.

[0132] It should be understood that, although Figure 1 The steps in the flowchart are shown sequentially as indicated by the arrows, but these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order in which these steps are executed, and they can be performed in other orders. Figure 1 At least some of the steps in the process may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these sub-steps or stages is not necessarily sequential, but can be executed in turn or alternately with other steps or at least some of the sub-steps or stages of other steps.

[0133] In one embodiment, such as Figure 3 As shown, an infrared remote sensing image generation device is provided, including: a function construction module 10, a radiance value acquisition module 11, an initial infrared remote sensing image acquisition module 12, a training set acquisition module 13, and a training module 14.

[0134] The function construction module 10 is used to construct multiple temperature prediction functions based on the set of temperature prediction parameters for multiple types of land cover materials included in the target shooting area at a preset time, the surface temperature value, and the multi-spectral reflectance of the land cover materials. The set of temperature prediction parameters includes at least the atmospheric temperature, atmospheric humidity, and wind speed of multiple types of land cover materials at a preset time. The surface temperature value and the multi-spectral reflectance of the land cover materials are obtained from the original visible light remote sensing image and the corresponding original infrared remote sensing image of the target shooting area captured at the preset time.

[0135] The radiance value acquisition module 11 is used to calculate the radiance values ​​of multiple types of land cover materials at multiple times based on the multi-spectral reflectance of the land cover materials corresponding to multiple categories of land cover materials, the temperature prediction function, the set of temperature parameters to be predicted at multiple times, the set of infrared radiation parameters, and the preset radiance value calculation function. The set of infrared radiation parameters includes at least: solar radiance value, sky radiance value, atmospheric transmittance, and atmospheric path radiance value.

[0136] The initial infrared remote sensing image acquisition module 12 is used to acquire initial infrared remote sensing images at multiple times based on multiple radiance values ​​using sensor imaging technology.

[0137] The training set acquisition module 13 is used to construct a data training set for the infrared remote sensing image generation model based on the original visible light remote sensing image and initial infrared remote sensing images at multiple times.

[0138] Training module 14 is used to train an infrared remote sensing image generation model using a data training set to obtain a trained target infrared remote sensing image generation model. The target infrared remote sensing image generation model is used to generate target infrared remote sensing images based on visible light remote sensing images. The infrared remote sensing image generation model includes a generator and a discriminator. The generator includes at least a downsampling layer, an intermediate upsampling layer, and a multi-head attention layer. The downsampling layer, intermediate layer, and upsampling layer contain dilated residual blocks. The downsampling layer and the upsampling layer are connected via skip connections for multi-head attention feature extraction.

[0139] In the above embodiments, the function construction module constructs multiple temperature prediction functions based on the set of temperature prediction parameters for multiple types of land cover materials included in the target imaging area at preset times, surface temperature values, and multi-spectral reflectance of the land cover materials. The radiance value acquisition module calculates the radiance values ​​of multiple types of land cover materials at multiple times based on the multi-spectral reflectance of the land cover materials corresponding to multiple types of land cover materials, the temperature prediction functions, the set of temperature prediction parameters at multiple times, the set of infrared radiation parameters, and the preset radiance value calculation function. Thus, the initial infrared remote sensing images at multiple times are obtained through the initial infrared remote sensing image acquisition module, which can reduce the parameters for obtaining the initial infrared remote sensing images and improve computational efficiency. Furthermore, the training set acquisition module uses the original visible light remote sensing images and the initial infrared remote sensing images at multiple times to construct a data training set for training the infrared remote sensing image generation model, enhancing the data training set and improving the image quality of the target infrared remote sensing images generated by the trained target infrared remote sensing image generation model.

[0140] Specific limitations regarding the infrared remote sensing image generation device can be found in the limitations of the infrared remote sensing image generation method described above, and will not be repeated here. Each module in the aforementioned server can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in hardware or independently of the processor in the computer device, or stored in software in the memory of the computer device, so that the processor can call and execute the corresponding operations of each module.

[0141] This invention provides an electronic device, including: a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it can implement the infrared remote sensing image generation method provided in this invention. For example, when the processor executes the computer program, it can implement... Figure 1The technical solutions of any of the method embodiments shown are similar in implementation principle and technical effect, and will not be described again here.

[0142] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the methods described above. Any references to memory, databases, or other media used in the embodiments provided by this invention can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, or optical storage, etc. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms, such as static random access memory (SRAM) and dynamic random access memory (DRAM), etc.

[0143] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0144] The embodiments described above are merely illustrative of several implementations of the present invention, and while the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the invention patent. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the present invention, and these all fall within the protection scope of the present invention. Therefore, the protection scope of this invention patent should be determined by the appended claims.

Claims

1. A method for generating infrared remote sensing images, characterized in that, include: Based on the set of temperature prediction parameters for multiple types of land cover materials in the target shooting area at a preset time, the surface temperature value, and the multi-spectral reflectance of the land cover materials, multiple temperature prediction functions are constructed. The set of temperature prediction parameters includes at least: atmospheric temperature, atmospheric humidity, and wind speed for multiple types of land cover materials at a preset time. The surface temperature value and the multi-spectral reflectance of the land cover materials are obtained from the original visible light remote sensing image and the corresponding original infrared remote sensing image of the target shooting area taken at the preset time. Based on the multi-spectral reflectance of the land cover materials corresponding to multiple categories of land cover materials, the temperature prediction function, the set of temperature parameters to be predicted at multiple times, the set of infrared radiation parameters, and the preset radiance value calculation function, the radiance values ​​of multiple categories of land cover materials at multiple times are calculated. The set of infrared radiation parameters includes at least: solar radiance value, sky radiance value, atmospheric transmittance, and atmospheric path radiance value. Based on multiple radiance values, initial infrared remote sensing images at multiple times are acquired using sensor imaging technology; Based on the original visible light remote sensing image and the initial infrared remote sensing images at multiple times, a data training set for constructing an infrared remote sensing image generation model is built. An infrared remote sensing image generation model is trained using a training data set to obtain a trained target infrared remote sensing image generation model. The target infrared remote sensing image generation model is used to generate target infrared remote sensing images based on visible light remote sensing images. The infrared remote sensing image generation model includes a generator and a discriminator. The generator includes at least a downsampling layer, an intermediate upsampling layer, and a multi-head attention layer. The downsampling layer, intermediate layer, and upsampling layer contain hollow residual blocks. The downsampling layer and the upsampling layer are connected by a skip connection for multi-head attention feature extraction.

2. The method according to claim 1, characterized in that, The surface temperature value and the multi-spectral reflectance of the ground material are obtained based on the original visible light remote sensing image and the corresponding original infrared remote sensing image of the target area captured at a preset time, including: Acquire the original visible light remote sensing image of the target shooting area at a preset time, and the original infrared remote sensing image corresponding to the original visible light remote sensing image; The original visible light remote sensing image contains multiple categories of land cover materials, and different categories of land cover materials are labeled. Based on the original visible light remote sensing image and the original infrared remote sensing image, the surface temperature values ​​of multiple types of land cover materials at a preset time and the initial multi-spectral reflectance curves of the land cover materials are obtained, wherein the initial multi-spectral reflectance curves of the land cover materials are the first infrared band. Based on the initial multi-spectral reflectance curve of the ground material and the preset multi-spectral reflectance curves of multiple preset unknown categories of ground materials in the preset software spectral data, the multi-spectral reflectance of the ground material is determined, wherein the multi-spectral reflectance curve of the ground material is the second infrared band, and the second infrared band is greater than the first infrared band.

3. The method according to claim 2, characterized in that, The step of determining the multispectral reflectance of the land cover material based on the initial multispectral reflectance curve of the land cover material and the preset multispectral reflectance curves of multiple preset unknown categories of land cover materials in the preset software spectral data includes: The initial multispectral reflectance curve of the ground feature material is matched with the preset multispectral reflectance curves of multiple preset unknown categories of ground feature materials. The preset unknown category ground feature material corresponding to the preset multispectral reflectance curve with the highest matching degree is determined as the ground feature material. The preset multispectral reflectance curve with the highest matching degree is the target multispectral reflectance curve. The preset multispectral reflectance curve is the third infrared band, and the third infrared band is greater than the second infrared band. The multispectral reflectance of the ground material is determined based on the target multispectral reflectance curve.

4. The method according to claim 1, characterized in that, The method involves constructing multiple temperature prediction functions based on a set of temperature prediction parameters for multiple types of land cover materials within the target shooting area at a preset time, surface temperature values, and multi-spectral reflectance of the land cover materials. These functions include: By using the least squares method, the set of temperature prediction parameters, the surface temperature value, and the multi-spectral reflectance of the ground material at a preset time are fitted to obtain multiple temperature prediction factors corresponding to each type of ground material. Multiple temperature prediction functions are constructed based on the temperature prediction parameter sets corresponding to multiple categories of land cover materials, the multi-spectral reflectance of the land cover materials, and the multiple temperature prediction factors.

5. The method according to claim 1, characterized in that, The calculation of the radiance values ​​of multiple types of land cover materials at multiple times is based on the multi-spectral reflectance of the land cover materials corresponding to multiple categories, the temperature prediction function, the set of temperature parameters to be predicted at multiple times, the set of infrared radiation parameters, and the preset radiance value calculation function. This includes: Based on the multi-spectral reflectance of the land cover materials corresponding to multiple categories of land cover materials, and the set of temperature parameters to be predicted at multiple times, the predicted temperature values ​​of the multiple categories of land cover materials at multiple times are obtained through the temperature prediction function. Based on the predicted temperature values, infrared radiation parameter sets, and preset radiance value calculation functions of multiple types of land cover materials at multiple times, the radiance values ​​of multiple types of land cover materials at multiple times are calculated.

6. The method according to claim 5, characterized in that, The calculation of radiance values ​​for multiple types of land cover materials at multiple times, based on predicted temperature values, infrared radiation parameter sets, and a preset radiance value calculation function at multiple times, includes: Based on the predicted temperature values ​​of multiple types of ground features at multiple times, the radiance of multiple blackbody objects was calculated using Planck's formula. Based on the radiance of multiple blackbody objects, the set of infrared radiation parameters, and the preset radiance value calculation function, the radiance values ​​of multiple types of ground features at multiple times are calculated.

7. The method according to claim 1, characterized in that, The data training set for constructing the infrared remote sensing image generation model based on the original visible light remote sensing image and initial infrared remote sensing images at multiple times includes: The original visible light remote sensing image and the initial infrared remote sensing image corresponding to each time moment are determined to be a data training subset; A data training set for the infrared remote sensing image generation model is constructed based on the data training subsets corresponding to multiple time points.

8. The method according to claim 1, characterized in that, The downsampling layer and the upsampling layer are connected via a skip connection for multi-head attention feature extraction, including: Determine the target upsampling sub-layer corresponding to each downsampling sub-layer; The downsampled feature maps output by each downsampled sub-layer are input to the target upsampled sub-layer through the multi-head attention layer.

9. An infrared remote sensing image generation device, characterized in that, include: The function construction module is used to construct multiple temperature prediction functions based on the set of temperature prediction parameters for multiple types of land cover materials contained in the target shooting area at a preset time, the surface temperature value, and the multi-spectral reflectance of the land cover materials. The set of temperature prediction parameters includes at least: atmospheric temperature, atmospheric humidity, and wind speed for multiple types of land cover materials at a preset time. The surface temperature value and the multi-spectral reflectance of the land cover materials are obtained based on the original visible light remote sensing image and the corresponding original infrared remote sensing image of the target shooting area captured at the preset time. The radiance value acquisition module is used to calculate the radiance values ​​of multiple types of land cover materials at multiple times based on the multi-spectral reflectance of the land cover materials corresponding to multiple categories of land cover materials, the temperature prediction function, the set of temperature parameters to be predicted at multiple times, the set of infrared radiation parameters, and the preset radiance value calculation function. The set of infrared radiation parameters includes at least: solar radiance value, sky radiance value, atmospheric transmittance, and atmospheric path radiance value. The initial infrared remote sensing image acquisition module is used to acquire initial infrared remote sensing images at multiple times based on multiple radiance values ​​using sensor imaging technology. The training set acquisition module is used to construct a data training set for the infrared remote sensing image generation model based on the original visible light remote sensing image and initial infrared remote sensing images at multiple times. The training module is used to train an infrared remote sensing image generation model using a training dataset to obtain a trained target infrared remote sensing image generation model. The target infrared remote sensing image generation model is used to generate target infrared remote sensing images based on visible light remote sensing images. The infrared remote sensing image generation model includes a generator and a discriminator. The generator includes at least a downsampling layer, an intermediate upsampling layer, and a multi-head attention layer. The downsampling layer, intermediate layer, and upsampling layer contain dilated residual blocks. The downsampling layer and the upsampling layer are connected via skip connections for multi-head attention feature extraction.

10. An electronic device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 8.

Images