A thin cloud area thermal infrared band reconstruction method, system, device and medium
By generating false-color reference images and establishing a clear-sky similar pixel library, the applicability and stability issues of the thermal infrared band reconstruction method in thin cloud areas were resolved, achieving accurate reconstruction and spatial continuity of thermal infrared brightness temperature data.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- GUANGZHOU URBAN PLANNING & DESIGN SURVEY RES INST
- Filing Date
- 2026-03-25
- Publication Date
- 2026-06-09
Smart Images
Figure CN122176106A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of remote sensing image processing technology, and in particular to a method, system, device and medium for reconstructing thermal infrared bands in thin cloud areas. Background Technology
[0002] Thermal infrared remote sensing imagery reflects surface thermal radiation information and is an important data source for applications such as surface temperature retrieval, evapotranspiration estimation, and urban thermal environment monitoring. However, under thin cloud cover, the absorption and scattering of thermal infrared radiation by clouds introduces brightness-temperature deviations, making it difficult to accurately characterize the true surface thermal conditions in thin cloud areas and resulting in discontinuous spatial distribution of the thermal field. To address this issue, existing technologies can be divided into two categories: thermal infrared band thin cloud cover area reconstruction methods based on multi-source / multi-temporal data and thermal infrared band thin cloud cover area reconstruction methods based on single-temporal data.
[0003] Multi-source / multi-temporal methods typically incorporate auxiliary data such as nearby temporal thermal infrared images, surface temperature sequences, multi-sensor observation data, or atmospheric reanalysis data. They achieve the completion and smoothing of thermal information in thin cloud regions through time series fitting, spatiotemporal fusion, or multi-source constraints. While these methods can achieve good spatial continuity in some scenarios, their applicability is highly dependent on the availability and consistency of the auxiliary data. In scenarios with continuous cloud cover, long satellite revisit periods, or spatiotemporal resolution differences and radiometric consistency deviations in multi-source data, these methods are prone to problems such as scale mismatch, loss of surface details, or error propagation. Methods based on single-temporal data, on the other hand, do not rely on additional data. They primarily weaken the impact of thin clouds by mining information within a single image. These methods can be further divided into two technical approaches: physical model-based and data-driven. Physical model-based methods typically correct the influence of clouds and the atmosphere based on radiative transfer relationships, but they require prior parameters such as cloud optical properties and atmospheric profiles. When these parameters are difficult to obtain or have large errors, the correction results lack stability, and their generalization ability to different regions and imaging conditions is limited. Data-driven methods primarily utilize the correlation between visible light, near-infrared multispectral information, and thermal infrared information within the same scene. They establish mapping relationships or similarity measures in clear-sky areas and extrapolate these to thin cloud regions for thermal infrared estimation and reconstruction. Typical approaches include end-to-end deep learning methods. However, existing data-driven methods often suffer from several drawbacks. First, most assume that multispectral-thermal infrared relationships established under clear-sky conditions can be directly transferred to thin cloud regions. Thin clouds alter the observational characteristics of visible light and near-infrared, causing feature drift and matching distortion, leading to unstable matching and large errors in cloud regions. Second, many methods equate cloud regions with information-deficient areas for replacement and completion, failing to utilize the fact that thermal infrared still contains some effective surface information under the "semi-transparent" nature of thin clouds. This easily leads to the loss of usable information from the original observations, resulting in block artifacts or over-smoothing issues. Furthermore, deep learning methods rely on a large number of labeled samples and complex training strategies, exhibiting weak cross-regional and cross-seasonal generalization capabilities.
[0004] In summary, existing methods for reconstructing thermal infrared bands in thin cloud regions suffer from limitations in applicability, insufficient stability, weak generalization, and the inability to collaboratively utilize information from multiple bands. Summary of the Invention
[0005] The purpose of this invention is to provide a method, system, device, and medium for thermal infrared band reconstruction in thin cloud regions, thereby improving the robustness, generalizability, and applicability of thermal infrared reconstruction in thin cloud regions.
[0006] To achieve the above objectives, the present invention provides a method for reconstructing the thermal infrared band in thin cloud regions, comprising: A false-color reference image is generated based on the thin cloud region of the input image; Based on the false-color reference image, within the clear sky region, according to the matching constraints, clear sky similar pixels are found for each pixel to be reconstructed in the thin cloud region, forming a clear sky similar pixel library. Based on the clear sky similar pixel library, the brightness temperature of each pixel to be reconstructed is predicted in the thermal infrared band to obtain the predicted brightness temperature value of each pixel to be reconstructed in the thermal infrared band. Based on the predicted brightness temperature value and the original observed brightness temperature value of each pixel to be reconstructed, the brightness temperature deviation field of the thin cloud region is determined; The brightness temperature deviation field is spatially smoothed, and the reconstructed thermal infrared brightness temperature data of the thin cloud region is calculated based on the smoothed brightness temperature deviation field and the original observed brightness temperature value.
[0007] Optionally, generating a false-color reference image based on the thin cloud region of the input image includes: Based on the texture features of pixels within the thin cloud region of the input image, the thin cloud region is divided into multiple sub-regions; For each sub-region, cloud and fog radiation perturbation components in each visible light band are extracted, and based on the perturbation components, the relative relationship between cloud and fog radiation perturbation components in different bands is established. Based on the relative relationship, the visible light band of the input image is processed to generate a differential feature image that cancels out thin cloud radiation interference. At least three sets of the differential feature images are selected as the three channels of the false color image and processed to generate a false color reference image.
[0008] Optionally, processing the visible light band of the input image according to the relative relationship to generate a differential feature image that cancels out thin cloud radiation interference includes: Select any two different visible light bands in the input image as the target band pair; Based on the relative relationship, a linear transformation is performed on the first visible light band in the target band pair so that the thin cloud radiation interference intensity of the first visible light band is aligned with the thin cloud radiation interference intensity of the second visible light band in the target band pair in terms of order of magnitude. The second visible light band is differentially analyzed with the aligned first visible light band to generate a differential feature image that cancels out thin cloud radiation interference.
[0009] Optionally, the step of using the false-color reference image as a reference, and within a spatially adjacent clear-sky region, searching for clear-sky similar pixels for the pixels to be reconstructed in the thin cloud region according to matching constraints to form a clear-sky similar pixel library, includes: Based on the false-color reference image, and centered on the pixel to be reconstructed in the thin cloud region, a candidate set of clear sky pixels is retrieved in the spatially adjacent clear sky region according to matching constraints. Calculate the comprehensive matching weight between the pixel to be reconstructed and each candidate clear sky pixel based on the matching constraints; Based on the comprehensive matching weight, a predetermined number of candidate clear sky pixels are selected from the candidate clear sky pixel set as clear sky similar pixels corresponding to the pixel to be reconstructed, forming a clear sky similar pixel library.
[0010] Optionally, the matching constraints include false color numerical consistency constraints, visible light scattering constraints, and spatial proximity constraints: The false color numerical consistency constraint includes requiring that the difference in false color channel values between the pixel to be reconstructed and the candidate clear sky pixel does not exceed a preset false color channel threshold. The visible light scattering constraint includes sorting the visible light bands from shortest to longest center wavelength, and within a preset tolerance range, requiring that the difference between the pixel to be reconstructed and the candidate clear sky pixel in each band decreases as the wavelength increases. The spatial proximity constraint includes prioritizing candidate clear-sky pixels that are spatially closer to the pixel to be reconstructed.
[0011] Optionally, the step of predicting the brightness temperature of each pixel to be reconstructed in the thermal infrared band based on the clear-sky similar pixel library, to obtain the predicted brightness temperature value of each pixel to be reconstructed in the thermal infrared band, includes: Obtain the original observed brightness temperature of each clear-sky similar pixel in the thermal infrared band in the clear-sky similar pixel library; Based on the original observed brightness temperature of each clear-sky similar pixel and the corresponding comprehensive matching weight, the predicted brightness temperature value of the pixel to be reconstructed in the thermal infrared band is obtained.
[0012] Optionally, determining the brightness temperature deviation field of the thin cloud region based on the predicted brightness temperature value and the original observed brightness temperature value of each pixel to be reconstructed includes: Calculate the difference between the predicted brightness temperature value and the original observed brightness temperature value for each pixel to be reconstructed, and use it as the initial value of the brightness temperature deviation of the pixel to be reconstructed due to thin cloud interference. The initial values of brightness temperature deviation of all pixels to be reconstructed within the thin cloud region are summarized, and the brightness temperature deviation field of the thin cloud region is determined according to the spatial position of each pixel to be reconstructed in the input image.
[0013] To achieve the above objectives, the present invention also provides a thermal infrared band reconstruction system for thin cloud regions, comprising: The reference image generation module is used to generate a false-color reference image based on the thin cloud region of the input image; The clear sky similar image filtering module is used to find clear sky similar pixels for each pixel to be reconstructed in the thin cloud region based on the false color reference image and matching constraints within the clear sky region, thereby forming a clear sky similar pixel library. The brightness temperature prediction module is used to predict the brightness temperature of each pixel to be reconstructed in the thermal infrared band based on the clear sky similar pixel library, so as to obtain the predicted brightness temperature value of each pixel to be reconstructed in the thermal infrared band. The brightness temperature deviation determination module is used to determine the brightness temperature deviation field of the thin cloud region based on the predicted brightness temperature value and the original observed brightness temperature value of each pixel to be reconstructed. The reconstruction module is used to perform spatial smoothing on the brightness temperature deviation field, and calculate the reconstructed thermal infrared brightness temperature data of the thin cloud region based on the smoothed brightness temperature deviation field and the original observed brightness temperature value.
[0014] To achieve the above objectives, the present invention also provides a terminal device, including a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor, wherein the processor executes the computer program to implement the thin cloud region thermal infrared band reconstruction method as described above.
[0015] To achieve the above objectives, the present invention also provides a computer-readable storage medium comprising a stored computer program, wherein, when the computer program is executed, it controls the device where the computer-readable storage medium is located to perform the thin cloud region thermal infrared band reconstruction method as described above.
[0016] Compared with existing technologies, this invention provides a method, system, device, and medium for thermal infrared band reconstruction of thin cloud areas. Firstly, this method utilizes the relatively stable relationship of interference intensity of thin clouds in the visible light band. Through band mapping and differential generation, a false-color reference image unaffected by thin clouds is generated, solving the problem that existing data-driven methods cannot establish a reliable matching benchmark due to visible light feature drift caused by thin clouds. Secondly, using this reference image as a bridge, physically meaningful clear-sky similar pixels are matched for thin cloud pixels under both spatial and spectral constraints, improving the robustness and accuracy of the matching. Finally, a reconstruction path with brightness temperature deviation correction is adopted, which fully utilizes the effective information from the original thermal infrared observations under thin clouds and, by smoothing the deviation field, effectively suppresses matching noise and block artifacts while preserving the spatial continuity of true surface thermal details to the greatest extent. This invention can improve the robustness, generalization, and applicability of thermal infrared reconstruction in thin cloud areas. Attached Figure Description
[0017] To more clearly illustrate the technical solution of the present invention, the accompanying drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0018] Figure 1 This is a flowchart of a method for reconstructing thermal infrared bands in thin cloud regions provided by an embodiment of the present invention; Figure 2 This is a schematic diagram of a thin cloud region texture segmentation process provided in an embodiment of the present invention; Figure 3 This is a schematic diagram illustrating the process of establishing the relative relationship of cloud and fog radiation disturbance components between bands, provided by an embodiment of the present invention. Figure 4 This is a schematic diagram of the process for aligning inter-band cloud and fog radiation disturbances and generating false-color reference images provided in an embodiment of the present invention; Figure 5 This is a schematic diagram illustrating the construction and similarity relationship transfer process of a clear sky similar pixel library provided in an embodiment of the present invention; Figure 6 This is a schematic diagram of a calculation process for thermal infrared brightness temperature reconstruction in a thin cloud region provided by an embodiment of the present invention; Figure 7 This is a structural block diagram of a thermal infrared band reconstruction system for thin cloud regions provided in an embodiment of the present invention; Figure 8 This is a structural block diagram of a terminal device provided in an embodiment of the present invention. Detailed Implementation
[0019] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.
[0020] See Figure 1 , Figure 1 This is a flowchart of a method for reconstructing the thermal infrared band in a thin cloud region according to an embodiment of the present invention. The method includes steps S1 to S5: Step S1: Generate a false-color reference image based on the thin cloud region of the input image; In one optional embodiment, step S1 includes steps S101 to S105: Step S101: Based on the texture features of the pixels within the thin cloud region of the input image, divide the thin cloud region into multiple sub-regions; Understandably, due to the semi-transparent nature of thin clouds and their slow spatial intensity variation, the spectral and textural features of the mixture of thin clouds and the ground surface differ. Based on this inherent characteristic, this embodiment of the invention employs a texture feature segmentation method to divide the thin cloud region into blocks. This segmentation method can more accurately adapt to the spatial intensity gradient of thin cloud radiation interference, ensuring that the thin cloud interference intensity is relatively uniform in each sub-region, thereby achieving more targeted subsequent radiation interference correction and effectively improving the rationality and accuracy of the segmentation. Preferably, the texture feature segmentation method includes Gabor filtering and Local Binary Patterns (LBP).
[0021] See Figure 2 , Figure 2 This is a schematic diagram of a thin cloud region texture segmentation process provided by an embodiment of the present invention. For example, as shown... Figure 2 As shown, for the input image Thin cloud regions in the input image can be identified and corresponding masks can be generated using manual interpretation and delineation or existing mature cloud detection algorithms. The white areas represent areas covered by thin clouds, while the black areas represent areas not covered by thin clouds, i.e., clear skies.
[0022] Furthermore, masking in thin cloud regions Under the constraints, a set of multi-scale methods is adopted. ... , multi-directional ... The Gabor filter extracts the texture features of pixels within the thin cloud region and performs texture feature segmentation, with a segmentation scale of [scale value missing]. This allows for the segmentation of thin cloud regions into blocks, resulting in... Sub-regions .
[0023] It should be noted that the segmentation scale is These are parameters used to adjust the segmentation scale, such as filter scale, window size, or aggregation threshold, to adapt to the hierarchical differentiation requirements of different thin cloud thickness regions.
[0024] Step S102: For each sub-region, extract the cloud and fog radiation disturbance components in each visible light band, and establish the relative relationship between cloud and fog radiation disturbance components between bands based on the disturbance components. See Figure 3 , Figure 3 This is a schematic diagram illustrating the process for establishing the relative relationship of cloud and fog radiation perturbation components between bands, provided by an embodiment of the present invention. For example, as shown... Figure 3 As shown, the thin clouds are divided into block regions. Within, for each visible light band Search for the minimum pixel in each band, and record the minimum value of that sub-region in the band. The dark target on Its pixel value is denoted as .Will Defined as a dark target, and considered Characterizing subregions In the band The cloud and fog radiation disturbance component.
[0025] It should be noted that, due to the wide imaging range of remote sensing images, dark water bodies, shadows, and black rocks are widely present within a single image. The surface reflectance of such targets is almost zero under cloudless conditions. Therefore, theoretically, dark targets with zero reflectance should also exist in thin cloud areas. However, due to the radiative perturbation of thin clouds, the reflectance or radiance of these dark targets is not close to zero. Therefore, in this field, the smallest pixel value within a thin cloud area can be considered a dark target, and its pixel value is equal to the radiative perturbation component.
[0026] Furthermore, using dark target values in different visible light bands within the same sub-region... Using samples, a mapping relationship between the visible light band and dark targets is established to determine the relative relationships of cloud and fog radiation perturbation components between bands, i.e., the relative relationships of cloud and fog radiation perturbation components between bands. Specifically, for the visible light band... For dark targets, a linear mapping is preferred. For example, For any two visible light bands (such as...) and ), with dark target pairs of all sub-regions { , Using} as samples, a linear mapping relationship is obtained through fitting. (in, For mapping coefficients, (This is the bias term). Similarly, mapping relationships for all visible light band pairs can be established, yielding the corresponding relative mapping coefficients. and bias terms .
[0027] Step S103: Based on the relative relationship, process the visible light band of the input image to generate a differential feature image that cancels out thin cloud radiation interference. In an optional embodiment, step S103 includes steps S1031 to S1033: Step S1031: Select any two different visible light bands in the input image as the target band pair; Step S1032: Based on the relative relationship, perform a linear transformation on the first visible light band in the target band pair so that the inter-band cloud and fog radiation perturbation component of the first visible light band is aligned with the inter-band cloud and fog radiation perturbation component of the second visible light band in the target band pair in terms of magnitude. Step S1033: Perform differential operation on the second visible light band and the aligned first visible light band to generate a differential feature image that cancels out thin cloud radiation interference.
[0028] See Figure 4 , Figure 4 This is a schematic diagram illustrating the process of inter-band cloud and fog radiation perturbation alignment and false-color reference image generation provided in an embodiment of the present invention. For example, as shown... Figure 4 As shown, in the visible light band... The corresponding linear parameters For example, regarding the band. Perform a linear transformation to obtain the aligned bands. ,make and The cloud and fog radiation perturbation components contained within are of the same order of magnitude, thus achieving inter-band alignment of cloud and fog radiation perturbations. After completing band alignment, the entire image is then... and Perform difference operations to obtain the difference feature image. This is to offset or significantly reduce cloud and fog radiation disturbances caused by thin clouds.
[0029] Step S104: Select at least three sets of the differential feature images as three channels of the false color image for processing to generate a false color reference image.
[0030] For example, such as Figure 4 As shown, at least three sets of differential feature images (e.g.) are selected. These are combined as three channels, and linear stretching / normalization can be performed if necessary to generate a near-cloud-free false-color reference image. This is used for subsequent similar pixel matching and thermal infrared brightness temperature reconstruction based on false color features.
[0031] Step S2: Using the false-color reference image as a reference, within the clear sky area, find clear sky similar pixels for each pixel to be reconstructed in the thin cloud area according to the matching constraint conditions, and form a clear sky similar pixel library. In an optional embodiment, step S2 includes steps S201 to S203: Step S201: Based on the false color reference image, and with the pixel to be reconstructed in the thin cloud region as the center, retrieve a set of candidate clear sky pixels in the spatially adjacent clear sky region according to the matching constraint conditions. See Figure 5 , Figure 5 This is a schematic diagram illustrating the construction and similarity relationship transfer process of a clear sky similar pixel library provided in an embodiment of the present invention. For example, as shown... Figure 5 As shown, based on the original input image Thin cloud region mask Determine the set of thin cloud pixels With clear sky pixel set For any thin cloud pixel to be reconstructed In false-color reference image Extract the spectral consistency features of its false color from the clear sky pixel set. Similar pixels are retrieved from the database. To reflect the comparability constraint of thermal environments, the preferred method is to search for similar pixels within the database. Centered on, with scale parameter as Local search window Internal screening of candidate clear sky pixels, that is, only within Similarity matching is performed; then, similarity is calculated based on false-color spectral and numerical consistency characteristics, and the clear sky pixel with the highest similarity is selected as the pixel. The set of similar pixels is used to obtain the candidate set of clear sky pixels.
[0032] In one alternative embodiment, the constraints include false color numerical consistency constraints, visible light scattering constraints, and spatial proximity constraints. The false color numerical consistency constraint includes requiring that the difference in false color channel values between the pixel to be reconstructed and the candidate clear sky pixel does not exceed a preset false color channel threshold. For example, in a false-color reference image Above, it is required that thin clouds await reconstruction of pixels. With candidate clear sky pixels The false color channel values are consistent or close, meaning their false color characteristics differ. Not exceeding the threshold (The threshold can be determined empirically or statistically adaptively), and candidate clear sky pixels that do not meet the consistency constraint are removed.
[0033] The visible light scattering constraint includes sorting the visible light bands by center wavelength from shortest to longest, and within a preset tolerance range, requiring that the difference between the pixel to be reconstructed and the candidate clear sky pixel in each band decreases as the wavelength increases, in order to respond to the physical law that the intensity of thin clouds decreases with wavelength. For example, in the input image In the visible light band, the wavelength-dependent scattering effect of thin clouds is used to constrain the reconstruction of pixels from thin clouds. With candidate clear sky pixels The difference decreases as wavelength increases. Specifically, the visible light band is sorted from shortest to longest center wavelength as follows: For any candidate clear sky pixel ,Require and Differences between different bands ,satisfy (Allowed to be within a given tolerance) (Internal satisfaction required). Candidate clear-sky pixels that do not satisfy this scattering constraint are discarded.
[0034] The spatial proximity constraint includes prioritizing candidate clear-sky pixels that are spatially closer to the pixel to be reconstructed.
[0035] For example, considering the spatial correlation of thermal infrared brightness temperature, spatial distance is preferably selected to improve the comparability of thermal environments. Closer candidate pixels.
[0036] It should be noted that many factors influence the thermal environment, such as topography and soil moisture. However, for a single image in this embodiment of the invention, if too many factors are considered in detail, additional data would need to be introduced, which would increase the difficulty of implementation. Furthermore, since this step mainly constrains similarity responses, the thermal environment characterization of pixels in adjacent spaces should be more consistent.
[0037] Step S202: Calculate the comprehensive matching weight between the pixel to be reconstructed and each candidate clear sky pixel according to the matching constraints; Step S203: Based on the comprehensive matching weight, a predetermined number of candidate clear sky pixels are selected from the candidate clear sky pixel set as clear sky similar pixels corresponding to the pixel to be reconstructed, forming a clear sky similar pixel library.
[0038] For example, such as Figure 5 As shown, after the above constraint screening, the comprehensive similarity of the retained candidate pixels is calculated based on the combined false color difference, the degree of satisfaction of scattering constraints, and spatial distance: ; The overall similarity is normalized to obtain the normalized matching weight for each candidate clear sky pixel. : ; Then, select the one with the highest weight. A clear sky pixel, forming a thin cloud pixel. Similar pixels , .
[0039] Step S3: Based on the clear sky similar pixel library, predict the brightness temperature of each pixel to be reconstructed in the thermal infrared band to obtain the predicted brightness temperature value of each pixel to be reconstructed in the thermal infrared band. In an optional embodiment, step S3 includes steps S301 to S302: Step S301: Obtain the original observed brightness temperature of each clear sky similar pixel in the thermal infrared band in the clear sky similar pixel library; Step S302: Based on the original observed brightness temperature of each clear-sky similar pixel and the corresponding comprehensive matching weight, obtain the predicted brightness temperature value of the pixel to be reconstructed in the thermal infrared band.
[0040] See Figure 6 , Figure 6 This is a schematic diagram of a calculation process for reconstructing the thermal infrared brightness temperature of a thin cloud region according to an embodiment of the present invention. For example, as shown... Figure 6 The image shows a reconstructed pixel for any thin cloud. Read its similar pixel library Location of similar pixels in clear sky In the thermal infrared band Original brightness temperature The brightness temperature prediction values of the thin cloud image to be reconstructed are obtained by weighting and summing according to their weights. : .
[0041] Step S4: Determine the brightness temperature deviation field of the thin cloud region based on the predicted brightness temperature value and the original observed brightness temperature value of each pixel to be reconstructed; In an optional embodiment, step S4 includes steps S401 to S402: Step S401: Calculate the difference between the predicted brightness temperature value and the original observed brightness temperature value of each pixel to be reconstructed, and use it as the initial value of the brightness temperature deviation of the pixel to be reconstructed due to thin cloud interference. Step S402: Summarize the initial values of brightness temperature deviation of all pixels to be reconstructed within the thin cloud region, and determine the brightness temperature deviation field of the thin cloud region based on the spatial position of each pixel to be reconstructed in the input image.
[0042] For example, such as Figure 6 As shown, within the thin cloud region, the initial value of the brightness temperature deviation is calculated using the obtained predicted brightness temperature and the original brightness temperature, forming a deviation field. That is, for each reconstructed pixel calculate .
[0043] Step S5: Perform spatial smoothing on the brightness temperature deviation field, and calculate the reconstructed thermal infrared brightness temperature data of the thin cloud region based on the smoothed brightness temperature deviation field and the original observed brightness temperature value.
[0044] For example, such as Figure 6 As shown, based on the characteristic that the radiation perturbation of thin clouds changes slowly in space, the deviation field is... Perform low-pass filtering and smoothing, with the filter scale parameter denoted as... Or the window size is denoted as The smoothed deviation field is obtained. The low-pass smoothing is used to suppress local outliers introduced by similar pixel matching errors, while avoiding the loss of surface thermal details caused by directly smoothing the brightness temperature results; preferably, the specific implementation of low-pass filtering includes mean smoothing, Gaussian smoothing, etc.
[0045] Then, within the thin cloud region, the smoothed bias field is added back to the original brightness temperature to obtain the reconstructed brightness temperature. For pixels in clear sky areas, their brightness temperature remains the original brightness temperature. The thermal infrared brightness temperature reconstruction result of the entire image can be obtained by either keeping the image unchanged or directly stitching the output. .
[0046] In summary, the thermal infrared band reconstruction method for thin cloud regions provided by this invention relies solely on a single-temporal input image, eliminating the need for multi-source / multi-temporal auxiliary data or prior cloud and atmospheric parameters. This effectively overcomes the limitations of multi-source methods and the inadequacies in stability and generalization of physical model-based methods, improving applicability in complex scenarios (such as continuous cloud cover or missing data). By generating a near-cloud-free false-color reference image as a similar pixel matching benchmark, the interference of thin clouds on visible / near-infrared observation features can be eliminated, avoiding feature drift and matching errors caused by directly transferring clear-sky multispectral-thermal infrared relationships in traditional data-driven methods. The invention ensures the stability and accuracy of similar pixel matching. By employing brightness temperature deviation correction instead of directly replacing thin cloud region information, it fully preserves the effective surface information from the original thermal infrared observations under the "semi-transparent" characteristics of thin clouds, thus solving the problems of block artifacts, over-smoothing, and information loss caused by the substitution-based completion methods of traditional methods. Furthermore, this invention does not rely on a large number of labeled samples and complex training strategies, avoiding the shortcomings of deep learning methods such as weak generalization and high implementation barriers. Simultaneously, it establishes an effective collaborative channel between visible / near-infrared information and thermal infrared correction through false-color reference images, realizing the linkage of multi-band information. In summary, this invention can improve the robustness, generalization, and applicability of thermal infrared reconstruction in thin cloud regions, ensuring that the reconstructed thermal infrared brightness temperature data can accurately represent the real surface thermal conditions while maintaining the spatial continuity of the thermal field, effectively improving the quality of thermal infrared remote sensing data.
[0047] Based on the above method items, the present invention provides corresponding system items embodiments.
[0048] See Figure 7 , Figure 7 This is a structural block diagram of a thermal infrared band reconstruction system for thin cloud regions provided in an embodiment of the present invention. The thermal infrared band reconstruction system for thin cloud regions includes: Reference image generation module 21 is used to generate a false-color reference image based on the thin cloud region of the input image; The clear sky similar image filtering module 22 is used to find clear sky similar pixels for each pixel to be reconstructed in the thin cloud region based on the false color reference image and according to the matching constraint conditions in the clear sky region, thereby forming a clear sky similar pixel library. Brightness temperature prediction module 23 is used to predict the brightness temperature of each pixel to be reconstructed in the thermal infrared band based on the clear sky similar pixel library, and obtain the predicted brightness temperature value of each pixel to be reconstructed in the thermal infrared band. Brightness temperature deviation determination module 24 is used to determine the brightness temperature deviation field of the thin cloud region based on the predicted brightness temperature value and the original observed brightness temperature value of each pixel to be reconstructed. The reconstruction module 25 is used to perform spatial smoothing on the brightness temperature deviation field, and calculate the reconstructed thermal infrared brightness temperature data of the thin cloud region based on the smoothed brightness temperature deviation field and the original observed brightness temperature value.
[0049] In one optional embodiment, the reference image generation module 21 includes: Thin cloud region division unit, used to divide the thin cloud region into multiple sub-regions based on the texture features of pixels within the thin cloud region of the input image; The cloud and fog radiation disturbance mapping unit is used to extract the cloud and fog radiation disturbance components in each visible light band for each sub-region, and establish the relative relationship between the cloud and fog radiation disturbance components between bands based on the disturbance components. The differential feature image generation unit is used to process the visible light band of the input image according to the relative relationship to generate a differential feature image that cancels out thin cloud radiation interference. The reference image generation unit is used to select at least three sets of the differential feature images as three channels of the false color image for processing to generate a false color reference image.
[0050] In one optional embodiment, the differential feature image generation unit is configured to: Select any two different visible light bands in the input image as the target band pair; Based on the relative relationship, a linear transformation is performed on the first visible light band in the target band pair so that the thin cloud radiation interference intensity of the first visible light band is aligned with the thin cloud radiation interference intensity of the second visible light band in the target band pair in terms of order of magnitude. The second visible light band is differentially analyzed with the aligned first visible light band to generate a differential feature image that cancels out thin cloud radiation interference.
[0051] In one optional embodiment, the clear sky similarity image filtering module 22 is used for: Based on the false-color reference image, and centered on the pixel to be reconstructed in the thin cloud region, a candidate set of clear sky pixels is retrieved in the spatially adjacent clear sky region according to matching constraints. Calculate the comprehensive matching weight between the pixel to be reconstructed and each candidate clear sky pixel based on the matching constraints; Based on the comprehensive matching weight, a predetermined number of candidate clear sky pixels are selected from the candidate clear sky pixel set as clear sky similar pixels corresponding to the pixel to be reconstructed, forming a clear sky similar pixel library.
[0052] In one alternative embodiment, the brightness temperature prediction module 23 is configured to: Obtain the original observed brightness temperature of each clear-sky similar pixel in the thermal infrared band in the clear-sky similar pixel library; Based on the original observed brightness temperature of each clear-sky similar pixel and the corresponding comprehensive matching weight, the predicted brightness temperature value of the pixel to be reconstructed in the thermal infrared band is obtained.
[0053] In one optional embodiment, the brightness temperature deviation determination module 24 is used for: Calculate the difference between the predicted brightness temperature value and the original observed brightness temperature value for each pixel to be reconstructed, and use it as the initial value of the brightness temperature deviation of the pixel to be reconstructed due to thin cloud interference. The initial values of brightness temperature deviation of all pixels to be reconstructed within the thin cloud region are summarized, and the brightness temperature deviation field of the thin cloud region is determined according to the spatial position of each pixel to be reconstructed in the input image.
[0054] It should be noted that the thin cloud region thermal infrared band reconstruction system provided in this embodiment of the invention is used to execute all the process steps of the thin cloud region thermal infrared band reconstruction method in the above embodiment. The working principle and beneficial effects of the two are one-to-one, so they will not be described again.
[0055] This invention also provides a terminal device, such as... Figure 8The diagram shown is a structural block diagram of a preferred embodiment of a terminal device provided by the present invention. The terminal device includes a processor 31, a memory 32, and a computer program stored in the memory 32 and configured to be executed by the processor 31. When the processor 31 executes the computer program, it implements the thin cloud region thermal infrared band reconstruction method as described in any of the above embodiments.
[0056] In addition, embodiments of the present invention also provide a computer-readable storage medium, the computer-readable storage medium including a stored computer program, wherein, when the computer program is executed, it controls the device where the computer-readable storage medium is located to perform the thin cloud region thermal infrared band reconstruction method as described in any of the above embodiments.
[0057] When the processor 31 executes the computer program, it implements the steps in the above embodiments of the thin cloud region thermal infrared band reconstruction method, for example... Figure 1 All steps of the thin cloud region thermal infrared band reconstruction method shown. Alternatively, when the processor 31 executes the computer program, it implements the functions of each module in the above-described thin cloud region thermal infrared band reconstruction system embodiment, for example... Figure 7 The functions of each module in the thin cloud region thermal infrared band reconstruction system are shown.
[0058] Preferably, the computer program can be divided into one or more modules / units, which are stored in the memory 32 and executed by the processor 31 to complete the present invention. The one or more modules / units can be a series of computer program instruction segments capable of performing specific functions, which describe the execution process of the computer program in the terminal device.
[0059] The processor 31 can be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor, or the processor 31 can be any conventional processor. The processor 31 is the control center of the terminal device, connecting various parts of the terminal device through various interfaces and lines.
[0060] The memory 32 mainly includes a program storage area and a data storage area. The program storage area can store the operating system, applications required for at least one function, etc., and the data storage area can store related data, etc. In addition, the memory 32 can be a high-speed random access memory, or a non-volatile memory, such as a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, and a flash card, etc., or the memory 32 can also be other volatile solid-state storage devices.
[0061] It should be noted that the aforementioned terminal devices may include, but are not limited to, processors and memory, as will be understood by those skilled in the art. Figure 8 The structural block diagram shown is merely a structural example of the terminal device described above and does not constitute a limitation on the structure of the terminal device. The terminal device may include more or fewer components than shown, or combine certain components, or use different components.
[0062] The above description represents the preferred embodiments of the present invention. It should be noted that those skilled in the art can make various improvements and modifications without departing from the principles of the present invention, and these improvements and modifications are also considered to be within the scope of protection of the present invention.
Claims
1. A method for reconstructing thermal infrared bands in thin cloud regions, characterized in that, include: A false-color reference image is generated based on the thin cloud region of the input image; Based on the false-color reference image, within the clear sky region, according to the matching constraints, clear sky similar pixels are found for each pixel to be reconstructed in the thin cloud region, forming a clear sky similar pixel library. Based on the clear sky similar pixel library, the brightness temperature of each pixel to be reconstructed is predicted in the thermal infrared band to obtain the predicted brightness temperature value of each pixel to be reconstructed in the thermal infrared band. Based on the predicted brightness temperature value and the original observed brightness temperature value of each pixel to be reconstructed, the brightness temperature deviation field of the thin cloud region is determined; The brightness temperature deviation field is spatially smoothed, and the reconstructed thermal infrared brightness temperature data of the thin cloud region is calculated based on the smoothed brightness temperature deviation field and the original observed brightness temperature value.
2. The method for reconstructing the thermal infrared band in thin cloud regions as described in claim 1, characterized in that, The generation of a false-color reference image based on the thin cloud region of the input image includes: Based on the texture features of pixels within the thin cloud region of the input image, the thin cloud region is divided into multiple sub-regions; For each sub-region, cloud and fog radiation perturbation components in each visible light band are extracted, and based on the perturbation components, the relative relationship between cloud and fog radiation perturbation components in different bands is established. Based on the relative relationship, the visible light band of the input image is processed to generate a differential feature image that cancels out thin cloud radiation interference. At least three sets of the differential feature images are selected as the three channels of the false color image and processed to generate a false color reference image.
3. The method for reconstructing the thermal infrared band in thin cloud regions as described in claim 2, characterized in that, The step of processing the visible light band of the input image according to the relative relationship to generate a differential feature image that cancels out thin cloud radiation interference includes: Select any two different visible light bands in the input image as the target band pair; Based on the relative relationship, a linear transformation is performed on the first visible light band in the target band pair so that the thin cloud radiation interference intensity of the first visible light band is aligned with the thin cloud radiation interference intensity of the second visible light band in the target band pair in terms of order of magnitude. The second visible light band is differentially analyzed with the aligned first visible light band to generate a differential feature image that cancels out thin cloud radiation interference.
4. The method for reconstructing the thermal infrared band in thin cloud regions as described in claim 1, characterized in that, The step of using the false-color reference image as a reference, within a spatially adjacent clear-sky region, and according to matching constraints, searching for clear-sky similar pixels for the pixels to be reconstructed in the thin cloud region, forming a clear-sky similar pixel library, includes: Based on the false-color reference image, and centered on the pixel to be reconstructed in the thin cloud region, a candidate set of clear sky pixels is retrieved in the spatially adjacent clear sky region according to matching constraints. Calculate the comprehensive matching weight between the pixel to be reconstructed and each candidate clear sky pixel based on the matching constraints; Based on the comprehensive matching weight, a predetermined number of candidate clear sky pixels are selected from the candidate clear sky pixel set as clear sky similar pixels corresponding to the pixel to be reconstructed, forming a clear sky similar pixel library.
5. The method for reconstructing the thermal infrared band in thin cloud regions as described in claim 4, characterized in that, The matching constraints include false color numerical consistency constraints, visible light scattering constraints, and spatial proximity constraints: The false color numerical consistency constraint includes requiring that the difference in false color channel values between the pixel to be reconstructed and the candidate clear sky pixel does not exceed a preset false color channel threshold. The visible light scattering constraint includes sorting the visible light bands from shortest to longest center wavelength, and within a preset tolerance range, requiring that the difference between the pixel to be reconstructed and the candidate clear sky pixel in each band decreases as the wavelength increases. The spatial proximity constraint includes prioritizing candidate clear-sky pixels that are spatially closer to the pixel to be reconstructed.
6. The method for reconstructing the thermal infrared band in thin cloud regions as described in claim 4, characterized in that, The step of predicting the brightness temperature of each pixel to be reconstructed in the thermal infrared band based on the clear-sky similar pixel library, to obtain the predicted brightness temperature value of each pixel to be reconstructed in the thermal infrared band, includes: Obtain the original observed brightness temperature of each clear-sky similar pixel in the thermal infrared band in the clear-sky similar pixel library; Based on the original observed brightness temperature of each clear-sky similar pixel and the corresponding comprehensive matching weight, the predicted brightness temperature value of the pixel to be reconstructed in the thermal infrared band is obtained.
7. The method for reconstructing the thermal infrared band in thin cloud regions as described in claim 1, characterized in that, The step of determining the brightness temperature deviation field of the thin cloud region based on the predicted brightness temperature value and the original observed brightness temperature value of each pixel to be reconstructed includes: Calculate the difference between the predicted brightness temperature value and the original observed brightness temperature value for each pixel to be reconstructed, and use it as the initial value of the brightness temperature deviation of the pixel to be reconstructed due to thin cloud interference. The initial brightness temperature deviation values of all pixels to be reconstructed within the thin cloud region are summarized, and the brightness temperature deviation field of the thin cloud region is determined based on the spatial position of each pixel to be reconstructed in the input image.
8. A thermal infrared band reconstruction system for thin cloud regions, characterized in that, include: The reference image generation module is used to generate a false-color reference image based on the thin cloud region of the input image; The clear sky similar image filtering module is used to find clear sky similar pixels for each pixel to be reconstructed in the thin cloud region based on the false color reference image and matching constraints within the clear sky region, thereby forming a clear sky similar pixel library. The brightness temperature prediction module is used to predict the brightness temperature of each pixel to be reconstructed in the thermal infrared band based on the clear sky similar pixel library, so as to obtain the predicted brightness temperature value of each pixel to be reconstructed in the thermal infrared band. The brightness temperature deviation determination module is used to determine the brightness temperature deviation field of the thin cloud region based on the predicted brightness temperature value and the original observed brightness temperature value of each pixel to be reconstructed. The reconstruction module is used to perform spatial smoothing on the brightness temperature deviation field, and calculate the reconstructed thermal infrared brightness temperature data of the thin cloud region based on the smoothed brightness temperature deviation field and the original observed brightness temperature value.
9. A terminal device, characterized in that, The method includes a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor, wherein the processor, when executing the computer program, implements the thin cloud region thermal infrared band reconstruction method as described in any one of claims 1 to 7.
10. A computer-readable storage medium, characterized in that, The computer-readable storage medium includes a stored computer program, wherein, when the computer program is executed, it controls the device on which the computer-readable storage medium is located to perform the thermal infrared band reconstruction method for thin cloud regions as described in any one of claims 1 to 7.