Mask removal of monospectral remote sensing data

Abstract

Embodiments of the invention relate to a computer-implemented method (100) for generating an image of a geographic area. The method comprises receiving a base image (11), wherein the base image comprises monospectral remote sensing data. The method further comprises identifying a masked area (12) in the base image, defining an offset area (13) around the masked area (12) of the base image (11) and selecting a supplementary image (21) from a set of supplementary images (20) of the geographic area. The selected supplementary image (21) comprises complementary monospectral remote sensing data of the masked area (12) of the base image (11). Further steps include defining an initial patch (24) encompassing a geographic area in the supplementary image (21) corresponding to the masked area (12) and the offset area (13) of the base image (11), computing a scaled patch (25) by performing a scaling of the initial patch (24) and replacing the offset area (13) and the masked area (12) of the base image (11) with the scaled patch (25). Further aspects relate to a computing system, and a corresponding computer program product.

Description

[0001] Mask removal of monospectral remote sensing data

[0002] Technical Field

[0003] The present invention pertains to a computer- implemented method for generating an image of a geographic area, in particular an image compri sing monospectral remote sensing data .

[0004] Further aspects relate to a computing system and a corresponding computer program product .

[0005] Background Art

[0006] Remote sensing generally uses data provided by satellites or aircrafts to analyse the surface of the earth . One challenge of remote sensing is the analysis of geographic areas which are masked by clouds or other elements . While images in the visible spectrum are typically composed of three colour bands , such as RGB ( red, green, blue ) , monospectral remote sensing data has only a single channel . This channel can be thought of as a two- dimensional array of values , with each pixel representing the land surface temperature .

[0007] A particular challenge with cloud removal in monospectral remote sensing data, e . g . LST data, is that no additional colour information is available as with images in the visible spectrum

[0008] Disclosure of the Invention

[0009] Accordingly, one obj ect of an aspect of the invention is to provide an improved method for generating

[0010] MIM

[0011] 13.08.2024 P193219PC00 an image of a geographic area, in particular images comprising monospectral remote sensing data comprising masked areas .

[0012] According to an embodiment of an aspect of the invention, a computer-implemented method for generating an image of a geographic area is provided . The method comprises receiving or providing a base image , the base image comprising monospectral remote sensing data . A further step includes identi fying a masked area in the base image . Further steps include defining an of fset area around the masked area of the base image and selecting a supplementary image from a set of supplementary images of the geographic area . The selected supplementary image comprises complementary (missing) monospectral remote sensing data of the masked area of the base image . Further steps include defining an initial patch encompassing a geographic area in the supplementary image corresponding to the masked area and the of fset area of the base image , computing a scaled patch by performing a scaling of the initial patch and replacing the of fset area and the masked area of the base image with the scaled patch .

[0013] This is an ef ficient and elegant method for generating an image of a geographic area and allows to replace masked areas of a base image with missing information .

[0014] By providing an of fset area around the masked area of the base image and a corresponding of fset area in the initial patch a buf fer or in other words an interface or in yet other words an adaptation area is provided between the masked area of the base image and the remote sensing data of the supplementary image . The of fset area can be used for an ef ficient scal ing of the initial patch

[0015] MIM

[0016] 13.08.2024 P193219PC00 and a subsequent replacement of the masked area and its offset area with the scaled initial patch.

[0017] The scaling technique of the initial patch according to embodiments of the invention allows to use supplementary images which have been taken at different times of the day and to adapt and adjust the initial patch to the base image in an efficient manner.

[0018] Furthermore, the set of supplementary images may comprise images of different satellite missions and / or remote sensing missions. This increases the chance to find chronologically close remote sensing data for the masked areas. In addition, this provides data of the respective geographic area more frequently.

[0019] According to embodiments, the selected supplementary image is the supplementary image which is chronologically closest to the base image. This may provide the best starting point to find the best missing data of the masked area.

[0020] It should be noted that the processing of historical data may be performed differently than the processing of current data. As an example, if the base image has a date in the past of e.g. 2024.01.02, and there are supplementary images of 2024.01.06 as well as 2023.12.26 available, the chronologically closest supplementary image would be the supplementary image of 2024.01.06 and hence be in the future from the point of view of the base image. On the contrary, for the processing of current data the chronologically closest images would be always in the past.

[0021] The time of the day establishes a variable disturbance value in dependence on the sun emissions.

[0022] The scaling performed by methods according to embodiments of the invention is a very efficient

[0023] MIM

[0024] 13.08.2024 P193219PC00 computation method which can work only with monospectral (single-channel) data. In particular, it can avoid the use of additional spectra. Compared with complex machine learning methods as used e.g. for cloud removal of multi- spectral data, e.g. RGB images, the outcome is more predictable .

[0025] The embodied invention is in particular suited for the observance of small and urban / industrial areas.

[0026] According to an embodiment, the computing of the scaled patch is performed by computing a scaling function by performing an optimization method, in particular a mathematical optimization method, to match the monospectral remote sensing data of the scaled patch in the offset area to the monospectral remote sensing data of the base image in the offset area. Then the computed scaling function is applied on the initial patch.

[0027] According to such an embodiment at first a scaling function is computed. The optimization method aims at computing a scaling function which optimally matches the monospectral remote sensing data of the scaled patch in the offset area to the monospectral remote sensing data of the base image in the offset area. Then, at a second step, once the scaling function has been computed, it is applied to the whole initial patch. In other words, then each monospectral remote sensing data piece, in particular each pixel, is scaled by applying the scaling function. There may be various optimization methods used for this purpose, e.g. regression techniques and minimization techniques .

[0028] According to an embodiment, the scaling function is computed by minimizing a loss function between scaled monospectral remote sensing data of the initial patch in the offset area and the monospectral remote

[0029] MIM

[0030] 13.08.2024 P193219PC00 sensing data of the base image in the offset area. The scaled monospectral remote sensing data is computed by applying a respective candidate scaling function or in other words a test scaling function on the monospectral remote sensing data of the initial patch.

[0031] According to such an embodiment, a plurality of scaling functions are tested or probed as candidate functions to match the monospectral remote sensing data of the base image to the scaled monospectral remote sensing data of the scaled patch in the respective offset area. As underlying optimization technique a loss function is minimized. As a result, the scaling function which provides the least loss is chosen and then applied to the whole initial patch.

[0032] According to embodiments, the scaling may be in particular a linear scaling function. This has been proven to provide good results with high computational efficiency.

[0033] According to embodiments, the loss function may be the means squared error (MSE) , the root means squared error (RMSE) , the mean absolute error (MAE) , the Mean Percentage Error (MPE) or the Mean Absolute Percentage Error (MAPE) . These loss function are particularly suited. However, also other loss functions may be used, e.g. R2(Determination Coefficient) , Mean Bias Deviation (MBD) , Median Absolute Error (MedAE) , Symmetric Mean Absolute Percentage Error (sMAPE) , Logarithmic Error Metrics (e.g. MSLE, RMSLE) , Hinge Loss, Cohen's Kappa or Fl-Score.

[0034] According to embodiments, the base image and the set of supplementary images comprise land surface temperature data.

[0035] According to an embodiment, the initial patch may be divided into two or more initial sub-patches and

[0036] MIM

[0037] 13.08.2024 P193219PC00 the scaled patch may be computed by performing a separate scaling for the two or more initial sub-patches . According to such an embodiment the scaled patch may be computed by applying separate scaling functions on the two or more initial sub-patches .

[0038] This allows to adapt and optimi ze the scaling function for the respective sub-patches .

[0039] According to embodiments , the two or more initial sub-patches are patches of geographic areas having di f ferent physical conditions . The di f ferent physical conditions may encompass in particular thermal capacity, thermal conductivity, reflectance , emissivity, moisture and vegetation index, in particular the normali zed di f ferential vegetation index (NDVI ) .

[0040] According to such an embodiment , the di f ferent physical conditions such as the thermal capacity may be taken into account for the scaling .

[0041] The respective physical conditions of the subpatches may be derived from other sources or databases . As an example , the data of the vegetation index of the respective geographical area may me derived from a database that is run by the computing system that performs the method . The vegetation index may then be used to assess or compute the thermal capacity of the sub-patch .

[0042] According to a further embodiment it may be checked i f the initial patch of the selected supplementary image comprises also a masked area . I f this is true , steps corresponding to the steps as presented above may be repeated .

[0043] More particularly, an of fset area around the masked area of the initial patch may be defined and a further supplementary image from the set of supplementary images may be selected . The selected further supplementary

[0044] MIM

[0045] 13.08.2024 P193219PC00 image comprises complementary (missing) monospectral remote sensing data of the masked area of the initial patch .

[0046] Then a further initial patch encompassing a geographic area in the further supplementary image corresponding to the masked area and the of fset area of the initial patch may be defined . In addition, a further scaled patch may be computed by performing a scaling of the further initial patch . Finally, the of fset area and the masked area of the initial patch may be replaced with the further scaled patch .

[0047] These steps may again be repeated in case there are additional masked areas in the initial patch as appropriate .

[0048] According to an embodiment , the of fset area comprises a number n of pixels , wherein n is an integer greater than or equal to 1 . According to embodiments , the si ze of the of fset area is smaller than 200% of the masked area, in particular smaller than 100% of the masked area . According to further embodiments the si ze of the of fset area may be smaller than 50% of the masked area .

[0049] According to an embodiment of another aspect of the invention, a computing system is provided . The computing system may be configured to perform steps of the method aspect of the invention .

[0050] According to an embodiment of another aspect of the invention, a computer program product for operating a computing system is provided . The computer program product comprises a computer readable storage medium having program instructions embodied therewith, the program instructions executable by the computing system to cause

[0051] MIM

[0052] 13.08.2024 P193219PC00 the computing system to perform steps of the method aspect of the invention .

[0053] Features and advantages of one aspect of the invention may be applied to the other aspects of the invention as appropriate .

[0054] Other advantageous embodiments are listed in the dependent claims as well as in the description below .

[0055] Brief Description of the Drawings

[0056] The invention will be better understood and obj ects other than those set forth above will become apparent from the following detailed description thereof . Such description makes reference to the annexed drawings , wherein :

[0057] FIG . 1 shows an exemplary illustration of method steps of a computer-implemented method for generating an image of a geographic area according to an embodiment of the invention;

[0058] FIG . 2 shows a corresponding flow chart of a computer-implemented method for generating an image of a geographic area according to an embodiment of the invention;

[0059] FIG . 3 shows a flow chart of further method steps according to an embodiment of the invention;

[0060] FIG . 4 illustrates in an exemplary way corresponding geographical images to the flow chart of FIG . 3 ;

[0061] FIG . 5 illustrates method steps of a method according to embodiments of the invention, wherein the

[0062] MIM

[0063] 13.08.2024 P193219PC00 respective base images and supplementary images are shown as a 3-dimensional view;

[0064] FIG . 6 illustrates a two-dimensional / cross- sectional view of the scaling of an initial patch and its insertion into a masked area of a base image ; and

[0065] Fig . 7 shows a block diagram of a computing system for performing a computer-implemented method according to embodiments of the invention .

[0066] Modes for Carrying Out the Invention

[0067] At first , some general aspects and terms of embodiments of the invention will be introduced .

[0068] Geographic area : An area of the earth, in particular of the surface of the earth, in particular the land . The geographic area may be in particular represented as a 2-dimensional array .

[0069] Monospectral remote sensing data : Data that is derived from a single spectral band or in other words from one discrete spectral band . Monospectral remote sensing data comprises in particular one respective value in dependence on coordinates of the geographic area, wherein the geographic area may be represented as a 2- dimensional array . The remote sensing data may be provided in particular by satellites . The monospectral remote sensing data may be embodied in particular as land surface temperature . Monospectral remote sensing data may also be denoted as single-channel remote sensing data .

[0070] Base image : An image of a geographic area comprising in particular a 2-dimensional array of the

[0071] MIM

[0072] 13.08.2024 P193219PC00 land surface and the corresponding remote sensing data, in particular the land surface temperature.

[0073] Masked area: Generally any area of an image, in particular a base image, which lacks monospectral remote sensing information in a part of the corresponding geographic area. The masked area may be covered by a cloud, an aeroplane, steam, smoke, smog or other causes.

[0074] Offset area: An area around the masked area which serves as a buffer for matching supplementary information into the masked area. The offset area may comprise a number n of pixels, wherein n is an integer greater than or equal to 1. The size of the offset area may also be defined as a percentage of the masked area or by other means, e.g. 200% of the masked area, 100% of the masked area or 50% of the masked area.

[0075] Scaling: A scaling according to embodiments of the invention encompasses in particular a multiplication with a scaling factor (scaling coefficient) . In addition, the scaling may encompass an adjustment by a predefined adjustment coefficient or in other words an addition of a predefined adjustment coefficient. A scaling may be performed in particular by applying a scaling function on the values to be scaled, in particular a linear scaling function of the form f (z)= az + b, wherein a is the scaling factor, b an adjustment coef f icient / adjustment value and z the value to be scaled, e.g. the land surface temperature. According to embodiments, also higher order scaling functions, e.g. quadratic scaling functions may be used, e.g. of the form f (z)= az2+ bz +c.

[0076] FIG. 1 shows an exemplary illustration 100 of method steps of a computer-implemented method for generating an image of a geographic area according to an

[0077] MIM

[0078] 13.08.2024 P193219PC00 embodiment of the invention. FIG. 2 shows a corresponding flow chart of the computer-implemented method for generating an image of a geographic area according to an embodiment of the invention.

[0079] Referring at first to FIG. 1, it shows a base image 11 comprising monospectral remote sensing data. The monospectral remote sensing data may be in particular land surface temperature data. The values of the monospectral remote sensing data are illustrated in a simplified manner by dotted patterns. In other words, the base image 11 establishes a 2-dimensional array comprising the land surface temperatures for the corresponding geographic location. The base image may be e.g received by a computing system at a step 210, e.g. from a database. The base image 11 comprises a masked area 12 which does not comprise monospectral remote sensing data. The masked area 12 may be in particular an area that is covered by a cloud and hence for this area no remote sensing data is available. However, it should be noted that there may be also other reasons than a cloud for the missing data in the masked area 12 such as other obstructions as well as missing data due to technical or measurement failures. The masked area may be identified at a step 211, e.g. by image processing or image recognition techniques.

[0080] At a step 212, an offset area 13 is defined around the masked area 12 of the base image. The offset area 13 surrounds the masked area 12 and may comprise a predefined number of pixels or a predefined overlap with the masked area 12.

[0081] Then, at a step 213, a supplementary image 21 is selected from a set 20 of supplementary images. The set of supplementary images 20 may comprise images of the geographic area of different points in time, in

[0082] MIM

[0083] 13.08.2024 P193219PC00 particularly chronologically ordered images . Such supplementary images may be also received from a database . Furthermore , the set 20 of supplementary images may comprise images of di f ferent satellite missions and / or di f ferent remote sensing missions .

[0084] According to preferred embodiments , the selected supplementary image 21 may be in particular the supplementary image which is chronologically closest to the base image . The chronologically closest image has usually the best chance to provide complementary or in other words missing data of the masked area which matches the data of the base image in the best way . The selected supplementary image 21 comprises complementary or in other words missing monospectral remote sensing data of the masked area 12 of the base image 11 .

[0085] At a step 214 , an initial patch 24 is defined in the supplementary image 21 which encompasses a geographic area in the supplementary image 21 which corresponds to the masked area 12 and the of fset area 13 of the base image 11 . Accordingly, the initial patch 24 comprises also an of fset area 23 and a main patch area 22 . The main patch area 22 corresponds to the masked area 12 and the of fset area 23 to the of fset area 13 .

[0086] At a step 215 , a scaled patch 25 is computed by performing a scaling of the initial patch 24 . This adj usts or in other words adapts the scaled patch to the base image 11 .

[0087] At a step 216 , the of fset area 13 and the masked area 12 of the base image 11 are replaced with the scaled patch 25 .

[0088] The of fset area 13 of the base image 11 and the corresponding of fset area 23 of the initial patch 24 serve as a kind buf fer between the main patch area 22 and the

[0089] MIM

[0090] 13.08.2024 P193219PC00 area of the base image surrounding the of fset area 13 . The of fset area 13 of the base image and the of fset area 23 of the initial patch may be in particular used for performing mathematical matching or optimi zation techniques to adapt the initial patch 24 in an optimal way to the base image 11 .

[0091] According to embodiments the computing of the scaled patch 25 may be performed in particular by computing at first a scaling function and by applying then the computed scaling function on the initial patch .

[0092] According to embodiments , the scaling function may be computed by performing an optimi zation method, in particular a mathematical optimi zation method, to match the monospectral remote sensing data of the scaled patch 25 in the of fset area 23 to the monospectral remote sensing data of the base image 11 in the of fset area 13 .

[0093] According to embodiments , the scaling function may be in particular a linear scaling function .

[0094] According to embodiments , the scaling function may be computed by minimi zing a loss function between scaled monospectral remote sensing data of the initial patch 24 in the of fset area 23 and the monospectral remote sensing data of the base image 11 in the of fset area 13 . According to such an embodiment , the scaled monospectral remote sensing data is computed by applying a respective candidate-scaling function on the monospectral remote sensing data of the initial patch 24 . The loss function may be e . g . MSE , RMSE or MAE .

[0095] As an example , the scaling function may be a linear scaling function of the form f ( z ) = az + b

[0096] MIM

[0097] 13.08.2024 P193219PC00 According to such an embodiment , for a set of of candidate scaling functions having di f ferent coef ficients a and b, the chosen loss function is minimi zed to compute the optimal coef ficients a and b . After the optimal coef ficients a and b and therewith the optimal scaling function has been computed, the computed scaling function is applied to the monospectral remote sensing data of the complete initial patch 24 , thereby computing the scaled patch 25 at the step 215 .

[0098] Finally, at the step 216 , the of fset area 13 and the masked area 12 of the base image 11 are replaced with the scaled patch 25 .

[0099] FIG . 3 shows a flow chart of further method steps according to an embodiment of the invention . FIG . 4 illustrates in an exemplary way the corresponding images .

[0100] At a step 311 , which may be performed according to embodiments after step 216 of FIG . 2 , it is checked whether the initial patch 24 of the selected supplementary 21 image comprises also a masked area . As an example , also the initial patch 24 may comprise a part that is covered by a cloud and hence does not comprise all the information which is required to reconstruct the masked area 12 of the base image 11 . In the example as i llustrated in FIG . 4 the initial patch 24 comprises a masked area 32 .

[0101] Then, at a step 312 , an of fset area 33 i s defined around the masked area 32 of the initial patch 24 .

[0102] At a step 313 , a further supplementary image 41 is selected from the set of supplementary images 20 . The selected further supplementary image 41 comprises complementary (missing) monospectral remote sensing data of the masked area 32 of the initial patch 24 .

[0103] MIM

[0104] 13.08.2024 P193219PC00 Next , at a step 314 , a further initial patch 44 encompassing a geographic area in the further supplementary image 41 corresponding to the masked area 32 and the of fset area 33 of the initial patch 24 . Accordingly, the further initial patch 44 comprises also an of fset area 43 and a main patch area 42 . The main patch area 42 corresponds to the masked area 32 and the of fset area 43 to the of fset area 33 .

[0105] At a step 315 a further scaled patch 45 is computed by performing a scaling of the further initial patch 44 .

[0106] Next , at a step 316 , the of fset area 33 and the masked area 32 of the initial patch 24 is replaced with the further scaled patch 45 .

[0107] The steps 311-316 may be further repeated in case there is still a masked area left .

[0108] FIG . 5 illustrates method steps of a method according to embodiments of the invention, wherein the respective base images and supplementary images are shown as a 3-dimensional view . More particularly, an x-y-plane is shown for the dimensions of the surface of the geographic area and a vertical z-plane defining the corresponding values of the monospectral remote sensing data, e . g . the land surface temperature .

[0109] A base image 51 comprises monospectral remote sensing data . In this view the base image 51 establishes a 3-dimensional array comprising in the x-y plane surface coordinates and in the z-direction the land surface temperatures for the corresponding geographic location . The base image 51 comprises a masked area 52 which does not comprise monospectral remote sensing data due to e . g . cloud coverage .

[0110] MIM

[0111] 13.08.2024 P193219PC00 As explained before , an of fset area 53 is defined around the masked area 52 of the base image 51 . The of fset area 53 surrounds the masked area 52 .

[0112] In order to improve the base image 51 , it may be searched for the missing data in a set 60 o f supplementary images . The set 60 of supplementary images may comprise images of the geographic area of di f ferent points in time , in particularly chronologically ordered images . Furthermore , the set 60 of supplementary images may comprise images of di f ferent satellite missions and / or di f ferent remote sensing missions .

[0113] Now the chronologically nearest supplementary image 61 is selected which comprises the missing data or at least a part of the missing data . The selected supplementary image 61 comprises complementary or in other words missing monospectral remote sensing data of the masked area 52 of the base image 51 .

[0114] An initial patch 64 is defined which encompasses a geographic area in the supplementary image 61 which corresponds to the masked area 52 and the of fset area 53 of the base image 51 . Accordingly, the initial patch 64 comprises also an of fset area 63 and a main patch area 62 . The main patch area 62 corresponds to the masked area 52 and the of fset area 63 to the of fset area 53 .

[0115] According to this embodiment , the initial patch 64 is divided into three initial sub-patches 64a, 64b and 64c .

[0116] Then, a scaled patch 65 may be computed by performing a separate scaling for the three initial subpatches 64a, 64b and 64c, in particular by applying separate and possibly di f ferent scaling functions on the three initial sub-patches 64a, 64b and 64c, thereby computing three scaled sub-patches 65a, 65b and 65c .

[0117] MIM

[0118] 13.08.2024 P193219PC00 The three initial sub-patches 64a, 64b and 64c may have e.g. different thermal capacities. Such different thermal capacities may be e.g. derived from a database comprising data of vegetation indexes of the geographic area, in particular data comprising the different normalized differential vegetation indexes (NDVIs) . As an example, the NDVI may indicate that sub-patch 64a has an asphalt surface, sub-patch 64b a grass surface and subpatch 64c has a water surface, e.g. by being a lake.

[0119] These different thermal capacities may then be taken into account by applying different scaling factors to the sub-patches 64a, 64b and 64c. As an example, it may be assumed that the lake has a higher thermal capacity than the grass and that it will cool down less during evening / night than the grass. If the base image has been taken during the day, e.g. at 3 pm, and the closest image in time has been taken during the evening, e.g. at 9pm, the different thermal capacities may be taken into account for improving the scaled sub-patches.

[0120] A corresponding Pseudocode for the method as illustrated in FIG. 5 may read as follows. The method may be implemented in the computing system 70 as described with reference to FIG. 7 further below.

[0121] FUNCTION restoreMissingValues ( ) :

[0122] Identify base images that need mask removal, in particular cloud removal, based on their metadata and the presence of masks, in particular clouds.

[0123] MIM

[0124] 13.08.2024 P193219PC00 Retrieve a list of base image IDs that meet the criteria for mask / cloud removal. The base images may be stored e.g. in a database.

[0125] For each base image in the list:

[0126] Retrieve the cloud-masked array from the view collection using the base image ID.

[0127] If the array is mostly valid (not completely NaN) :

[0128] Obtain arrays from temporally close supplementary images to use as reference data.

[0129] Reconstruct the original array by:

[0130] Identifying segments in the cloud-masked array that are affected by clouds.

[0131] For each segment, extract patches from the reference arrays that overlap (offset) with the affected areas .

[0132] Scale and adjust these patches using optimization / scale techniques to match the affected areas in the cloud-masked array.

[0133] Replace the affected segments in the cloud- masked array with the optimized patches.

[0134] Update the base image in the collection with the reconstructed array if improvements were made.

[0135] FIG. 6 illustrates a two-dimensional (cross- sectional) view in an x-z-plane of the scaling of an initial patch and its insertion into a masked area of a base image.

[0136] The base image 11 comprises monospectral remote sensing data. The base image 11 comprises a masked area 12 which does not comprise monospectral remote sensing data. A supplementary image 21 is selected from a set of supplementary images, in particular the chronologically

[0137] MIM

[0138] 13.08.2024 P193219PC00 closest one to the base image 11 . An of fset area 13 is defined around the masked area 12 of the base image 11 and a corresponding initial patch 24 is defined in the supplementary image 21 . The initial patch 24 comprises two initial sub-patches 22a and 22b . Each of the two initial sub-patches 22a and 22b comprise an of fset area 23 . The area formed by the two sub-patches 22a and 22b corresponds to the masked area 12 . The of fset areas 23 correspond to the of fset areas 13 .

[0139] Then the two initial sub-patches 22a and 22b are separately scaled by applying separate scaling functions fscaiei and fscaie2. This increases the respective values of the monospectral remote sensing data of the initial sub-patches 22a and 22b in order to fit it into the masked area 12 of the base image . The scaled patch 25 comprises two scaled sub-patches 25a and 25b and is then inserted into the base image 11 .

[0140] The of fset area 13 of the base image 11 and the corresponding of fset area 23 of the initial patch 24 are used for performing mathematical matching or optimi zation techniques to adapt the initial sub-patches 22a and 22b in an optimal way to the base image 11 as described above .

[0141] According to embodiments the computing of the scaled patch 25 may be performed in particular by computing at first a scaling function or two or more separate scaling functions and by applying then the computed scaling function or the two or more separate scaling functions on the initial patch and the two or more initial sub-patches respectively .

[0142] According to embodiments , the scaling function ( s ) may be computed by performing an optimi zation method, in particular a mathematical optimi zation method, to match the monospectral remote sensing data of the scaled

[0143] MIM

[0144] 13.08.2024 P193219PC00 patch 25 in the of fset area 23 to the monospectral remote sensing data of the base image 11 in the of fset area 13 . According to the simpli fied example in FIG . 6 , the scaling may be performed by modi fying the values of the initial patch 24 according to the respective scaling function ( s ) .

[0145] Referring now to Fig . 7 , a more detailed block diagram of a computing system 70 according to embodiments of the invention is shown . The computing system 70 may perform computing functions and may hence be generally embodied as a computing system or computer . The computing system 70 may be e . g . a server computer . The computing system 70 may be configured to perform a computer- implemented method for generating an image of a geographic area as described above . The computing system 70 may be operational with numerous other general purpose or special purpose computing system environments or configurations .

[0146] The computing system 70 may be described in the general context of computer system-executable instructions , such as program modules , being executed by a computer system . Generally, program modules may include routines , programs , obj ects , components , logic, data structures , and so on that perform particular tasks or implement particular abstract data types . The computing system 70 is shown in the form of a general-purpose computing device . The components of computing system 70 may include , but are not limited to , one or more processors or processing units 715 , a system memory 720 , and a bus 716 that couples various system components including system memory 720 to processor 715 .

[0147] Bus 716 represents one or more of any of several types of bus structures , including a memory bus or memory controller, a peripheral bus , an accelerated

[0148] MIM

[0149] 13.08.2024 P193219PC00 graphics port, and a processor or local bus using any of a variety of bus architectures. By way of example, and not limitation, such architectures include Industry Standard Architecture (ISA) bus, Micro Channel Architecture (MCA) bus, Enhanced ISA (EISA) bus, Video Electronics Standards Association (VESA) local bus, and Peripheral Component Interconnect (PCI) bus.

[0150] Computing system 70 typically includes a variety of computer system readable media. Such media may be any available media that is accessible by computing system 70, and it includes both volatile and non-volatile media, removable and non-removable media.

[0151] System memory 720 can include computer system readable media in the form of volatile memory, such as random access memory (RAM) 721 and / or cache memory 722. Network node 710 may further include other removable / non- removable, volatile / non-volatile computer system storage media. By way of example only, storage system 723 can be provided for reading from and writing to a non-removable, non-volatile magnetic media (not shown and typically called a "hard drive") . Although not shown, a magnetic disk drive for reading from and writing to a removable, non-volatile magnetic disk (e.g., a "floppy disk") , and an optical disk drive for reading from or writing to a removable, nonvolatile optical disk such as a CD-ROM, DVD-ROM or other optical media can be provided. In such instances, each can be connected to bus 716 by one or more data media interfaces. As will be further depicted and described below, memory 720 may include at least one computer program product having a set (e.g., at least one) of program modules that are configured to carry out the functions of embodiments of the invention.

[0152] MIM

[0153] 13.08.2024 P193219PC00 Program / utility 730 , having a set ( at least one ) of program modules 731 , may be stored in memory 720 by way of example , and not limitation, as well as an operating system, one or more application programs , other program modules , and program data . Each of the operating system, one or more application programs , other program modules , and program data or some combination thereof , may include an implementation of a networking environment . Program modules 731 generally carry out the functions and / or methodologies of embodiments of the invention as described herein . Program modules 731 may carry out in particular one or more steps of a computer-implemented method as described above .

[0154] Computing system 70 may also communicate with one or more external devices 717 such as a keyboard or a pointing device as well as a display 718 . Such communication can occur via Input / Output ( I / O) interfaces 719 . Still yet , computing system 70 can communicate with one or more networks 40 such as a local area network ( LAN) , a general wide area network (WAN) , and / or a public network ( e . g . , the Internet ) via network adapter 741 . As depicted, network adapter 741 communicates with the other components of computing system 70 via bus 716 . It should be understood that although not shown, other hardware and / or software components could be used in conj unction with computing system 70 .

[0155] Aspects of the present invention may be embodied as a computing system, a method, and / or a computer program product . The computer program product may include a computer readable storage medium ( or media ) having computer readable program instructions thereon for causing a processor to carry out aspects o f the present invention .

[0156] MIM

[0157] 13.08.2024 P193219PC00 The computer readable storage medium can be a tangible device that can retain and store instructions for use by an instruction execution device . The computer readable storage medium may be , for example , but is not limited to , an electronic storage device , a magnetic storage device , an optical storage device , an electromagnetic storage device , a semiconductor storage device , or any suitable combination of the foregoing . A computer readable storage medium, as used herein, is not to be construed as being transitory signals per se , such as radio waves or other freely propagating electromagnetic waves , electromagnetic waves propagating through a waveguide or other transmission media ( e . g . , light pulses passing through a fiber-optic cable ) , or electrical signals transmitted through a wire .

[0158] Computer readable program instructions described herein can be downloaded to respective computing / processing devices from a computer readable storage medium or to an external computer or external storage device via a network, for example , the Internet , a local area network, a wide area network and / or a wireless network . The network may comprise copper transmission cables , optical transmission fibers , wireless transmission, routers , firewalls , switches , gateway computers and / or edge servers . A network adapter card or network interface in each computing / processing device receives computer readable program instructions from the network and forwards the computer readable program instructions for storage in a computer readable storage medium within the respective computing / processing device .

[0159] Computer readable program instructions for carrying out operations of the present invention may be assembler instructions , instruction-set-architecture ( ISA)

[0160] MIM

[0161] 13.08.2024 P193219PC00 instructions , machine instructions , machine dependent instructions , microcode , firmware instructions , statesetting data, or either source code or obj ect code written in any combination of one or more programming languages , including an obj ect oriented programming language such as Smalltalk, C++ or the like , and conventional procedural programming languages , such as the "C" programming language or similar programming languages .

[0162] Computer readable program instructions according to embodiments of the invention may be provided to a processor of a general purpose computer, special purpose computer, or other programmable data processing apparatus to produce a machine , such that the instructions , which execute via the processor of the computer or other programmable data processing apparatus , create means for implementing the functions / acts speci fied in the flowchart and / or block diagram block or blocks . These computer readable program instructions may also be stored in a computer readable storage medium that can direct a computer, a programmable data processing apparatus , and / or other devices to function in a particular manner, such that the computer readable storage medium having instructions stored therein comprises an article of manufacture including instructions which implement aspects of the function / act speci fied in the flowchart and / or block diagram block or blocks .

[0163] The computer readable program instructions may also be loaded onto a computer, other programmable data processing apparatus , or other device to cause a series of operational steps to be performed on the computer, other programmable apparatus or other device to produce a computer implemented process , such that the instructions which execute on the computer, other programmable

[0164] MIM

[0165] 13.08.2024 P193219PC00 apparatus , or other device implement the functions / acts speci fied in the flowchart and / or block diagram block or blocks .

[0166] The flowchart and block diagrams in the figures illustrate the architecture , functionality, and operation of possible implementations of systems , methods , and computer program products according to various embodiments of the present invention .

[0167] While there are shown and described presently preferred embodiments of the invention, it is to be distinctly understood that the invention is not limited thereto but may be otherwise variously embodied and practiced within the scope of the following claims .

[0168] MIM

[0169] 13.08.2024 P193219PC00

Claims

Claims1. A computer-implemented method for generating an image of a geographic area, the method comprising receiving a base image (11) , the base image comprising monospectral remote sensing data; identifying a masked area (12) in the base image ; defining an offset area (13) around the masked area of the base image; selecting a supplementary image (21) from a set (20) of supplementary images of the geographic area, the selected supplementary image comprising complementary monospectral remote sensing data of the masked area of the base image; defining an initial patch (24) encompassing a geographic area (22, 23) in the supplementary image (21) corresponding to the masked area (12) and the offset area (13) of the base image; computing a scaled patch (25) by performing a scaling of the initial patch (24) ; and replacing the offset area (13) and the masked area (12) of the base image (11) with the scaled patch (25) .

2. A method according to claim 1, wherein the computing of the scaled patch is performed by computing a scaling function by performing an optimization method to match the monospectral remote sensing data of the scaled patch in the offset area to theMIM13.08.2024 P193219PC00monospectral remote sensing data of the base image in the of fset area ; and applying the computed scaling function on the initial patch .3 . A method according to claim 2 , wherein the scaling function is computed by minimi zing a loss function between scaled monospectral remote sensing data of the initial patch in the of fset area and the monospectral remote sensing data values of the base image in the of fset area, wherein the scaled monospectral remote sensing data is computed by applying a respective candidate scaling function on the monospectral remote sensing data of the initial patch .4 . A method according to anyone of the preceding claims 2 or 3 , wherein the scaling function is a linear scaling function .5 . A method according to claim 3 or claim 4 , wherein the loss function is selected from the group consisting of : Mean Square Error (MSE ) , Root Mean Square Error (RMSE ) , Mean Absolut Error (MAE ) , Mean Percentage Error (MPE ) and Mean Absolute Percentage Error (MAPE ) .6 . A method according to any of the preceding claims , wherein the masked area is an area that is covered by a cloud .7 . A method according to any of the preceding claims , wherein the base image and the set of supplementary images comprise land surface temperature data .MIM13.08.2024 P193219PC008. A method according to any of the preceding claims, wherein the selected supplementary image is the supplementary image which is chronologically closest to the base image.

9. A method according to any of the preceding claims, wherein the set of supplementary images comprises images of different satellite missions and / or remote sensing missions.

10. A method according to any of the preceding claims, further comprising dividing the initial patch (64) into two or more initial sub-patches (64a, 64b, 64c) ; and computing the scaled patch (65) by performing a separate scaling for the two or more initial sub-patches.

11. A method according to claim 10, further comprising computing the scaled patch by applying separate scaling functions on the two or more initial sub-patches.

12. A method according to claim 10 or 11, wherein the two or more initial sub-patches (64a, 64b, 64c) are patches of geographic areas having different physical conditions .

13. A method according to claim 12, wherein the different physical conditions are selected from the group consisting of: thermal capacity, thermal conductivity, reflectance, emissivity, moisture and vegetation index, inMIM13.08.2024 P193219PC00particular the normalized differential vegetation index(NDVI) .

14. A method according to any of the preceding claims, further comprising if the initial patch (24) of the selected supplementary image comprises a masked area (32) defining an offset area (33) around the masked area (32) of the initial patch (24) ; selecting a further supplementary image (41) from the set (20) of supplementary images, the selected further supplementary image comprising complementary (missing) monospectral remote sensing data of the masked area of the initial patch; defining a further initial patch (44) encompassing a geographic area (42, 43) in the further supplementary image corresponding to the masked area (32) and the offset area (33) of the initial patch; compute a further scaled patch (45) by performing a scaling of the further initial patch; and replace the offset area (33) and the masked area (32) of the initial patch (24) with the further scaled patch (45) .

15. A method according to any of the preceding claims, wherein the offset area comprises a number n of pixels, wherein n is an integer greater than 1.

16. A computing system configured to perform a computer-implemented method for generating an image of a geographic area, the method comprising receiving a base image, the base image comprising monospectral remote sensing data;MIM13.08.2024 P193219PC00identi fying a masked area in the base image ; defining an of fset area around the masked area of the base image ; selecting a supplementary image from a set of supplementary images of the geographic area, the selected supplementary image comprising complementary monospectral remote sensing data of the masked area of the base image ; defining an initial patch encompassing a geographic area in the supplementary image corresponding to the masked area and the of fset area of the base image ; computing a scaled patch by performing a scaling of the initial patch; and replacing the of fset area and the masked area of the base image with the scaled patch .17 . A computer program product for operating a computing system, the computer program product comprising a computer readable storage medium having program instructions embodied therewith, the program instructions executable by the computing system to cause the computing system to perform a computer- implemented method for generating an image of a geographic area, the method comprising receiving a base image , the base image comprising monospectral remote sensing data ; identi fying a masked area in the base image ; defining an of fset area around the masked area of the base image ; selecting a supplementary image from a set of supplementary images of the geographic area, the selected supplementary image comprising complementary monospectral remote sensing data of the masked area of the base image ;MIM13.08.2024 P193219PC0031 defining an initial patch encompassing a geographic area in the supplementary image corresponding to the masked area and the of fset area of the base image ; computing a scaled patch by performing a scaling of the initial patch; and replacing the of fset area and the masked area of the base image with the scaled patch .MIM13.08.2024 P193219PC00

Images