Satellite observation data repair method and device based on generative collaborative adversarial network

Abstract

The present invention provides a satellite observation data repair method and device based on a generative collaborative adversarial network, which relates to the field of data repair technology, including: extracting cloud-covered sub-regions and cloud-free sub-regions from a target area, and based on the respective corresponding current satellite observation data subsets, performing spatiotemporal registration and radiation correction on the historical satellite observation data subsets corresponding to the cloud-covered sub-regions in the absence of cloud coverage, so as to obtain a corrected historical satellite observation data set corresponding to the target area; using the current satellite observation data set and the corrected historical satellite observation data set of the target area, a generative collaborative adversarial network is progressively trained in a collaborative manner. The present invention can alleviate the problems of poor repair effect and insufficient physical consistency of missing cloud coverage data in traditional methods.

Description

Technical Field

[0001] The present invention relates to the technical field of data repair, and in particular to a satellite observation data repair method and device based on a generative collaborative adversarial network. Background Art

[0002] With the rapid development of remote sensing technology, satellite data has played an increasingly important role in many fields such as meteorological monitoring, environmental monitoring, disaster warning, agriculture and ocean. However, satellite remote sensing data is often affected by factors such as cloud cover, sensor failure or data loss in practical applications, resulting in missing observation data. If these missing data cannot be repaired in time, it will affect the temporal and spatial consistency of the data, and further affect the scientific analysis and decision support based on these data.

[0003] At present, traditional satellite observation data repair methods mainly include repair techniques based on interpolation, physical models and statistical models. These methods can provide acceptable repair effects in some cases, but they also have the following limitations: on the one hand, repair based on interpolation often cannot fully consider the temporal and spatial dependencies of data, and the repaired data may change unreasonably; on the other hand, the repair of physical models depends on accurate physical parameters and model assumptions. If the model assumptions are inaccurate, the repair effect may be greatly reduced; although statistical models can capture the correlation of data to a certain extent, their effectiveness is often limited by the quality and diversity of data.

[0004] In order to solve the above problems, deep learning technology, especially generative adversarial networks (GAN), has been widely used in the field of image restoration in recent years because it can effectively generate missing data and maintain a certain structural and texture consistency. GAN uses adversarial training of the generator and the discriminator to make the generated restoration results closer and closer to the real data. For the cloud cover problem, existing research mainly focuses on GAN-based image restoration methods, but most of these methods only focus on static image restoration, lack consideration of the dynamic changes of spatiotemporal information, and do not make full use of cloud masks and physical constraints. Summary of the invention

[0005] In view of this, the purpose of the present invention is to provide a satellite observation data repair method and device based on a generative collaborative adversarial network to alleviate the problems of poor repair effect and insufficient physical consistency of missing cloud coverage data in traditional methods.

[0006] In a first aspect, the present invention provides a satellite observation data repair method based on a generative collaborative adversarial network, comprising:

[0007] Obtain current satellite observation datasets and historical satellite observation datasets for the target area;

[0008] A cloud-covered sub-region and a cloud-free sub-region are extracted from the target area, and based on the current satellite observation data subsets corresponding to the cloud-covered sub-region and the cloud-free sub-region, the historical satellite observation data subset corresponding to the cloud-covered sub-region under the cloud-free condition is subjected to spatiotemporal registration and radiation correction to obtain a corrected historical satellite observation data set corresponding to the target area;

[0009] The current satellite observation dataset and the corrected historical satellite observation dataset of the target area are used to perform progressive collaborative training on the generative collaborative adversarial network, which includes a multi-channel generator and a multi-channel discriminator. The trained multi-channel generator is used to repair the satellite observation data to be repaired in the target area.

[0010] In one embodiment, based on the current satellite observation data subsets corresponding to the cloud-covered sub-region and the cloud-free sub-region, respectively, a historical satellite observation data subset corresponding to the cloud-covered sub-region under the cloud-free condition is subjected to spatiotemporal registration and radiation correction to obtain a corrected historical satellite observation data set corresponding to the target area, including:

[0011] Extract the current satellite observation data subset corresponding to the cloud covered sub-region and the current satellite observation data subset corresponding to the cloud-free sub-region from the current satellite observation data set;

[0012] Perform similarity matching on the current satellite observation data subset corresponding to the cloud cover sub-area and the historical satellite observation data set corresponding to the target area, so as to screen out the historical satellite observation data subset corresponding to the cloud cover sub-area when there is no cloud cover; wherein the similarity matching includes one or more of geographical location matching, atmospheric condition matching and radiation characteristic matching;

[0013] After performing spatiotemporal registration on the current satellite observation data subset corresponding to the cloud-covered sub-area and the historical satellite observation data subset corresponding to the cloud-free sub-area, radiometric correction is performed on the historical satellite observation data subset corresponding to the cloud-covered sub-area in the cloud-free sub-area using the current satellite observation data subset corresponding to the cloud-free sub-area, so as to obtain the corrected historical satellite observation data subset corresponding to the cloud-covered sub-area;

[0014] The corrected historical satellite observation data subset corresponding to the cloud cover sub-area is used to replace and complete the historical satellite observation data subset corresponding to the target area, so as to obtain the corrected historical satellite observation data set corresponding to the target area.

[0015] In one embodiment, using the current satellite observation data subset corresponding to the cloud-free sub-region, radiometric correction is performed on the historical satellite observation data subset corresponding to the cloud-covered sub-region in the cloud-free condition, including:

[0016] Determine the radiance deviation value between the radiance value of the current satellite observation data subset corresponding to the cloud-free sub-region and the radiance value of the historical satellite observation data subset corresponding to the cloud-free sub-region under the condition of no cloud coverage;

[0017] The sum of the radiation brightness deviation value and the radiation brightness value of the historical satellite observation data subset corresponding to the cloud cover sub-area under the cloud-free condition is determined to achieve radiation correction for the historical satellite observation data subset corresponding to the cloud cover sub-area under the cloud-free condition.

[0018] In one embodiment, the current satellite observation dataset and the corrected historical satellite observation dataset of the target area are used to perform progressive collaborative training on the generative collaborative adversarial network, including:

[0019] Generate a repaired satellite observation dataset corresponding to the target area based on the current satellite observation dataset of the target area through a multi-channel generator;

[0020] Through the multi-channel discriminator, the repaired satellite observation dataset and the corrected historical satellite observation dataset corresponding to the target area generate a judgment result, and the judgment result is used to describe the difference between the repaired satellite observation dataset and the corrected historical satellite observation dataset;

[0021] Based on the repaired satellite observation dataset and the judgment results, the generation collaboration loss value and reward value corresponding to the multi-channel generator and the discrimination collaboration loss value corresponding to the multi-channel discriminator are determined respectively to perform progressive collaborative training on the multi-channel generator and the multi-channel discriminator.

[0022] In one implementation, the multi-channel generator includes a small cloud block generator, a large cloud block generator, and a special cloud block generator; generating a repaired satellite observation dataset corresponding to the target area based on the current satellite observation dataset of the target area through the multi-channel generator includes:

[0023] Through the small cloud block generator, local details are repaired based on the current satellite observation data set of the target area to obtain the small cloud block repair result;

[0024] And, through the large cloud block generator, global texture restoration is performed based on the current satellite observation data set of the target area to obtain the large cloud block restoration result;

[0025] And, through the special cloud block generator, complex area repair is performed based on the current satellite observation data set of the target area to obtain the special cloud block repair result;

[0026] The repair results of small cloud blocks, large cloud blocks and special cloud blocks are weighted fused to obtain the repaired satellite observation dataset corresponding to the target area.

[0027] In one implementation, the multi-channel discriminator includes a spatial discriminator and a global discriminator based on a collaborative mechanism; through the multi-channel discriminator, the repaired satellite observation data set and the corrected historical satellite observation data set corresponding to the target area generate a judgment result, including:

[0028] Extracting a local cropped image corresponding to the cloud cover sub-area from the repaired satellite observation data set, and using a spatial discriminator to determine the first expected value of the local cropped image belonging to the real image based on the corrected historical satellite observation data set;

[0029] Through a global discriminator, it is determined that the restored satellite observation dataset belongs to the second expected value of the real image based on the corrected historical satellite observation dataset;

[0030] The determination result is determined according to the first expected value and the second expected value.

[0031] In one embodiment, the progressive collaborative training includes a local feature learning stage, a global consistency optimization stage, and a multi-scale fusion stage; wherein the local feature learning stage is training based on a subset of current satellite observation data corresponding to a cloud coverage sub-region; the global consistency optimization stage is training based on a current satellite observation data set corresponding to a target region of the same scale; and the multi-scale fusion stage is training based on a current satellite observation data set corresponding to a target region of different scales.

[0032] The generated collaborative loss value is obtained by weighted summing up the reconstruction error sub-loss value, the global consistency sub-loss value, the adversarial sub-loss value, the physical consistency sub-loss value, and the multi-scale sub-loss value; the weights corresponding to the same sub-loss value are different in different stages, and the target-aware weight allocation mechanism is used in the local feature learning stage to dynamically adjust the weights of the reconstruction error sub-loss value and the adversarial sub-loss value;

[0033] The reward value is obtained by weighted summing the texture sub-loss value and the radiation sub-loss value;

[0034] The discriminant collaborative loss value is obtained by weighted summing the spatial sub-loss value and the global sub-loss value.

[0035] In one embodiment, a multi-channel generator and a multi-channel discriminator are progressively and collaboratively trained, including:

[0036] In the local feature learning stage, global consistency optimization stage and multi-scale fusion stage, the multi-channel generator and multi-channel discriminator are trained by AdamW optimizer, and the training process is optimized by gradient clipping strategy and periodic learning rate strategy.

[0037] In a second aspect, the present invention further provides a satellite observation data repair device based on a generative collaborative adversarial network, comprising:

[0038] A data acquisition module is used to acquire the current satellite observation data set and the historical satellite observation data set of the target area;

[0039] The data correction module is used to extract the cloud-covered sub-region and the cloud-free sub-region from the target area, and based on the current satellite observation data subsets corresponding to the cloud-covered sub-region and the cloud-free sub-region, perform spatiotemporal registration and radiation correction on the historical satellite observation data subset corresponding to the cloud-covered sub-region under the cloud-free condition, so as to obtain the corrected historical satellite observation data set corresponding to the target area;

[0040] The network training module is used to use the current satellite observation data set and the corrected historical satellite observation data set of the target area to perform progressive collaborative training on the generative collaborative adversarial network. The generative collaborative adversarial network includes a multi-channel generator and a multi-channel discriminator. The trained multi-channel generator is used to perform data repair on the satellite observation data to be repaired in the target area.

[0041] In a third aspect, the present invention further provides an electronic device, comprising a processor and a memory, wherein the memory stores computer executable instructions that can be executed by the processor, and the processor executes the computer executable instructions to implement any one of the methods provided in the first aspect.

[0042] The present invention provides a satellite observation data repair method and device based on a generative collaborative adversarial network. First, a current satellite observation data set and a historical satellite observation data set of a target area are obtained; then a cloud-covered sub-area and a cloud-free sub-area are extracted from the target area, and based on a current satellite observation data subset corresponding to the cloud-free sub-area, a historical satellite observation data subset corresponding to the cloud-covered sub-area under a cloud-free condition is subjected to spatiotemporal registration and radiation correction to obtain a corrected historical satellite observation data set corresponding to the target area; finally, the current satellite observation data set and the corrected historical satellite observation data set of the target area are used to perform progressive collaborative training on a generative collaborative adversarial network, wherein the generative collaborative adversarial network includes a multi-channel generator and a multi-channel discriminator, and the trained multi-channel generator is used to perform data repair on the satellite observation data to be repaired in the target area. The above method uses the current satellite observation data subset corresponding to the cloud-free sub-area to perform spatiotemporal alignment and radiation correction on the historical satellite observation data subset corresponding to the cloud-covered sub-area under cloud-free conditions, so as to overcome the problems such as poor spatiotemporal consistency faced by traditional data repair methods when processing satellite observation data; in addition, unlike the existing repair methods based on interpolation, physical models or single generative adversarial network models, the present invention innovatively introduces a multi-channel generator and multi-channel discriminator structure of a collaborative generative adversarial network, allowing multiple generators to repair the current satellite observation data set from different perspectives in parallel. At the same time, multiple discriminators further ensure the spatiotemporal consistency, detail retention and physical rationality of the repaired satellite observation data set by comparing the authenticity of different repaired satellite observation data sets.

[0043] Other features and advantages of the present invention will be described in the following description, and partly become apparent from the description, or understood by practicing the present invention. The purpose and other advantages of the present invention are realized and obtained by the structures particularly pointed out in the description, claims and drawings.

[0044] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, preferred embodiments are given below and described in detail with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] In order to more clearly illustrate the specific implementation methods of the present invention or the technical solutions in the prior art, the drawings required for use in the specific implementation methods or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some implementation methods of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0046] Figure 1 A schematic diagram of a flow chart of a satellite observation data repair method based on a generative collaborative adversarial network provided by an embodiment of the present invention;

[0047] Figure 2 A schematic diagram of a framework for generating a collaborative adversarial network provided by an embodiment of the present invention;

[0048] Figure 3 A schematic diagram of the structure of a satellite observation data repair device based on a generative collaborative adversarial network provided by an embodiment of the present invention;

[0049] Figure 4 A schematic diagram of the structure of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0050] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution of the present invention will be clearly and completely described in combination with the embodiments below. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0051] At present, most traditional data restoration methods only focus on static image restoration, lack consideration of dynamic changes in spatiotemporal information, and do not fully utilize cloud masks and physical constraints. Therefore, there is an urgent need for a satellite data restoration method that can combine spatiotemporal information and physical constraints to overcome the limitations of existing methods. The present invention provides a satellite observation data restoration method and device based on a generative collaborative adversarial network, which is precisely to solve the problems of poor restoration effect and insufficient physical consistency of missing cloud coverage data in traditional methods, and provides an effective solution. By generating a multi-generator and multi-discriminator structure of a collaborative adversarial network, it can work together in multiple dimensions to capture the spatiotemporal dependency of data and ensure the spatiotemporal consistency and physical rationality of the restoration results.

[0052] To facilitate understanding of this embodiment, a satellite observation data repair method based on a generative collaborative adversarial network disclosed in an embodiment of the present invention is first introduced in detail. This method is suitable for satellite observation data repair, especially in the fields of meteorological satellites and environmental monitoring satellites. It solves the problems of poor physical consistency and insufficient spatiotemporal consistency in traditional repair methods and has broad application prospects. Figure 1 The flowchart of a satellite observation data repair method based on a generative collaborative adversarial network is shown, and the method mainly includes the following steps S102 to S106:

[0053] Step S102: Acquire a current satellite observation data set and a historical satellite observation data set of a target area.

[0054] Step S104, extracting a cloud-covered sub-region and a cloud-free sub-region from the target area, and based on the current satellite observation data subsets corresponding to the cloud-covered sub-region and the cloud-free sub-region respectively, performing spatiotemporal registration and radiation correction on the historical satellite observation data subset corresponding to the cloud-covered sub-region under the cloud-free condition, so as to obtain a corrected historical satellite observation data set corresponding to the target area.

[0055] In one example, first, the cloud detection result corresponding to the current satellite observation dataset is obtained, and the cloud detection result is fully used to identify the cloud covered sub-region and the cloud-free sub-region in the target area, and cloud mask data is generated; then, the cloud mask data is used to extract the current satellite observation data subset corresponding to the cloud covered sub-region and the current satellite observation data subset corresponding to the cloud-free sub-region from the current satellite observation dataset; then, similarity matching is performed between the current satellite observation data subset corresponding to the cloud covered sub-region and the historical satellite observation dataset corresponding to the target area, so as to extract the historical satellite observation data subset corresponding to the cloud covered sub-region under the condition of no cloud coverage; finally, the current satellite observation data subset corresponding to the cloud covered sub-region is used to perform spatiotemporal registration on the historical satellite observation data subset corresponding to the cloud covered sub-region under the condition of no cloud coverage, and the current satellite observation data subset corresponding to the cloud-free sub-region is used to perform radiometric correction on the historical satellite observation data subset corresponding to the cloud covered sub-region under the condition of no cloud coverage, so as to obtain the corrected historical satellite observation dataset corresponding to the target area, so as to ensure the spatiotemporal consistency between the current satellite observation data dataset and the historical satellite observation data dataset.

[0056] Step S106, using the current satellite observation data set and the corrected historical satellite observation data set of the target area, the generative collaborative adversarial network is progressively collaboratively trained, the generative collaborative adversarial network includes a multi-channel generator and a multi-channel discriminator, and the trained multi-channel generator is used to perform data repair on the satellite observation data to be repaired in the target area.

[0057] Among them, the multi-channel generators included in the generative collaborative adversarial network (Collaborative GAN) are specifically small cloud block generators, large cloud block generators and special cloud block generators, the multi-channel discriminators are specifically spatial discriminators and global discriminators, and the progressive collaborative training includes local feature learning stage, global consistency optimization stage and multi-scale fusion stage. Due to the different training objectives in different stages, the weight distribution of the generation collaborative loss value of the multi-channel generator in different stages is different.

[0058] In one example, the current satellite observation dataset of the target area is used as the input of the multi-channel generator, and the multi-channel generator will output the repaired satellite observation dataset. The repaired satellite observation dataset and the corrected historical satellite observation dataset are used as the input of the multi-channel discriminator, and the multi-channel discriminator will output the judgment result, which is used to describe the difference between the repaired satellite observation dataset and the corrected historical satellite observation dataset. According to the outputs of the multi-channel generator and the multi-channel discriminator, as well as the sub-loss weights corresponding to the current stage, the generation collaboration loss value and reward value corresponding to the multi-channel generator and the discriminative collaboration loss value corresponding to the multi-channel discriminator are determined respectively, and the entire generation collaborative adversarial network is trained in combination with the update mechanism of the AdamW (Adaptive Moment Estimation with Weight Decay) optimizer, the gradient clipping strategy and the cyclical learning rate (CLR) strategy and other optimization strategies. When the training task of the current stage is completed, the next stage of training tasks can be continued for the generation collaborative adversarial network to obtain the trained generation collaborative adversarial network.

[0059] Optionally, the generative collaborative adversarial network is trained in the order of a local feature learning stage, a global consistency optimization stage, and a multi-scale fusion stage.

[0060] The satellite observation data repair method based on the generative collaborative adversarial network provided in the embodiment of the present invention relates to the field of data repair, and in particular to a method of using a collaborative generative adversarial network to repair data caused by missing observation data due to cloud cover. This technology combines the generative collaborative adversarial network with the spatiotemporal data repair mechanism, and can efficiently repair the data missing problem caused by cloud cover, and improve the quality of the repaired data while ensuring the spatial continuity, temporal consistency and physical rationality of the data. As a deep learning technology, the generative collaborative adversarial network can handle complex spatiotemporal dependencies through the collaborative work of multi-channel generators and multi-channel discriminators, ensuring that the repair results show higher accuracy and stability in multiple dimensions.

[0061] For ease of understanding, the embodiment of the present invention takes the observation data collected by the FY-4B satellite as an example. The FY-4B satellite is an important part of the second-generation geostationary meteorological satellite. Its core observation instrument, the multi-channel scanning imaging radiometer (AGRI), has high resolution and multi-spectral observation capabilities, providing rich data support for meteorological observation and environmental monitoring. AGRI covers 14 spectral channels, with wavelengths ranging from visible light to infrared, including 0.47μm (blue light), 0.65μm (red light), 3.9μm (mid-infrared), 10.8μm (infrared window) and 12.0μm (long-wave infrared). Among them, the visible light band is mainly used for daytime surface and cloud reflectivity monitoring, while the infrared and long-wave infrared channels can work all day long, and are particularly suitable for monitoring clouds, surface temperature and atmospheric radiation characteristics.

[0062] The high spatial resolution of the FY-4B satellite's AGRI is its significant advantage. In the visible light band, the resolution can reach 1 km, and in the infrared channel it is 4 km. This fine spatial resolution can capture subtle changes in the surface and clouds, providing high-quality data support for the accurate identification of cloud boundaries and radiation repair. AGRI's rich channel information, high resolution, and high temporal resolution provide basic data support for this invention. Its multispectral observation capabilities and operational cloud detection products (CLM) make the repair of cloud occlusion data more scientific and practical, effectively improving the accuracy and applicability of the model.

[0063] However, cloud cover is a common and influential problem in satellite observations, especially in the data assimilation process of numerical weather forecasting, which has a significant adverse effect on data quality. The interference of clouds on satellite radiation data is mainly reflected in the following aspects: First, clouds will significantly change the radiation transmission characteristics of the atmosphere, so that the surface radiation signal cannot directly penetrate the clouds and be observed by satellites, which leads to large errors in the observation data. Secondly, due to the differences in the thickness and cloud phase (such as ice clouds or water clouds) of different clouds, their absorption and scattering effects on radiation are also different, which introduces higher uncertainty. In addition, the dynamic change characteristics of clouds will cause the observation data at the same location at different times to be inconsistent in time and space, which makes it difficult for satellite observation data to accurately reflect the real surface and atmospheric state in the assimilation model. Therefore, the cloud cover problem often leads to a decline in data quality in numerical weather forecast assimilation, and even some channel data are directly discarded, thereby weakening the potential value of satellite data.

[0064] In response to the cloud cover problem, the embodiment of the present invention repairs and enhances the FY-4B satellite's multi-channel scanning imaging radiometer (AGRI) data to maximize its data availability and accuracy under cloud cover conditions. This method is not only applicable to FY-4B satellite observation data, but can also be applied to the completion and repair of other multispectral remote sensing data.

[0065] On this basis, an embodiment of the present invention provides a specific implementation method of a satellite observation data repair method based on a generative collaborative adversarial network.

[0066] The embodiment of the present invention provides a specific implementation of the aforementioned step S104. The embodiment of the present invention makes full use of the cloud detection results generated by the CLM (Cloud Mask) product provided by the FY-4B satellite. The CLM product can accurately identify cloud-covered sub-regions and cloud-free sub-regions, and generate cloud mask data. By introducing cloud mask data, the embodiment of the present invention enables the data processing process to effectively separate cloud-covered sub-regions from cloud-free sub-regions, thereby avoiding the problem of false repair of cloud-free data. The introduction of cloud mask data also provides clear boundary information for the repair process of the generative collaborative adversarial network, so that the generative collaborative adversarial network can focus on learning the cloud-occluded area. Different from the traditional method of directly discarding the satellite observation data corresponding to the cloud-covered sub-region, the embodiment of the present invention can retain the complete data spatiotemporal distribution information, and significantly improve the temporal and spatial continuity of the satellite observation data by repairing the satellite observation data corresponding to the cloud-covered sub-region. This improvement greatly improves the spatial and temporal continuity of the FY4B-AGRI satellite observation data, and provides a more complete observation data input for the numerical assimilation model.

[0067] In terms of the satellite training data set generation strategy, the embodiment of the present invention selects the multi-channel data of the FY-4B satellite as the core input source, making full use of its advantages in high resolution and multi-spectral observation capabilities. The multi-channel data of the FY-4B satellite, including infrared channels such as 10.8μm, 3.9μm and 12.0μm, can better reflect the different radiation characteristics of clouds and the surface. This multi-channel data selection provides sufficient feature input for the generation of collaborative adversarial networks, which helps to improve the recognition ability of the generation of collaborative adversarial networks under different cloud conditions. In order to overcome the difficulty of directly obtaining true value data, the embodiment of the present invention innovatively introduces the "multi-temporal and spatial registration" method, which constructs the true value by using the historical satellite observation data set under cloud-free coverage in the past month (the reason for selecting the time window as one month is that the atmospheric conditions and surface characteristics in this period of time change little, which can provide sufficient training samples). Specifically, for each cloud cover sub-area, cloud-free data in similar geographical locations are found in the historical satellite observation data set (that is, the subset of historical satellite observation data corresponding to the cloud cover sub-area under cloud-free coverage), and spatial registration and radiation correction are performed to ensure spatiotemporal consistency.

[0068] In the specific implementation, please refer to the following steps A1 to A5:

[0069] Step A1: extracting a cloud-covered sub-region and a cloud-free sub-region from the target region.

[0070] First, the cloud detection results (CLM) in the FY-4B L2 product are used to extract the cloud mask data of the cloud cover sub-area. Defined as:

[0071] ;

[0072] Then, the features of the cloud-covered sub-area are extracted: the extracted features include the boundary, geographic center point and geographic range (latitude and longitude range) of the cloud-covered sub-area. The geographic center point of the cloud-covered sub-area is calculated by the following formula :

[0073] ;

[0074] in: is the longitude and latitude of the geographic center of the cloud-covered sub-region, that is, the weighted average of the longitude and latitude of all pixels in the cloud-covered sub-region, representing the geographic center of the cloud-covered sub-region; R is the pixel set of the cloud-covered sub-region, representing all pixels in the image belonging to the cloud-covered sub-region, and each pixel has a corresponding longitude and latitude. and latitude ; is the two-dimensional coordinate of each pixel in the image, where is the pixel index in the horizontal direction, is the pixel index in the vertical direction; is the longitude and latitude of each pixel in the image, that is, each pixel corresponds to the longitude and latitude coordinates of a certain geographical location; is the total number of pixels in the cloud-covered sub-area, that is, the number of pixels contained in the set R, which is used to normalize the longitude and latitude of all pixels and calculate the weighted average.

[0075] Therefore, this formula calculates the geographic center (latitude and longitude) of the cloud-covered sub-area by averaging the longitude and latitude of all pixels in the cloud-covered sub-area.

[0076] Step A2: extracting, from the current satellite observation data set, a subset of current satellite observation data corresponding to the cloud-covered sub-region and a subset of current satellite observation data corresponding to the cloud-free sub-region.

[0077] Step A3, performing similarity matching on the current satellite observation data subset corresponding to the cloud coverage sub-area and the historical satellite observation data set corresponding to the target area, so as to screen out the historical satellite observation data subset corresponding to the cloud coverage sub-area when there is no cloud coverage; wherein the similarity matching includes one or more of geographic location matching, atmospheric condition matching and radiation characteristic matching.

[0078] In an embodiment of the present invention, a specific indicator is used to evaluate the similarity between the target area and the cloud cover sub-area in the historical satellite observation data set (past month). The similarity matching process includes:

[0079] (1) Geographic location matching: By aligning the geographic coordinates, the spatial consistency of the historical satellite observation dataset and the cloud cover sub-area is ensured. In the embodiment of the present invention, the spatial consistency of the historical satellite observation dataset and the target area is first ensured. By aligning the geographic coordinates, it can be ensured that the target area in the historical satellite observation dataset corresponds to the area in the cloud cover sub-data. If the spatial location of the cloud cover sub-data is inconsistent, the subsequent meteorological and radiation characteristic matching will lose its meaning.

[0080] (2) Atmospheric condition matching: Using reanalysis data (such as ERA5), compare the meteorological parameters (such as temperature, humidity, atmospheric pressure) at the target time point and the historical time point, and select the time point with similar atmospheric conditions. In an embodiment of the present invention, after ensuring the consistency of spatial positions, the meteorological conditions are matched. The matching of meteorological conditions helps to ensure that the meteorological state of the cloud-covered sub-area is similar to the state at the corresponding time point in the historical satellite observation data set. Atmospheric condition matching can eliminate differences caused by different weather conditions.

[0081] (3) Radiant characteristic matching: Calculate the average radiant brightness in a cloud-free channel (such as the 0.65 μm or 10.8 μm channel) , select the time point with smaller difference:

[0082] ;

[0083] in, is the average radiation brightness of the cloud-covered sub-area at the current moment; is the average radiation brightness of the cloud-covered sub-area at historical moments; ΔL is the difference between the average radiation brightness of the cloud-covered sub-area at the current moment and the average radiation brightness of the cloud-covered sub-area at historical moments. If ΔL is less than the preset threshold, it is considered that the radiation characteristics of the current moment match the historical satellite observation dataset successfully, and then it is determined that the cloud coverage state at the historical moment is similar to the cloud coverage state at the current moment.

[0084] Step A4, after performing spatiotemporal alignment on the current satellite observation data subset corresponding to the cloud-covered sub-area and the historical satellite observation data subset corresponding to the cloud-free sub-area, perform radiation correction on the historical satellite observation data subset corresponding to the cloud-covered sub-area in the cloud-free sub-area using the current satellite observation data subset corresponding to the cloud-free sub-area, and obtain the corrected historical satellite observation data subset corresponding to the cloud-covered sub-area.

[0085] (1) Spatiotemporal registration: In order to ensure the spatial consistency between the historical satellite observation data subset obtained in step A3 and the current cloud cover sub-area, strict spatial registration is required. This includes two steps: geographic coordinate alignment and registration operation:

[0086] (1.1) Geographic coordinate alignment: In actual use, the resolution and grid structure of the historical satellite observation data subset may be different from the current satellite observation data subset, or the observation angle of the historical satellite observation data subset may deviate from that of the current satellite observation data subset. Therefore, strict spatial alignment is still required for further analysis, especially when comparing data from different sensors. The purpose of spatial alignment is to ensure that the coordinates and perspectives of the historical satellite observation data subset are consistent with those of the current satellite observation data subset by remapping the longitude and latitude grid of the historical satellite observation data subset, so that subsequent analysis results are more accurate. For example, sensors on different satellites may have different perspectives, and the spatial position of the data needs to be accurately aligned through techniques such as projection transformation and interpolation.

[0087] The process of geographic coordinate alignment is as follows: remap the latitude and longitude grid of the historical satellite observation data subset to the geographic coordinates of the cloud cover sub-area. Use the sensor geometry model of FY-4B to calculate the observation angle of each pixel and perform spatial reprojection:

[0088] ;

[0089] in, is the target latitude and longitude; g is the geometric function of the sensor; h is the ground height; The observation angle of the satellite (i.e., the angle from the satellite to the ground target) affects the path of the light received by the sensor, so correction is required; are the indices or coordinates of pixels in a subset of historical satellite observation data, which in practice may represent pixel locations (rows, columns) of an image; It refers to the latitude and longitude coordinates of the cloud-covered sub-area after reprojection, that is, the geographic coordinates of the cloud-covered sub-area after reprojection; It is the longitude and latitude corresponding to the target area calculated based on the sensor's geometric model.

[0090] Therefore, the understanding of this expression is: for each pixel point (x, y) in the historical satellite observation data subset, the longitude and latitude of the cloud cover sub-area corresponding to the pixel point are calculated through the sensor's geometric function g, combined with the observation angle (θ) and the surface height (h), so as to align the pixels of the historical satellite observation data subset with the geographic coordinates of the cloud cover sub-area.

[0091] (1.2) Registration operation: Perform bilinear interpolation on the reprojected pixel data to ensure that each pixel value is aligned with the cloud cover sub-area grid:

[0092] ;

[0093] in, is the pixel value after resampling, is the original pixel value, is the interpolation weight, It is calculated based on the distance between the target point and the neighboring points. Specifically, the closer the distance between the target point and the neighboring points, the lower the weight. During the interpolation process, each point in the cloud cover sub-area is calculated by weighting the pixel values ​​of the surrounding neighboring points. The closer neighboring points provide more information, while the farther neighboring points provide less information.

[0094] (2) Radiation correction: The purpose is to ensure that the historical data is consistent with the current observations in terms of radiation characteristics. The following are the specific steps:

[0095] (2.1) Determine the radiance deviation between the radiance value of the current satellite observation data subset corresponding to the cloud-free sub-region and the radiance value of the historical satellite observation data subset corresponding to the cloud-free sub-region under cloud-free cover.

[0096] In one example, the radiance value of the current satellite observation data subset corresponding to the cloud-free sub-area and the radiance value of the historical satellite observation data subset corresponding to the cloud-free sub-area under cloud-free coverage can be directly read. In another example, the calibration coefficient (gain coefficient G, offset O) of the historical satellite observation data subset can be used to convert the digital count value (DN) into the radiance value (L): ; where L is the radiance value (unit: ), is the raw digital count value. The calibration parameters are obtained from the FY-4B satellite calibration file. Using the same formula, the current satellite observation data subset is also converted into radiance value (𝐿current).

[0097] In an embodiment of the present invention, a current satellite observation data subset corresponding to a cloud-free sub-region at the current moment is selected and compared with a historical satellite observation data subset corresponding to the cloud-free sub-region under cloud-free coverage. Among them, the process of determining the historical satellite observation data subset corresponding to the cloud-free sub-region under cloud-free coverage can be referred to the aforementioned step A3, and the embodiment of the present invention will not be repeated here. Only under cloud-free conditions will the radiation characteristics be less affected by atmospheric conditions and sensors, and therefore can be used for radiation correction. The radiation brightness value of the current satellite observation data subset corresponding to the cloud-free sub-region is compared with the radiation brightness value of the historical satellite observation data subset corresponding to the cloud-free sub-region under cloud-free coverage to calculate the radiation brightness deviation. :

[0098] ;

[0099] in, is the radiance value of the current satellite observation data subset corresponding to the cloud-free sub-region, is the radiance value of the historical satellite observation data subset corresponding to the cloud-free sub-area under cloud-free cover. The radiance deviation reflects the difference between the current satellite observation dataset and the historical satellite observation dataset. Usually, the radiance deviation is related to factors such as sensor differences and changes in atmospheric conditions.

[0100] (2.2) Determine the sum of the radiation brightness deviation value and the radiation brightness value of the historical satellite observation data subset corresponding to the cloud cover sub-area under the condition of no cloud cover, so as to realize the radiation correction of the historical satellite observation data subset corresponding to the cloud cover sub-area under the condition of no cloud cover.

[0101] In one example, radiometric correction is performed using radiometric brightness bias to ensure that the historical satellite observation dataset is physically consistent with the current satellite observation dataset under cloud-free conditions:

[0102] ;

[0103] in, is the corrected radiance value of the historical satellite observation data subset corresponding to the cloud-covered sub-area under the condition of no cloud cover, is the radiance value of the historical satellite observation data subset corresponding to the cloud-covered sub-area under cloud-free conditions. In this way, the historical data is adjusted to be consistent with the current observation data under cloud-free conditions.

[0104] Step A5, using the corrected historical satellite observation data subset corresponding to the cloud cover sub-area, the historical satellite observation data subset corresponding to the target area is replaced and completed to obtain the corrected historical satellite observation data set corresponding to the target area.

[0105] In one example, the process of replacing the cloud covered sub-region includes: , replace the radiance values ​​of the current satellite observation data subset with the corrected radiance values ​​of the historical satellite observation data subset:

[0106] ;

[0107] In one example, the process of outputting the completed data includes: generating a new multi-channel radiation dataset, completing the radiation values ​​of the cloud cover area, and ensuring spatiotemporal consistency and physical rationality.

[0108] The embodiment of the present invention improves the reliability of radiation data in the cloud-covered sub-region by selecting a subset of historical satellite observation data corresponding to the cloud-covered sub-region under similar geographical conditions in the absence of cloud coverage. At the same time, the embodiment of the present invention ensures spatiotemporal consistency and physical rationality through the joint processing of spatial registration and radiation correction. This training data generation method not only makes full use of existing satellite observation data, but also effectively solves the problem of being unable to obtain true radiation values ​​due to the influence of clouds, thereby improving the reliability and availability of training data.

[0109] Before explaining the aforementioned step S106, the embodiment of the present invention provides a specific implementation of a generative collaborative adversarial network. The generative collaborative adversarial network proposed in the embodiment of the present invention is optimized and innovated in structure and function compared to the traditional generative adversarial network (GAN) architecture. The detailed introduction is as follows:

[0110] The traditional Generative Adversarial Network (GAN) is a dual network architecture consisting of a generator and a discriminator, which generates target data through an adversarial game between the two. The function of the generator is to map random noise vectors to the target data distribution and generate samples similar to real data. Its goal is to maximize the probability that the generated samples are recognized as real data by the discriminator. The discriminator divides the input samples into real samples and generated samples through classification tasks, aiming to improve the ability to recognize false samples. The game between the two is carried out in an alternating optimization manner, that is, the generator is fixed to optimize the discriminator, and then the discriminator is fixed to optimize the generator to achieve the final balance. The training goal of GAN is to make the samples generated by the generator so realistic that the discriminator cannot distinguish between real and false. This process depends on the loss function of the generator and the discriminator. The loss function of the generator is based on the adversarial loss, that is, to maximize the probability of the generated samples passing the discriminator; the loss function of the discriminator uses the cross entropy loss to minimize the probability of classification errors. Its limitations are: First, GAN is prone to mode collapse, that is, the generator may concentrate on generating a limited number of sample modes during training, resulting in a lack of diversity in the generated samples. Second, the training process of GAN is prone to instability. Since the generator and the discriminator need to be optimized at the same time, the two sides may cause gradient explosion or disappearance problems during the game. In addition, traditional GAN ​​has the problem of insufficient physical consistency in specific application scenarios (such as satellite data repair). Although the generated samples may be visually realistic, their generated results often lack physical authenticity and consistency, and it is difficult to meet the requirements of strict matching of radiation values ​​and spatial distribution in satellite data repair.

[0111] In view of these limitations of traditional GAN, the embodiment of the present invention proposes a generative collaborative adversarial network. Through multi-channel input, collaborative discrimination mechanism and physical consistency optimization strategy, the generation quality and physical rationality of the model are significantly improved, and a more efficient and accurate solution is provided for satellite data repair and enhancement tasks. Specifically: First, the generator adopts multi-channel input optimization and multi-channel collaborative mechanism optimization strategy. The multi-channel input optimization strategy not only uses the satellite observation data of the current time period (that is, the current satellite observation data set mentioned above), but also integrates the observation data of the cloud-free sub-area in the historical time period (that is, the historical satellite observation data set) as auxiliary input. This design fully exploits the correlation of satellite observation data in time and spectral dimensions, and can restore missing data more accurately. The multi-channel collaborative mechanism optimization strategy designs a multi-generator output fusion mechanism to enhance the collaborative effect between different generators. The outputs of the three generators are weighted and integrated according to the task weights, so that the final output image has both detail richness and global consistency. This collaborative design can maximize the strengths of each generator and ensure the high-quality performance of the repaired image in complex scenes. In addition, a collaborative discriminant mechanism is introduced in the discriminator, using a spatial discriminator and a global discriminator to work together. The spatial discriminator focuses on the detail quality of the missing data completion area to ensure that the completion result is highly consistent with the real data in terms of texture and local structure; the global discriminator improves the authenticity of the completed data in the overall distribution by evaluating the physical consistency and radiation smoothness of the entire image. Finally, by introducing a collaborative loss function, combining adversarial loss and reconstruction error, the model can not only generate realistic completed data, but also directly optimize radiation consistency during the generation process, thereby overcoming the problem that it is difficult to balance local completion quality and overall physical consistency in traditional methods.

[0112] The above optimizations are described as follows:

[0113] (1) Optimization strategy of the generator’s multi-channel collaboration mechanism:

[0114] Among the optimization strategies of this generator, the first one is: multi-channel input optimization strategy.

[0115] Specifically, the input data not only includes the satellite observation dataset of the current time period, but also integrates the satellite observation dataset of the cloud-free sub-region in the historical time period (that is, the corrected historical satellite observation dataset) as auxiliary input to fully explore the potential correlation of satellite observation data in time and spectral dimensions. This design aims to improve the generator's ability to recover missing data, especially to achieve accurate completion in the cloud cover sub-region.

[0116] The input data is constructed as follows: is the satellite observation dataset for the current time period, is the corresponding cloud mask data (a value of 1 indicates that there are clouds, and a value of 0 indicates that there are no clouds). The satellite observation dataset of the cloud-free sub-region in the historical period. Build as:

[0117] ;

[0118] in, It is a decay factor based on time weight, which is used to reduce the weight of observation data that is far away in time. The formula is:

[0119] ;

[0120] in, is the time difference and τ is the time constant.

[0121] Among the optimization strategies of this generator, the second one is: multi-generator collaborative optimization strategy.

[0122] See also Figure 2 The schematic diagram of a generative collaborative adversarial network framework is shown in FIG. The multi-channel generator includes a small cloud block generator G1, a large cloud block generator G2, and a special cloud block generator G3. Generators G1, G2, and G3 each focus on different restoration tasks, so that they work together in the overall generation task to improve the accuracy and quality of image restoration. Specifically:

[0123] Small Cloud Block Generator G1: Focuses on small-scale cloud block repair.

[0124] Small cloud block generator G1 is mainly responsible for processing small cloud occlusion sub-areas, usually local small cloud blocks or light cloud layers. Through targeted structural design, small cloud block generator G1 can perform detailed image completion in a smaller repair area, ensuring that the repair result is highly consistent with the surrounding area in texture and details. To achieve this goal, small cloud block generator G1 uses shallow convolutional layers and a sophisticated feature extraction mechanism to quickly capture local information.

[0125] Large Cloud Block Generator G2: Focuses on large-scale cloud block repair.

[0126] The large cloud block generator G2 is responsible for repairing large-scale cloud occlusion areas, which usually involves the restoration of more complex large cloud blocks or extensive cloud layers. Unlike the small cloud block generator G1, the large cloud block generator G2 has a deeper network architecture and adopts a more powerful global feature extraction module to ensure the overall understanding of large-scale cloud occlusion sub-regions during the repair process. Its optimization goal is to balance local details with global consistency to ensure that the images repaired in large areas meet high standards in physical and radiometric consistency.

[0127] Special Cloud Block Generator G3: Focus on special cloud block repair.

[0128] Special Cloud Block Generator G3 focuses on the repair of special types of cloud blocks, such as thick cloud blocks, convolutional cloud blocks, or high reflectivity cloud layers. The repair of such cloud blocks requires specific algorithm design to accurately restore the surface information below them. To this end, Special Cloud Block Generator G3 uses a specially optimized feature extraction module that can complement the spectral and spatial characteristics of these special cloud blocks. Specifically, Special Cloud Block Generator G3 uses multi-scale convolution layers to capture spatial features at different scales. Since special cloud blocks (such as thick cloud layers, cirrus clouds, etc.) have strong spatial structural differences, traditional single-scale convolutions may not be able to effectively extract these features. Therefore, Special Cloud Block Generator G3 introduces multi-scale convolution operations to extract spatial information from local to global through convolution kernels of different sizes. This design not only helps to handle small-scale cloud block repairs, but also ensures an overall understanding of large-scale cloud blocks and improves the global consistency of the repaired image. These optimization strategies can help Special Cloud Block Generator G3 repair complex special cloud block areas more accurately, improve the overall repair quality, and ensure that the generated images meet high standards in physical and radiometric consistency.

[0129] The input data of each generator is shared, that is, based on the same data, including the current satellite observation dataset and the corrected historical satellite observation dataset. Although the input data is the same, the three generators have different tasks, so they have different network structures and optimize different image restoration goals. During the training process, the output results of the three generators will be integrated through weighted fusion or other collaborative mechanisms to ultimately produce an image with a comprehensive restoration effect.

[0130] (2) Discriminator’s collaborative discrimination mechanism optimization strategy:

[0131] The multi-channel discriminator includes a spatial discriminator (Ds) and a global discriminator (Dg) based on a collaborative mechanism. In an embodiment of the present invention, the discriminator improves the local details and global consistency of the generated results through the collaborative work of the spatial discriminator (Ds) and the global discriminator (Dg).

[0132] The spatial discriminator focuses on the missing data completion area. The input is the local cropped image Xcrop of the cloud coverage sub-area. The texture and detail distribution are learned through the convolutional neural network (CNN) to ensure the consistency of local features between the repaired satellite observation dataset of the cloud coverage sub-area and the real satellite observation dataset.

[0133] The input of the global discriminator is the repaired satellite observation dataset of the entire target area, focusing on the overall distribution, radiation consistency and boundary smoothness of the completed data.

[0134] Based on the above-mentioned generative collaborative adversarial network, an embodiment of the present invention provides a specific implementation method of performing progressive collaborative training on the generative collaborative adversarial network, including the following steps B1 to B4:

[0135] Step B1, generating a restored satellite observation dataset corresponding to the target area based on the current satellite observation dataset of the target area through a multi-channel generator.

[0136] In one example, a small cloud block generator is used to perform local detail repair based on the current satellite observation data set of the target area to obtain a small cloud block repair result; and a large cloud block generator is used to perform global texture repair based on the current satellite observation data set of the target area to obtain a large cloud block repair result; and a special cloud block generator is used to perform complex area repair based on the current satellite observation data set of the target area to obtain a special cloud block repair result; the small cloud block repair result, the large cloud block repair result, and the special cloud block repair result are weightedly fused to obtain a repaired satellite observation data set corresponding to the target area. For details, please refer to the aforementioned explanation of the multi-channel generator, which will not be described in detail in the embodiment of the present invention.

[0137] Step B2, through a multi-channel discriminator, the repaired satellite observation dataset and the corrected historical satellite observation dataset corresponding to the target area generate a judgment result, and the judgment result is used to describe the difference between the repaired satellite observation dataset and the corrected historical satellite observation dataset.

[0138] In one example, a local cropped image corresponding to a cloud cover sub-region is extracted from the restored satellite observation data set, and a spatial discriminator is used to determine that the local cropped image belongs to a first expected value of a real image based on the corrected historical satellite observation data set; a global discriminator is used to determine that the restored satellite observation data set belongs to a second expected value of a real image based on the corrected historical satellite observation data set; and a determination result is determined based on the first expected value and the second expected value. For details, please refer to the aforementioned explanation of the multi-channel discriminator, which will not be described in detail in the embodiment of the present invention.

[0139] Step B3, based on the repaired satellite observation data set and the judgment results, respectively determine the generation collaboration loss value and reward value corresponding to the multi-channel generator and the discrimination collaboration loss value corresponding to the multi-channel discriminator, so as to perform progressive collaborative training on the multi-channel generator and the multi-channel discriminator.

[0140] Specifically, the embodiment of the present invention adopts a multi-stage progressive training strategy, and divides the model training into three stages: a local feature learning stage, a global consistency optimization stage, and a multi-scale fusion stage. Among them, the local feature learning stage is based on the current satellite observation data subset corresponding to the cloud coverage sub-area for training. In the local feature learning stage, the model only trains on a small sample of the cloud coverage sub-area, focusing on detail recovery; the global consistency optimization stage is based on the current satellite observation data set corresponding to the target area of ​​the same scale for training. In the global consistency optimization stage, the entire image is gradually introduced, and the physical consistency of the completed data with the overall scene is ensured through additional global loss terms; the multi-scale fusion stage is based on the current satellite observation data set corresponding to the target area of ​​different scales for training. In the multi-scale fusion stage, the model optimizes the completion quality at different scales through multi-resolution input and output, and improves its adaptability to large-scale cloud coverage scenes.

[0141] In one example, the local feature learning phase: In this phase, the generator is trained only on small sample data in the cloud-covered sub-region, focusing on recovering detailed features. The goal of the generator is to maximize the local consistency of missing data in the region, and the optimization process minimizes the reconstruction error sub-loss value to measure the difference between generated data and real data.

[0142] In one example, the global consistency optimization stage: as the training progresses, the model gradually introduces the entire image for training to ensure the physical consistency of the completed data with the overall scene. In this stage, a global sub-loss value is added , whose goal is to optimize the radiometric consistency of the generated data and ensure that the spectral distribution of the entire image is consistent with the real data.

[0143] In one example, the multi-scale fusion stage: In order to further improve the adaptability to large-scale cloud coverage scenes, the model adopts multi-resolution input and output in this stage. Specifically, the generator and discriminator are trained at different scales to ensure that the completion quality at different scales is optimized. In multi-scale training, the generator's loss function weightedly sums the reconstruction error sub-loss values ​​of different resolutions to obtain a multi-scale sub-loss value, which ensures that the loss of each stage has an appropriate contribution during optimization. For example, the local feature learning stage focuses on the reconstruction error. , while the global consistency optimization stage focuses on the global sub-loss value .

[0144] In the embodiment of the present invention, the emphasis of the generator is different at different stages, that is, the weight distribution of each sub-loss value in the generated collaborative loss value is different. Based on the above training stage, the embodiment of the present invention explains the generated collaborative loss value, the reward value, and the discriminated collaborative loss value respectively.

[0145] (1) Generate collaborative loss values: In the training process of the generative adversarial network (GAN), in order to better optimize the generator and discriminator and ensure the efficiency and accuracy of the image restoration task, the final loss function is composed of multiple sub-loss functions. These sub-loss functions are optimized for different training stages or task objectives and combined by weighted summation.

[0146] Specifically, the collaborative loss value is generated by weighted summing up the reconstruction error sub-loss value, the global consistency sub-loss value, the adversarial sub-loss value, the physical consistency sub-loss value, and the multi-scale sub-loss value; among which, the weight corresponding to the same sub-loss value is different in different stages, and the target-aware weight allocation mechanism is used in the local feature learning stage to dynamically adjust the weights of the reconstruction error sub-loss value and the adversarial sub-loss value.

[0147] Generate collaborative loss value The expression is as follows:

[0148] ;

[0149] The meanings of the parameters in the above expression are as follows:

[0150] Meaning of (weight of reconstruction error sub-loss value): reconstruction error sub-loss value The weight coefficient is used to control the importance of the reconstruction error loss in the final loss; The role of : This parameter helps adjust the generator's goal in the local feature learning phase, focusing on minimizing the local difference between the generated image (i.e., the restored satellite observation dataset) and the real image (i.e., the corrected historical satellite observation dataset). Increasing the value of α means that the training focuses more on detail recovery.

[0151] (Weight of global consistency sub-loss value) Meaning: The weight coefficient of the global consistency sub-loss value, which is used to adjust the consistency of the generated image with the real image in the global range; Role: The global consistency sub-loss value ensures the radiometric consistency and physical consistency of the generated image, especially in the spectral distribution and other global features of the image. Increasing the value of β helps the generated image to be more physically consistent and have a global structure.

[0152] (Weight for adversarial loss) Meaning: γ is the weight coefficient of the adversarial loss, which is used to balance the adversarial training between the generator and the discriminator; Function: The adversarial loss is usually a key component used to train the discriminator and generator, ensuring that the discriminator can distinguish between real data and generated data, while pushing the generator to improve the quality of generated data. Increasing γ will make the generator pay more attention to the authenticity of generated data in adversarial training.

[0153] (Weight of physical consistency sub-loss value) Meaning: δ is the weight coefficient of the physical consistency sub-loss value, which is based on the physical characteristics of the generated image such as radiation consistency and spectral distribution; Function: The physical consistency sub-loss value can ensure that the generated image conforms to specific rules or conditions in terms of physical properties. For satellite image restoration tasks, this loss is very important because it ensures that the restored image is not only visually reasonable, but also maintains consistency in terms of radiation characteristics.

[0154] λs (weight of multi-scale sub-loss value) Meaning: λs is the weight coefficient of each scale loss sub-loss value, which is used to adjust the reconstruction error sub-loss value at different resolutions in multi-scale training; λs function: In multi-scale training, images are processed at different resolutions. The loss function of each resolution is calculated according to different scales (s), and the influence of each scale loss can be adjusted by λs. By adjusting the weight of each scale, it can be ensured that the restoration of image details at different scales can be reasonably optimized.

[0155] 𝑆 (number of scales) means: the number of scales in the multi-scale training process, that is, the number of different resolutions involved in the training; 𝑆 role: In multi-scale training, S determines the number of scales used by the model. Generally, more scales mean that the model can process more image information, thereby enhancing the quality and details of the generated images.

[0156] Furthermore, the expression of the reconstruction error sub-loss value is as follows:

[0157] ;

[0158] in, Represents the data generated by the generator, xi is the real data, and N is the number of samples. The reconstruction error sub-loss value measures the similarity between the generated image and the real image in structure and content, usually using the mean square error (MSE) or structural similarity index (SSIM).

[0159] The expression of the global consistency sub-loss value is as follows:

[0160] ;

[0161] Among them, M is the number of global samples, |⋅|1 is the L1 norm, which is used to measure the difference in the global range. is the image output by the generator, It is a historical image after radiometric correction.

[0162] The physical consistency loss function is used to ensure that the generated image is consistent with the real image in terms of radiation consistency and spatial distribution consistency, which can be calculated by the weighted sum of radiation consistency loss and spatial distribution consistency loss.

[0163] The expression for the adversarial loss value is as follows:

[0164] ;

[0165] in, : The completed image (i.e., fake image) generated by the generator G. :The spatial discriminator generates images The output of represents the probability that the generated image belongs to the real completion area. :Global discriminator generates images The output of represents the probability that the generated image belongs to the real image. The adversarial sub-loss describes the gap between the generated image and the real image, which is realized through the output of the spatial and global discriminator. This sub-loss is a loss function that measures the similarity between the image generated by the generator and the real image and provides feedback through the output of the discriminator. Usually, this sub-loss is the core part of the adversarial game between the generator and the discriminator.

[0166] The expression of the multi-scale sub-loss value is as follows:

[0167] ;

[0168] Where S represents the number of scales, is the weight of each scale, is the reconstruction error sub-loss value at each scale.

[0169] In the embodiment of the present invention, based on the above generation of the collaborative loss value, the optimization target of the generator is: .

[0170] The overall optimization process is as follows: By adjusting the weights of each loss function (α, β, γ, δ and λs), the model can reasonably balance different task objectives (such as detail recovery, global consistency, adversarial training, physical consistency, multi-scale optimization, etc.) at different stages. Specifically:

[0171] In the local feature learning stage, the reconstruction error loss may dominate and the weight α can be large. In the global consistency optimization stage, the weight β of the global consistency loss will increase to ensure global physical consistency. In the adversarial training stage, the weight γ of the adversarial loss will increase to promote the generator to improve the authenticity of the generated image. The weight δ of the physical consistency loss can be adjusted according to the specific application scenario, which is especially critical in the satellite image restoration task. The weight coefficient λs of the multi-scale loss will be adjusted according to the image quality of different resolutions to ensure the optimization of multi-scale information. This weighted combination ensures that the generator and discriminator can be continuously optimized throughout the training process, and ultimately generate high-quality, physically consistent, and detail-recovery images.

[0172] Furthermore, in the local feature learning stage, in order to solve the problem of difficulty in balancing adversarial loss and reconstruction error in traditional methods, the present invention proposes a target perception weight allocation mechanism. This mechanism dynamically adjusts the ratio of adversarial loss and reconstruction error by analyzing the semantic difference between the generator output and the real data. For the repair of satellite data, especially in the cloud cover sub-area, for complex areas and areas with large changes, the weights of reconstruction error and adversarial loss need to be dynamically adjusted to ensure the accuracy of the completed radiation value and the optimization of texture details. For example, for areas with large changes in radiation values, the reconstruction error weight will be higher to ensure the physical accuracy of the generated results; while in areas with more complex detail features, the adversarial loss weight will be increased to optimize the texture generation effect. This mechanism overcomes the problem that fixed weights in traditional methods cannot adapt to different scenarios, so that the model can perform well under various complex cloud coverage conditions.

[0173] In traditional generative adversarial networks, the adversarial loss ( ) and reconstruction error ( ) is usually fixed, which is not applicable when processing satellite images of different regions. Therefore, the embodiment of the present invention dynamically adjusts the ratio of these two loss terms by analyzing the semantic difference between the generator output and the real data. For example, in areas with large changes in radiation values, the generator will focus on optimizing the reconstruction error sub-loss value, while in areas with complex textures and details, the weight of the adversarial sub-loss value will be enhanced. The expression of the target perception weight allocation mechanism is:

[0174] ;

[0175] Among them, α is a dynamically adjusted weight that controls the reconstruction error sub-loss value and adversarial loss The relative importance of . This value is adjusted according to the feature complexity of different regions. This mechanism enables the network to adjust the optimization target according to the specific conditions of each region and improve the completion quality.

[0176] (2) The reward value is obtained by weighted summing the texture sub-loss value and the radiation sub-loss value.

[0177] To further improve the performance of the generator, the embodiment of the present invention combines the reinforcement learning optimization strategy. Traditional training methods usually rely on fixed optimization goals, while reinforcement learning guides the behavior of the generator through a reward mechanism, enabling it to learn and optimize from the quality feedback of each generated data. Specifically, after the generator generates missing area data each time, the model scores according to the quality of the generated results and adjusts the generation strategy according to the score. The scoring calculation formula is:

[0178] ;

[0179] Among them, w1 and w2 are the weights of the texture sub-loss value and the radiation sub-loss value, and They represent the texture sub-loss value and radiation sub-loss value of the generated data respectively. By designing a reward mechanism, the generator can adaptively adjust its generation strategy to achieve a more accurate missing data completion effect.

[0180] The texture sub-loss value is used to measure the texture difference between the generated image and the real image. This loss function is usually calculated using the perceptual loss method. Perceptual loss compares the difference between the generated image and the real image in the feature space, not just the difference at the pixel level, so it can better capture the texture and high-level semantic information of the image. Specific formula and parameter description:

[0181] ;

[0182] in, : This is a feature extraction process of a pre-trained convolutional neural network (such as VGG). ϕ(X) represents the high-level features obtained after inputting image X into the network. Specifically, the feature map output by a certain layer (such as the convolutional layer) of the VGG network usually contains information such as texture, edge, shape, etc. of the image, which are very helpful for evaluating texture quality. : Generated image, that is, the image produced by the generator. Usually, it is a completed image of the missing area that has been repaired or generated. : The real image, that is, the actual image data corresponding to the generated image. For satellite image restoration tasks, the real image is usually a complete image without missing areas. Represents the L2 norm (Euclidean distance), which is used to measure the difference between two eigenvectors. The L2 norm calculates the sum of the squared differences between the two eigenvectors and then squares them to express the distance between them.

[0183] The texture sub-loss value measures the difference between the generated image and the real image in the high-level feature space, rather than the difference directly in the pixel space. This method allows the model to focus not only on the visual similarity of the image (pixel value), but also on the high-level semantic features of the image, such as edges, shapes, textures, etc. The smaller the loss value, the higher the texture similarity between the generated image and the real image, and the better the performance of the generator.

[0184] (3) The discriminant collaborative loss value is obtained by weighted summing the spatial sub-loss value and the global sub-loss value.

[0185] The expression for discriminating the collaborative loss value is:

[0186] ;

[0187] in, : The overall loss value of the collaborative discriminator combines the contributions of the spatial discriminator and the global discriminator, indicating the overall performance of the generated image in terms of local details and global consistency. : The loss value of the spatial discriminator, which measures the consistency of the completed area in local details. : The loss value of the global discriminator, which measures the consistency of the entire image in terms of radiation consistency and boundary smoothness. λs: The weight coefficient of the spatial discriminator, which controls the relative importance of the spatial discriminant loss. The larger the value, the greater the influence of the spatial discriminator in the collaborative loss. λg: The weight coefficient of the global discriminator, which controls the relative importance of the global discriminant loss. The larger the value, the greater the influence of the global discriminator in the collaborative loss.

[0188] The expression of the spatial sub-loss value is:

[0189] ;

[0190] in, : The loss value of the spatial discriminator, which measures the performance of the spatial discriminator in distinguishing between real and generated completion region images. : The expected value of the real image, which represents the judgment of the spatial discriminator on the real completion area. : The expected value of the generated (fake) image, which represents the decision of the spatial discriminator on the generated completion area. : The image of the real completion area, usually the historical radiation data after the radiation deviation correction. : The generated completed area image comes from the output of the generator, and the goal is to be as consistent as possible with the real data in local texture and details.

[0191] The expression of the global sub-loss value is:

[0192] ;

[0193] in, : The loss value of the global discriminator, which measures the performance of the global discriminator in distinguishing between real and generated complete images. : The expected value of the real image, which represents the judgment of the global discriminator on the real image. : The expected value of the generated (fake) image, which represents the judgment of the global discriminator on the generated image. : A complete real image, usually historical radiation data after radiation bias correction. Includes all areas (i.e., a composite image of the completed area and the cloud-free area). : The generated full image, containing the cloud covered sub-regions inpainted by the generator. : The output of the global discriminator, indicating the probability that the input image X belongs to a real image. , this value should be as close to 1 as possible, for generating images , the value should be as close to 0 as possible.

[0194] Through the combination of these collaborative mechanisms and optimization objectives, the generative collaborative adversarial network can effectively repair cloud-occluded areas and generate high-quality, physically-compliant images.

[0195] Example: Assuming that there is a cloud-occluded area in the satellite image on May 1, 2024, the generator uses historical cloud-free images as a reference and generates a repaired image based on the type of cloud occlusion (such as small cloud blocks or large cloud blocks). The discriminator evaluates the difference between the generated image and the real cloud-free image and feeds back the result to the generator. The collaborative mechanism combines the outputs of multiple generators to ultimately generate a high-quality image that incorporates different repair details. By optimizing the adversarial loss, reconstruction error, and physical consistency loss, the final generated image not only has visual authenticity, but also maintains radiometric and physical consistency to achieve the effect of repair.

[0196] Through the above optimization strategies, the embodiment of the present invention proposes a more efficient and accurate collaborative adversarial network (Collaborative GAN, Co-GAN) method based on the traditional adversarial generative network method (GAN). This method fully exploits the spatiotemporal correlation characteristics of satellite data through a multi-channel input strategy; the collaborative discrimination mechanism achieves a balance between detail completion and global consistency; the collaborative loss function dynamically adjusts between adversarial loss and physical consistency optimization, solving the problem of balancing local completion quality and overall physical consistency. These innovative strategies significantly improve the quality and reliability of satellite data repair and enhancement, and are particularly suitable for image processing tasks in complex cloud coverage sub-areas.

[0197] Step B4, in the local feature learning stage, the global consistency optimization stage and the multi-scale fusion stage, the multi-channel generator and the multi-channel discriminator are trained by the AdamW optimizer, the network parameters of the multi-channel generator are adjusted based on the generation collaboration loss value and the reward value, the network parameters of the multi-channel discriminator are adjusted based on the disk collaboration loss value, and the training process is optimized using the gradient clipping strategy and the periodic learning rate strategy.

[0198] The AdamW optimizer, gradient clipping strategy, and periodic learning rate strategy are explained below:

[0199] (1) AdamW optimizer:

[0200] The entire training process uses the AdamW (Adaptive Moment Estimation with Weight Decay) optimizer instead of the traditional batch gradient descent method. The AdamW optimizer not only combines momentum updates, but also introduces weight decay, effectively reducing the risk of overfitting. The update formula of the AdamW optimizer is as follows:

[0201] ;

[0202] in: and are the mean and variance of the gradient respectively; β1 and β2 are the attenuation coefficients of momentum; Represents the gradient, i.e. the loss function The partial derivative with respect to the model parameter θt. The gradient measures the rate of change of the loss function at the current parameter value θt and is the key information used to adjust the parameters during the optimization process; is the loss function of the model under given parameters θt (such as adversarial loss, reconstruction error, etc.). The goal of the optimization process is to minimize the loss function and thus improve the performance of the model. The gradient of the current parameter θt on the loss function L is calculated, which indicates the direction and magnitude of each parameter adjustment. η is the learning rate; λ is the weight decay coefficient; ϵ is a constant to prevent division by zero; is the updated value of the parameter θ, which represents the value of the model parameters (such as weights and biases in neural networks) in the tth iteration. The goal of the training process is to update these parameters so that the loss function of the model is as small as possible.

[0203] By using this optimizer, the model can converge more stably, especially in large-scale satellite data repair tasks. The AdamW optimizer provides a more stable and effective optimization mechanism. It is particularly suitable for deep learning tasks, especially when dealing with large-scale data or complex models, and can accelerate convergence and reduce overfitting.

[0204] (2) Gradient clipping strategy:

[0205] In order to avoid the problem of gradient explosion or gradient vanishing, the present invention also combines a gradient clipping strategy. During the gradient back propagation process, if the modulus of the gradient exceeds the set threshold τ, the gradient is clipped. The specific clipping formula is:

[0206] ;

[0207] in: is the gradient; is the modulus of the gradient (i.e. the norm of the gradient); is the threshold for gradient clipping (for example, set to 1.0); is the unit gradient direction; if the modulus of the gradient exceeds the threshold τ, the gradient will be clipped to this threshold to prevent excessive gradients from causing network instability. This strategy can effectively control gradient fluctuations during training and ensure the stability of the model. The AdamW optimizer combines the gradient clipping strategy to further accelerate the model convergence speed and prevent gradient explosion or disappearance problems.

[0208] (3) Periodic learning rate strategy:

[0209] In order to further improve the training efficiency and avoid the model falling into the local optimal solution, the present invention adopts the Cyclic Learning Rate (CLR) strategy. CLR increases the training robustness of the model and accelerates the convergence process of the model by dynamically adjusting the learning rate. The learning rate update formula is:

[0210] ;

[0211] Among them, ηmin and ηmax are the minimum and maximum learning rates, T is the cycle length, and t is the number of steps in the current training. By constantly changing the learning rate, CLR can prevent the model from falling into the local optimal solution and continuously explore new optimal solutions throughout the training process, thereby accelerating convergence.

[0212] The cyclical learning rate (CLR) strategy dynamically adjusts the learning rate during the training process to prevent the model from falling into a local optimal solution and improve training efficiency. The cyclical learning rate continuously changes the learning rate value within a certain range, guiding the generator and discriminator to find the best solution in different loss spaces, thereby enhancing the robustness of the model.

[0213] Among the above optimization strategies, the multi-stage progressive training strategy, the target-aware weight allocation mechanism, and the reinforcement learning optimization strategy are optimization strategies for the training phase of the generator. The update mechanism, gradient clipping strategy, and periodic learning rate strategy of the AdamW optimizer are optimization strategies for the entire model framework, not just individual optimizations for the generator or discriminator.

[0214] Through the above optimization methods, including multi-stage progressive training strategy, AdamW optimizer, periodic learning rate, target perception weight allocation mechanism and reinforcement learning optimization strategy, the embodiments of the present invention not only improve the convergence speed during the training process, but also enhance the quality and accuracy of the generated data compared to the traditional training methods. Especially when processing satellite cloud coverage data repair, these innovative methods can effectively improve the radiation consistency and texture restoration ability of missing data, overcome multiple technical difficulties in traditional methods, and provide a more accurate and robust data repair solution.

[0215] Furthermore, in order to verify the effect of the collaborative GAN restoration model and evaluate its performance in practical applications, the embodiment of the present invention adopts a variety of common image quality evaluation indicators and comparison algorithms. The evaluation process includes quantitative analysis and qualitative analysis, covering the visual effect, restoration accuracy and physical consistency of the image. The specific implementation steps are as follows:

[0216] (I) Quantitative evaluation: In terms of quantitative evaluation, the present invention uses the following common image quality evaluation indicators:

[0217] Peak Signal-to-Noise Ratio (PSNR): Peak Signal-to-Noise Ratio is an important indicator for measuring image quality. It reflects the difference between the restored image and the real image. The higher the PSNR value, the closer the restored image is to the real image. Its calculation formula is:

[0218] ;

[0219] Where Imax is the maximum pixel value of the image (for example, for an 8-bit image, Imax=255), and MSE is the mean square error, defined as:

[0220] ;

[0221] in, and are the values ​​of the real image and the repaired image at pixel position i, respectively, and N is the total number of pixels in the image.

[0222] Structural Similarity Index (SSIM):

[0223] The structural similarity index is an indicator to measure the structural similarity of images, which can capture the brightness, contrast and structural information in the image. Its formula is:

[0224] ;

[0225] Among them, μx and μy are the means of images x and y, σx and σy are variances, σxy is covariance, c1 and c2 are constants to avoid the denominator being zero. The closer the SSIM value is to 1, the more similar the structures of the two images are.

[0226] Physical consistency assessment: Physical consistency assessment mainly considers whether the radiometric characteristics of the restored image are consistent with the real image. In order to verify physical consistency, the Radiometric Consistency Index (RCI) is used to evaluate the radiometric consistency between the restored image and the real image.

[0227] The Radiometric Consistency Index (RCI) is defined as:

[0228] ;

[0229] in: is the radiation value of the real image at pixel position i. This is the radiation intensity value from the real or observed image, usually representing the reflectivity or brightness value of a specific pixel in the image. In satellite image processing, the radiation value is usually a value related to the spectral reflectivity; is the radiation value of the repaired image at pixel position 𝑖. This is the radiation intensity value of the image at pixel position 𝑖 after being repaired by the generative model (Collaborative GAN). It represents the spectrum or brightness intensity at a specific position in the repaired image. N is the total number of pixels in the image. The smaller the RCI value, the smaller the difference between the repaired image and the real image in radiation characteristics, that is, the better the radiation consistency of the repaired image. A smaller RCI value means that the repair model performs better in retaining the radiation information of the real image.

[0230] Through the above steps, the embodiment of the present invention can comprehensively evaluate the performance of the generative collaborative adversarial network repair model and perform strict verification from multiple aspects such as visual effect, repair accuracy and physical consistency.

[0231] Finally, the generator generates the completed data through a collaborative discrimination mechanism and outputs the repaired multi-channel radiation data set. The repaired satellite data can not only achieve accurate radiation value completion in the cloud coverage sub-area, but also ensure spatiotemporal consistency and physical rationality. These completed data will be used as input to the numerical assimilation model to improve the accuracy and reliability of weather forecasts. The present invention provides a satellite observation data repair method based on a generative collaborative adversarial network, which can effectively solve the interference of cloud coverage on satellite observation data observation, improve the reliability and availability of cloud coverage sub-area data, and provide more accurate and high-quality data support for meteorological observation and environmental monitoring.

[0232] In summary, the embodiments of the present invention propose a satellite cloud occlusion data repair method based on a generative collaborative adversarial network, which aims to effectively solve the interference problem of cloud coverage sub-areas on satellite image observation data, and significantly improve the repair accuracy and physical consistency of satellite images in complex cloud occlusion environments. Traditional cloud occlusion repair methods usually rely on a single generator or interpolation algorithm, which makes it difficult to handle local details and global consistency at the same time, and are prone to physical inconsistencies and radiation errors. Compared with traditional repair methods, the embodiments of the present invention overcome the shortcomings of traditional methods in detail recovery and global consistency by introducing technologies such as multi-generator collaboration mechanism, dynamic weight allocation, spatiotemporal alignment, and innovative collaborative loss functions, bringing significant technical innovation and optimization to the field of satellite data repair.

[0233] In the process of constructing the training data set, the embodiment of the present invention adopts the method of "multi-temporal and spatial registration", using the historical cloud-free data of the past month to perform spatial and temporal registration with the current cloud-covered sub-area. Through radiation correction, the consistency of the historical data and the current satellite observation data in radiation brightness is ensured, thereby constructing a high-quality cloud-free reference data set. This method can effectively improve the repair effect of the model, especially in the case of complex cloud types and long time spans, providing high-quality training samples.

[0234] The generative collaborative adversarial network of the embodiment of the present invention optimizes the input strategy of the generator and adopts a multi-channel input method. It not only uses the satellite observation data of the current period, but also integrates the data of historical cloud-free areas. This fusion design of spatiotemporal information helps to better restore the missing data of cloud-occluded areas, especially in the repair of cloud-covered sub-areas. It can utilize the time and radiation characteristics of historical data to improve the accuracy and physical consistency of the repaired image. In order to enhance the quality of the repaired image, the embodiment of the present invention also innovates in the discriminator design and adopts a collaborative discriminant mechanism. The discriminator consists of a spatial discriminator and a global discriminator. The former focuses on the quality of local details, while the latter evaluates the global physical consistency and radiation smoothness of the image, which enables the repair results to meet higher standards in terms of local details and global consistency.

[0235] In order to improve the training efficiency and restoration quality, the embodiment of the present invention proposes a multi-stage progressive training strategy. In the early stage of training, the model mainly focuses on the local restoration of cloud-occluded areas and optimizes through a smaller learning rate; as the training progresses, the learning rate gradually increases, and begins to focus on global consistency and physical rationality. Through this staged optimization strategy, the model can gradually improve its ability to repair cloud-occluded areas, ensuring that the restored image achieves the best effect in local details and global consistency. In addition, the embodiment of the present invention uses the AdamW optimizer combined with a periodic learning rate strategy to improve the stability and convergence speed of the training process, avoiding overfitting and gradient problems.

[0236] In addition, the embodiment of the present invention also introduces a target-aware dynamic weight allocation mechanism in the optimization of the model. This mechanism can dynamically adjust the weights of the adversarial loss and reconstruction error according to the difference between the generator output and the real data, so that the model can accurately restore the radiation value and detail features according to the complexity of different cloud occlusion scenes. In addition, combined with the reinforcement learning optimization strategy, the behavior of the generator is guided by the reward mechanism to further improve the quality of the generated results and ensure that the model can perform well in various cloud coverage scenarios. Overall, the embodiment of the present invention effectively improves the repair effect of satellite cloud occlusion data through a multi-generator collaboration mechanism, innovative loss function design, dynamic weight allocation and reinforcement learning optimization, providing more reliable and high-quality data support for meteorological observation and environmental monitoring.

[0237] Based on the above-mentioned embodiments, the present invention provides a satellite observation data repair device based on generating collaborative adversarial networks, see Figure 3 The structure diagram of a satellite observation data repair device based on a generative collaborative adversarial network is shown in FIG. The device mainly includes the following parts:

[0238] A data acquisition module 302 is used to acquire a current satellite observation data set and a historical satellite observation data set of a target area;

[0239] The data correction module 304 is used to extract the cloud-covered sub-region and the cloud-free sub-region from the target area, and based on the current satellite observation data subsets corresponding to the cloud-covered sub-region and the cloud-free sub-region, perform spatiotemporal registration and radiation correction on the historical satellite observation data subset corresponding to the cloud-covered sub-region under the cloud-free condition, so as to obtain a corrected historical satellite observation data set corresponding to the target area;

[0240] The network training module 306 is used to perform progressive collaborative training on the generative collaborative adversarial network using the current satellite observation data set and the corrected historical satellite observation data set of the target area. The generative collaborative adversarial network includes a multi-channel generator and a multi-channel discriminator. The trained multi-channel generator is used to perform data repair on the satellite observation data to be repaired in the target area.

[0241] The satellite observation data repair device based on the generative collaborative adversarial network provided by the embodiment of the present invention uses the current satellite observation data subset corresponding to the cloud-free sub-area to perform spatiotemporal alignment and radiation correction on the historical satellite observation data subset corresponding to the cloud-covered sub-area under the cloud-free condition, so as to overcome the problems such as poor spatiotemporal consistency faced by traditional data repair methods when processing satellite observation data; in addition, unlike the existing repair methods based on interpolation, physical models or single generative adversarial network models, the present invention innovatively introduces a multi-channel generator and a multi-channel discriminator structure of a collaborative generative adversarial network, allowing multiple generators to repair the current satellite observation data set from different perspectives in parallel. At the same time, multiple discriminators further ensure the spatiotemporal consistency, detail retention and physical rationality of the repaired satellite observation data set by comparing the authenticity of different repaired satellite observation data sets.

[0242] In one implementation, the data correction module 304 is specifically configured to:

[0243] Extract the current satellite observation data subset corresponding to the cloud covered sub-region and the current satellite observation data subset corresponding to the cloud-free sub-region from the current satellite observation data set;

[0244] Perform similarity matching on the current satellite observation data subset corresponding to the cloud cover sub-area and the historical satellite observation data set corresponding to the target area, so as to screen out the historical satellite observation data subset corresponding to the cloud cover sub-area when there is no cloud cover; wherein the similarity matching includes one or more of geographical location matching, atmospheric condition matching and radiation characteristic matching;

[0245] After performing spatiotemporal registration on the current satellite observation data subset corresponding to the cloud-covered sub-area and the historical satellite observation data subset corresponding to the cloud-free sub-area, radiometric correction is performed on the historical satellite observation data subset corresponding to the cloud-covered sub-area in the cloud-free sub-area using the current satellite observation data subset corresponding to the cloud-free sub-area, so as to obtain the corrected historical satellite observation data subset corresponding to the cloud-covered sub-area;

[0246] The corrected historical satellite observation data subset corresponding to the cloud cover sub-area is used to replace and complete the historical satellite observation data subset corresponding to the target area, so as to obtain the corrected historical satellite observation data set corresponding to the target area.

[0247] In one implementation, the data correction module 304 is specifically configured to:

[0248] Determine the radiance deviation value between the radiance value of the current satellite observation data subset corresponding to the cloud-free sub-region and the radiance value of the historical satellite observation data subset corresponding to the cloud-free sub-region under the condition of no cloud coverage;

[0249] The sum of the radiation brightness deviation value and the radiation brightness value of the historical satellite observation data subset corresponding to the cloud cover sub-area under the cloud-free condition is determined to achieve radiation correction for the historical satellite observation data subset corresponding to the cloud cover sub-area under the cloud-free condition.

[0250] In one implementation, the network training module 306 is specifically used to:

[0251] Generate a repaired satellite observation dataset corresponding to the target area based on the current satellite observation dataset of the target area through a multi-channel generator;

[0252] Through the multi-channel discriminator, the repaired satellite observation dataset and the corrected historical satellite observation dataset corresponding to the target area generate a judgment result, and the judgment result is used to describe the difference between the repaired satellite observation dataset and the corrected historical satellite observation dataset;

[0253] Based on the repaired satellite observation dataset and the judgment results, the generation collaboration loss value and reward value corresponding to the multi-channel generator and the discrimination collaboration loss value corresponding to the multi-channel discriminator are determined respectively to perform progressive collaborative training on the multi-channel generator and the multi-channel discriminator.

[0254] In one embodiment, the multi-channel generator includes a small cloud block generator, a large cloud block generator and a special cloud block generator; the network training module 306 is specifically used for:

[0255] Through the small cloud block generator, local details are repaired based on the current satellite observation data set of the target area to obtain the small cloud block repair result;

[0256] And, through the large cloud block generator, global texture restoration is performed based on the current satellite observation data set of the target area to obtain the large cloud block restoration result;

[0257] And, through the special cloud block generator, complex area repair is performed based on the current satellite observation data set of the target area to obtain the special cloud block repair result;

[0258] The repair results of small cloud blocks, large cloud blocks and special cloud blocks are weighted fused to obtain the repaired satellite observation dataset corresponding to the target area.

[0259] In one implementation, the multi-channel discriminator includes a spatial discriminator and a global discriminator based on a collaborative mechanism; the network training module 306 is specifically used for:

[0260] Extracting a local cropped image corresponding to the cloud cover sub-area from the repaired satellite observation data set, and using a spatial discriminator to determine the first expected value of the local cropped image belonging to the real image based on the corrected historical satellite observation data set;

[0261] Through a global discriminator, it is determined that the restored satellite observation dataset belongs to the second expected value of the real image based on the corrected historical satellite observation dataset;

[0262] The determination result is determined according to the first expected value and the second expected value.

[0263] In one embodiment, the progressive collaborative training includes a local feature learning stage, a global consistency optimization stage, and a multi-scale fusion stage; wherein the local feature learning stage is training based on a subset of current satellite observation data corresponding to a cloud coverage sub-region; the global consistency optimization stage is training based on a current satellite observation data set corresponding to a target region of the same scale; and the multi-scale fusion stage is training based on a current satellite observation data set corresponding to a target region of different scales.

[0264] The generated collaborative loss value is obtained by weighted summing up the reconstruction error sub-loss value, the global consistency sub-loss value, the adversarial sub-loss value, the physical consistency sub-loss value, and the multi-scale sub-loss value; the weights corresponding to the same sub-loss value are different in different stages, and the target-aware weight allocation mechanism is used in the local feature learning stage to dynamically adjust the weights of the reconstruction error sub-loss value and the adversarial sub-loss value;

[0265] The reward value is obtained by weighted summing the texture sub-loss value and the radiation sub-loss value;

[0266] The discriminant collaborative loss value is obtained by weighted summing the spatial sub-loss value and the global sub-loss value.

[0267] In one implementation, the network training module 306 is specifically used to:

[0268] In the local feature learning stage, global consistency optimization stage and multi-scale fusion stage, the multi-channel generator and multi-channel discriminator are trained by AdamW optimizer, and the training process is optimized by gradient clipping strategy and periodic learning rate strategy.

[0269] The device provided in the embodiment of the present invention has the same implementation principle and technical effects as those of the aforementioned method embodiment. For the sake of brief description, for matters not mentioned in the device embodiment, reference may be made to the corresponding contents in the aforementioned method embodiment.

[0270] An embodiment of the present invention provides an electronic device. Specifically, the electronic device includes a processor and a storage device. The storage device stores a computer program, and when the computer program is executed by the processor, it executes the method described in any one of the above-mentioned implementation methods.

[0271] Figure 4 A structural diagram of an electronic device provided in an embodiment of the present invention, the electronic device 100 includes: a processor 40, a memory 41, a bus 42 and a communication interface 43, wherein the processor 40, the communication interface 43 and the memory 41 are connected via the bus 42; the processor 40 is used to execute an executable module stored in the memory 41, such as a computer program.

[0272] The memory 41 may include a high-speed random access memory (RAM), and may also include a non-volatile memory, such as at least one disk memory. The communication connection between the system network element and at least one other network element is realized through at least one communication interface 43 (which may be wired or wireless), and the Internet, wide area network, local area network, metropolitan area network, etc. may be used.

[0273] The bus 42 may be an ISA bus, a PCI bus, or an EISA bus, etc. The bus may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 4 Only one bidirectional arrow is used in the diagram, but this does not mean that there is only one bus or only one type of bus.

[0274] Among them, the memory 41 is used to store programs, and the processor 40 executes the program after receiving the execution instruction. The method executed by the device for flow process definition disclosed in any embodiment of the above-mentioned embodiment of the present invention can be applied to the processor 40 or implemented by the processor 40.

[0275] The processor 40 may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method can be completed by the hardware integrated logic circuit or software instructions in the processor 40. The above processor 40 can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. The methods, steps and logic block diagrams disclosed in the embodiments of the present invention can be implemented or executed. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor. The steps of the method disclosed in the embodiment of the present invention can be directly embodied as a hardware decoding processor to be executed, or the hardware and software modules in the decoding processor can be executed. The software module can be located in a mature storage medium in the field such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory, or an electrically erasable programmable memory, a register, etc. The storage medium is located in the memory 41, and the processor 40 reads the information in the memory 41 and completes the steps of the above method in combination with its hardware.

[0276] The computer program product of the readable storage medium provided in the embodiment of the present invention includes a computer-readable storage medium storing program code, and the instructions included in the program code can be used to execute the methods described in the previous method embodiments. The specific implementation can be referred to the previous method embodiments, which will not be repeated here.

[0277] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions to enable a computer device (which can be a personal computer, server, or network device, etc.) to perform all or part of the steps of the methods described in various embodiments of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc., which can store program codes.

[0278] Finally, it should be noted that the above-described embodiments are only specific implementations of the present invention, which are used to illustrate the technical solutions of the present invention, rather than to limit them. The protection scope of the present invention is not limited thereto. Although the present invention is described in detail with reference to the above-described embodiments, ordinary technicians in the field should understand that any technician familiar with the technical field can still modify the technical solutions recorded in the above-described embodiments within the technical scope disclosed by the present invention, or can easily think of changes, or make equivalent replacements for some of the technical features therein; and these modifications, changes or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention shall be subject to the protection scope of the claims.

Claims

1. A satellite observation data repair method based on generative collaborative adversarial network, characterized in that: include: Obtain current satellite observation datasets and historical satellite observation datasets for the target area; Extracting a cloud-covered sub-region and a cloud-free sub-region from the target region, and performing spatiotemporal registration and radiation correction on a historical satellite observation data subset corresponding to the cloud-covered sub-region under a cloud-free condition based on current satellite observation data subsets corresponding to the cloud-covered sub-region and the cloud-free sub-region, so as to obtain a corrected historical satellite observation data set corresponding to the target region; The generative collaborative adversarial network is progressively collaboratively trained using the current satellite observation dataset and the corrected historical satellite observation dataset of the target area. The generative collaborative adversarial network includes a multi-channel generator and a multi-channel discriminator. The trained multi-channel generator is used to perform data repair on the satellite observation data to be repaired in the target area. The progressive collaborative training includes a local feature learning stage, a global consistency optimization stage and a multi-scale fusion stage; wherein the local feature learning stage is training based on the current satellite observation data subset corresponding to the cloud coverage sub-area; the global consistency optimization stage is training based on the current satellite observation dataset corresponding to the target area of ​​the same scale; and the multi-scale fusion stage is training based on the current satellite observation dataset corresponding to the target area of ​​different scales.

2. The satellite observation data repair method based on generative collaborative adversarial network according to claim 1 is characterized in that: Based on the current satellite observation data subsets corresponding to the cloud-covered sub-region and the cloud-free sub-region respectively, performing spatiotemporal registration and radiation correction on the historical satellite observation data subset corresponding to the cloud-covered sub-region in the cloud-free case, so as to obtain a corrected historical satellite observation data set corresponding to the target area, including: Extracting, from the current satellite observation data set, a current satellite observation data subset corresponding to the cloud-covered sub-region and a current satellite observation data subset corresponding to the cloud-free sub-region respectively; Performing similarity matching on the current satellite observation data subset corresponding to the cloud-covered sub-area and the historical satellite observation data set corresponding to the target area, so as to filter out the historical satellite observation data subset corresponding to the cloud-covered sub-area when there is no cloud coverage; wherein the similarity matching includes one or more of geographical location matching, atmospheric condition matching, and radiation characteristic matching; After performing spatiotemporal registration on the current satellite observation data subset corresponding to the cloud-covered sub-area and the historical satellite observation data subset corresponding to the cloud-free sub-area in a cloud-free state, performing radiometric correction on the historical satellite observation data subset corresponding to the cloud-covered sub-area in a cloud-free state using the current satellite observation data subset corresponding to the cloud-free sub-area to obtain a corrected historical satellite observation data subset corresponding to the cloud-covered sub-area; The corrected historical satellite observation data subset corresponding to the cloud cover sub-area is used to replace and complete the historical satellite observation data subset corresponding to the target area, so as to obtain a corrected historical satellite observation data set corresponding to the target area.

3. The satellite observation data repair method based on generative collaborative adversarial network according to claim 2 is characterized in that: Using the current satellite observation data subset corresponding to the cloud-free sub-area, performing radiation correction on the historical satellite observation data subset corresponding to the cloud-covered sub-area in the cloud-free sub-area, including: Determine a radiance deviation value between a radiance value of the current satellite observation data subset corresponding to the cloud-free sub-region and a radiance value of a historical satellite observation data subset corresponding to the cloud-free sub-region under cloud-free coverage; Determine the sum of the radiation brightness deviation value and the radiation brightness value of the historical satellite observation data subset corresponding to the cloud cover sub-area when there is no cloud cover, so as to achieve radiation correction of the historical satellite observation data subset corresponding to the cloud cover sub-area when there is no cloud cover.

4. The satellite observation data repair method based on generative collaborative adversarial network according to claim 1 is characterized in that: Using the current satellite observation dataset and the corrected historical satellite observation dataset of the target area, performing progressive collaborative training on a generative collaborative adversarial network, including: Generate a restored satellite observation dataset corresponding to the target area based on the current satellite observation dataset of the target area by the multi-channel generator; The repaired satellite observation dataset and the corrected historical satellite observation dataset corresponding to the target area generate a determination result through the multi-channel discriminator, wherein the determination result is used to describe the difference between the repaired satellite observation dataset and the corrected historical satellite observation dataset; Based on the repaired satellite observation data set and the judgment result, the generation collaboration loss value and reward value corresponding to the multi-channel generator and the discrimination collaboration loss value corresponding to the multi-channel discriminator are determined respectively, so as to perform progressive collaborative training on the multi-channel generator and the multi-channel discriminator.

5. The satellite observation data repair method based on generative collaborative adversarial network according to claim 4 is characterized in that: The multi-channel generator includes a small cloud block generator, a large cloud block generator and a special cloud block generator; Generating a repaired satellite observation dataset corresponding to the target area based on the current satellite observation dataset of the target area by the multi-channel generator includes: Using the small cloud block generator, local detail repair is performed based on the current satellite observation data set of the target area to obtain a small cloud block repair result; And, performing global texture restoration based on the current satellite observation data set of the target area through the large cloud block generator to obtain a large cloud block restoration result; And, by means of the special cloud block generator, complex area repair is performed based on the current satellite observation data set of the target area to obtain a special cloud block repair result; The small cloud block repair result, the large cloud block repair result, and the special cloud block repair result are weightedly fused to obtain a repaired satellite observation data set corresponding to the target area.

6. The satellite observation data repair method based on generative collaborative adversarial network according to claim 4 is characterized in that: The multi-channel discriminator includes a spatial discriminator and a global discriminator based on a collaborative mechanism; The multi-channel discriminator generates a determination result from the repaired satellite observation data set and the corrected historical satellite observation data set corresponding to the target area, including: Extracting a local cropped image corresponding to the cloud cover sub-area from the repaired satellite observation data set, and determining, by the spatial discriminator, based on the corrected historical satellite observation data set, that the local cropped image belongs to a first expected value of a real image; Determining, by the global discriminator, based on the corrected historical satellite observation dataset, that the restored satellite observation dataset belongs to a second expected value of a real image; A determination result is determined according to the first expected value and the second expected value.

7. The satellite observation data repair method based on generative collaborative adversarial network according to claim 4 is characterized in that: The generated collaborative loss value is obtained by weighted summing up the reconstruction error sub-loss value, the global consistency sub-loss value, the adversarial sub-loss value, the physical consistency sub-loss value, and the multi-scale sub-loss value; wherein the weight corresponding to the same sub-loss value is different in different stages, and the target-aware weight allocation mechanism is used in the local feature learning stage to dynamically adjust the weights of the reconstruction error sub-loss value and the adversarial sub-loss value; The reward value is obtained by weighted summing the texture sub-loss value and the radiation sub-loss value; The discriminant collaborative loss value is obtained by weighted summing the spatial sub-loss value and the global sub-loss value.

8. The satellite observation data repair method based on generative collaborative adversarial network according to claim 7 is characterized in that: The multi-channel generator and the multi-channel discriminator are progressively and collaboratively trained, comprising: In the local feature learning stage, the global consistency optimization stage and the multi-scale fusion stage, the multi-channel generator and the multi-channel discriminator are trained by the AdamW optimizer, and the training process is optimized by using the gradient clipping strategy and the periodic learning rate strategy.

9. A satellite observation data repair device based on generative collaborative adversarial network, characterized in that: include: A data acquisition module is used to acquire the current satellite observation data set and the historical satellite observation data set of the target area; A data correction module is used to extract a cloud-covered sub-region and a cloud-free sub-region from the target area, and based on the current satellite observation data subsets corresponding to the cloud-covered sub-region and the cloud-free sub-region, respectively, perform spatiotemporal registration and radiation correction on the historical satellite observation data subset corresponding to the cloud-covered sub-region in the cloud-free condition, so as to obtain a corrected historical satellite observation data set corresponding to the target area; A network training module is used to perform progressive collaborative training on a generative collaborative adversarial network using the current satellite observation data set and the corrected historical satellite observation data set of the target area, wherein the generative collaborative adversarial network includes a multi-channel generator and a multi-channel discriminator, and the trained multi-channel generator is used to perform data repair on the satellite observation data to be repaired in the target area, and the progressive collaborative training includes a local feature learning stage, a global consistency optimization stage, and a multi-scale fusion stage; wherein the local feature learning stage is training based on the current satellite observation data subset corresponding to the cloud coverage sub-area; the global consistency optimization stage is training based on the current satellite observation data set corresponding to the target area of ​​the same scale; and the multi-scale fusion stage is training based on the current satellite observation data set corresponding to the target area of ​​different scales.

10. An electronic device, characterized in that: The method comprises a processor and a memory, wherein the memory stores computer executable instructions that can be executed by the processor, and the processor executes the computer executable instructions to implement the method according to any one of claims 1 to 8.

Images