Image enhancement method and device for high-orbit satellite cloud layer image, and storage medium

Abstract

The invention discloses an image enhancement method and device for a cloud layer image of a high-orbit satellite and a storage medium, and the method comprises the steps: collecting a first cloud layer image corresponding to a first target region, and receiving second cloud layer images which are transmitted by a plurality of low-orbit satellites and respectively correspond to each second target region; performing image splicing on the second cloud layer images, and generating a first input image; determining a plurality of image areas corresponding to the second target areas in the first cloud layer image, and segmenting and combining the first cloud layer image based on the image areas to generate a second input image; inputting the first input image, the second input image and the first cloud layer image into a pre-trained discriminant generation network, and generating a first image quality analysis result corresponding to the first input image and the second input image by using the discriminant generation network; and outputting a standard cloud layer image corresponding to the first target area by using a discriminant generation network according to the first image quality analysis result.

Description

Technical Field

[0001] This application relates to the field of image processing technology, and in particular to an image enhancement method, apparatus and storage medium for high-orbit satellite cloud images. Background Art

[0002] Currently, because clouds are a visual indicator of atmospheric movement, their morphology, distribution, and evolution can directly reflect the changing trends of weather systems. Therefore, satellite imagery is frequently used to monitor meteorological changes. For example, low-level cumulus clouds may indicate localized rainfall. Furthermore, long-term cloud cover data can be used to analyze global trends in cloud cover and reveal climate feedback mechanisms.

[0003] In existing technologies, cloud images can be acquired using either high-orbit or low-orbit satellites. However, since high-orbit satellites are 36,000 kilometers above the ground, while clouds are 1 to 15 kilometers above the ground, although cloud images acquired by high-orbit satellites can cover a larger area, the image quality is far lower than that of cloud images acquired by low-orbit satellites due to the limitations of their orbital altitude. Therefore, it is impossible to accurately monitor weather changes based solely on cloud images from high-orbit satellites.

[0004] While low-Earth orbit (LEO) satellites produce high-quality cloud images, their altitude ranges from 200 to 2000 kilometers. Consequently, the cloud images captured by LEO satellites cover a smaller area compared to those captured by high-Earth orbit (HEO) satellites, making them insufficient to provide adequate data support for monitoring weather changes.

[0005] There is currently no effective solution to the technical problem in the existing technology that the image quality of cloud images acquired by high-orbit satellites is relatively poor compared to that acquired by low-orbit satellites, thus making it impossible to accurately monitor weather changes based solely on cloud images from high-orbit satellites. Summary of the Invention

[0006] The embodiments of this disclosure provide an image enhancement method, apparatus, and storage medium for cloud images from high-orbit satellites, to at least solve the technical problem in the prior art that the image quality of cloud images acquired by high-orbit satellites is relatively poor compared to that acquired by low-orbit satellites, thus making it impossible to accurately monitor weather changes based solely on cloud images from high-orbit satellites.

[0007] According to one aspect of the present disclosure, an image enhancement method for high-orbit satellite cloud images is provided, comprising: acquiring a first cloud image corresponding to a first target region, and receiving second cloud images transmitted by multiple low-orbit satellites corresponding to various second target regions, wherein each second target region is within the coverage area of ​​the first target region; stitching together the various second cloud images to generate a first input image; determining multiple image regions in the first cloud image corresponding to each second target region, and segmenting and combining the first cloud image based on each image region to generate a second input image; inputting the first input image, the second input image, and the first cloud image into a pre-trained discriminant generation network, and using the discriminant generation network to generate a first image quality analysis result corresponding to the first input image and the second input image; and using the discriminant generation network and based on the first image quality analysis result to output a standard cloud image corresponding to the first target region.

[0008] According to another aspect of the present disclosure, a storage medium is also provided, the storage medium including a stored program, wherein, when the program is executed, a processor performs any of the methods described above.

[0009] According to another aspect of the present disclosure, an image enhancement device for high-orbit satellite cloud images is also provided, comprising: a cloud image receiving module, configured to acquire a first cloud image corresponding to a first target region, and receive second cloud images transmitted by multiple low-orbit satellites corresponding to each second target region, wherein each second target region is within the coverage area of ​​the first target region; an image stitching module, configured to stitch the various second cloud images together and generate a first input image; an input image generation module, configured to determine multiple image regions in the first cloud image corresponding to each second target region, and to segment and combine the first cloud image based on each image region to generate a second input image; an image analysis module, configured to input the first input image, the second input image, and the first cloud image into a pre-trained discriminant generation network, and to generate a first image quality analysis result corresponding to the first input image and the second input image using the discriminant generation network; and a standard cloud image output module, configured to output a standard cloud image corresponding to the first target region using the discriminant generation network and based on the first image quality analysis result.

[0010] According to another aspect of the present disclosure, an image enhancement device for high-orbit satellite cloud images is also provided, comprising: a processor; and a memory connected to the processor, configured to provide the processor with instructions to process the following steps: acquiring a first cloud image corresponding to a first target region, and receiving second cloud images transmitted by multiple low-orbit satellites corresponding to each second target region, wherein each second target region is within the coverage area of ​​the first target region; stitching together the second cloud images to generate a first input image; determining multiple image regions in the first cloud image corresponding to each second target region, and segmenting and combining the first cloud image based on each image region to generate a second input image; inputting the first input image, the second input image, and the first cloud image into a pre-trained discriminant generation network, and using the discriminant generation network to generate a first image quality analysis result corresponding to the first input image and the second input image; and using the discriminant generation network and based on the first image quality analysis result to output a standard cloud image corresponding to the first target region.

[0011] This application provides an image enhancement method for cloud images from high-orbit satellites. First, the high-orbit satellite acquires a first cloud image corresponding to a first target region and receives second cloud images corresponding to various second target regions transmitted by multiple low-orbit satellites. Then, the high-orbit satellite stitches together the various second cloud images to generate a first input image. Further, the high-orbit satellite identifies multiple image regions within the first cloud image that correspond to each of the second target regions, and segments and combines the first cloud image based on these image regions to generate a second input image. The high-orbit satellite then inputs the first input image, the second input image, and the first cloud image into a pre-trained discriminative generation network (DGRN), which generates a first image quality analysis result corresponding to the first and second input images. Finally, the high-orbit satellite uses the DGRN and the first image quality analysis result to output a standard cloud image corresponding to the first target region.

[0012] As described above, in order to improve the quality of the first cloud image acquired by a high-orbit satellite, this application utilizes the characteristic that the image quality of cloud images acquired by multiple low-orbit satellites is much higher than that of cloud images acquired by high-orbit satellites. Multiple second cloud images corresponding to the first cloud image are acquired respectively. Then, the first input image corresponding to the multiple second cloud images and the second input image corresponding to the first cloud image are input into a pre-trained discriminative generation network. The discriminative generation network can then generate an enhanced cloud image corresponding to the first cloud image based on the first image quality analysis results of the first and second input images. Subsequently, based on the first image quality analysis results of the generated enhanced cloud image and the first input image, the image quality of the enhanced cloud image is continuously enhanced. This process is repeated until a standard cloud image is finally generated.

[0013] The aforementioned standard cloud images not only cover a large area but also possess image quality consistent with the second cloud images acquired by low-orbit satellites. This achieves the technical effect of obtaining high-quality cloud images with broad coverage, and enabling accurate monitoring of weather changes based solely on the first cloud images from high-orbit satellites.

[0014] This solves the technical problem in existing technologies where the image quality of cloud images collected by high-orbit satellites is relatively poor compared to that of cloud images collected by low-orbit satellites, making it impossible to accurately monitor weather changes solely based on cloud images from high-orbit satellites. Attached Figure Description

[0015] The accompanying drawings, which are included to provide a further understanding of this disclosure and form part of this application, illustrate exemplary embodiments of this disclosure and are used to explain this disclosure, but do not constitute an undue limitation of this disclosure. In the drawings:

[0016] Figure 1 This is a schematic diagram of cloud images collected by a high-orbit satellite and multiple low-orbit satellites according to Embodiment 1 of this application;

[0017] Figure 2 The hardware architecture diagram of the high-orbit satellite and various low-orbit satellites described in Embodiment 1 of this application;

[0018] Figure 3 This is a flowchart of the image enhancement method for high-orbit satellite cloud images according to Embodiment 1 of this application;

[0019] Figure 4 This is a schematic diagram of the various second cloud layer images and the first input image according to Embodiment 1 of this application;

[0020] Figure 5This is a schematic diagram of the first cloud image and the second input image according to Embodiment 1 of this application;

[0021] Figure 6 This is a schematic diagram illustrating the connection relationship between the image stitching module, the image segmentation and combination module, and the discrimination and generation network according to Embodiment 1 of this application;

[0022] Figure 7 This is a schematic diagram of an apparatus for an image enhancement method for high-orbit satellite cloud images according to Embodiment 2 of this application; and

[0023] Figure 8 This is a schematic diagram of an apparatus for an image enhancement method for high-orbit satellite cloud images according to Embodiment 3 of this application. Detailed Implementation

[0024] To enable those skilled in the art to better understand the technical solutions of this disclosure, the technical solutions of the embodiments of this disclosure will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are merely some embodiments of this disclosure, and not all embodiments. Based on the embodiments of this disclosure, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of this disclosure.

[0025] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this disclosure are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this disclosure described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0026] Example 1

[0027] According to this embodiment, an image enhancement method for high-orbit satellite cloud images is provided. It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions. Although a logical order is shown in the flowchart, in some cases, the steps shown or described may be executed in a different order than that shown here.

[0028] Figure 1This diagram illustrates the acquisition of cloud images by a high-orbit satellite and multiple low-orbit satellites according to this embodiment. (Reference) Figure 1 As shown, the system includes a high-orbit satellite 10 and multiple low-orbit satellites 201-20n. Furthermore, the coverage area of ​​the high-orbit satellite 10 is greater than the coverage area of ​​the multiple low-orbit satellites 201-20n, thus the coverage area of ​​the first cloud layer image acquired by the high-orbit satellite 10 is greater than the coverage area of ​​the second cloud layer images acquired by each of the second low-orbit satellites 201-20n.

[0029] In addition, referring to the above Figure 1 As shown, the coverage area of ​​the second cloud images acquired by each low-Earth orbit satellite 201–20n is within the coverage area of ​​the first cloud image acquired by high-Earth orbit satellite 10. That is, the sum of the coverage areas of the second cloud images acquired by multiple low-Earth orbit satellites 201–20n is less than or equal to the coverage area of ​​the first cloud image acquired by high-Earth orbit satellite 10.

[0030] Furthermore, the first cloud image acquired by the high-orbit satellite 10 and the second cloud image acquired by each low-orbit satellite 201-20n in this application can be, for example, an infrared image or a visible light image, which will not be elaborated here.

[0031] Figure 2 Further shown Figure 1 A schematic diagram of the hardware architecture of the medium-high orbit satellite 10 and the various low-Earth orbit satellites 201-20n. (Reference) Figure 2 As shown, the high-orbit satellite 10 and each low-orbit satellite 201-20n include an integrated electronic system, which includes a processor, a memory, a bus management module, and a communication interface. The memory is connected to the processor, allowing the processor to access the memory, read program instructions stored in the memory, read data from the memory, or write data to the memory. The bus management module is connected to the processor and also to a bus such as a CAN bus. Thus, the processor can communicate with onboard peripherals connected to the bus through the bus managed by the bus management module. Furthermore, the processor also communicates with devices such as cameras, star sensors, telemetry and control transponders, and data transmission equipment via the communication interface. Those skilled in the art will understand that... Figure 2 The structure shown is for illustrative purposes only and does not limit the structure of the aforementioned electronic device. For example, a satellite system may also include... Figure 2 The more or fewer components shown, or having the same Figure 2 The different configurations shown.

[0032] It should be noted that, Figure 2One or more processors and / or other data processing circuits shown herein may generally be referred to as "data processing circuitry". This data processing circuitry may be embodied, in whole or in part, in software, hardware, firmware, or any other combination thereof. Furthermore, the data processing circuitry may be a single, independent processing module, or may be integrated, in whole or in part, into any other element in a computing device. As involved in embodiments of this disclosure, the data processing circuitry serves as processor control (e.g., selection of a variable resistor termination path connected to an interface).

[0033] Figure 2 The memory shown can be used to store software programs and modules for application software, such as the program instructions / data storage device corresponding to the image enhancement method for high-orbit satellite cloud images in this embodiment of the present disclosure. The processor executes various functional applications and data processing by running the software programs and modules stored in the memory, thereby implementing the image enhancement method for high-orbit satellite cloud images described above. The memory may include high-speed random access memory, and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory.

[0034] It should be noted here that, in some optional embodiments, the above... Figure 2 The device shown may include hardware elements (including circuitry), software elements (including computer code stored on a computer-readable medium), or a combination of both hardware and software elements. It should be noted that... Figure 2 This is only one instance of a specific particular instance, and is intended to illustrate the types of components that may exist in the aforementioned devices.

[0035] Under the aforementioned operating environment, according to the first aspect of this embodiment, an image enhancement method for high-orbit satellite cloud images is provided, the method comprising: Figure 2 The high-orbit satellite 10 shown is realized. Figure 3 A flowchart illustrating the method is shown below. (Refer to...) Figure 3 As shown, the method includes:

[0036] S302: Acquire a first cloud image corresponding to the first target area, and receive second cloud images corresponding to each second target area sent by multiple low-orbit satellites, wherein each second target area is within the coverage area of ​​the first target area;

[0037] S304: Perform image stitching on each of the second cloud layer images and generate the first input image;

[0038] S306: Determine multiple image regions in the first cloud image that correspond to each of the second target regions, and segment and combine the first cloud image based on each image region to generate a second input image;

[0039] S308: Input the first input image, the second input image, and the first cloud image into a pre-trained discriminative generation network, and use the discriminative generation network to generate a first image quality analysis result corresponding to the first input image and the second input image; and

[0040] S310: Using a discriminative generative network and based on the first image quality analysis results, output a standard cloud image corresponding to the first target region.

[0041] Specifically, firstly, the high-orbit satellite 10 acquires a first cloud image corresponding to the first target area. Then, the high-orbit satellite 10 sends image acquisition commands to each of the second low-orbit satellites 201-20n within the coverage area of ​​the first target area. Each low-orbit satellite 201-20n receives and responds to the image acquisition commands sent by the high-orbit satellite 10, acquiring second cloud images corresponding to each second target area. Afterwards, each low-orbit satellite 201-20n transmits its second cloud images back to the high-orbit satellite 10, enabling the high-orbit satellite 10 to receive the second cloud images corresponding to each second target area (S302). Since each second target area is within the coverage area of ​​the first target area, the second cloud images acquired by each low-orbit satellite 201-20n are actually equivalent to a portion of the first cloud image. However, because the high-orbit satellite 10 is farther from the clouds, while the low-orbit satellites 201-20n are closer, the first cloud image, although covering a larger area, has a lower image quality compared to the second cloud images. Furthermore, in the embodiments of this application, the first cloud image acquired by the high-orbit satellite 10 may be, for example, an infrared image, and the second cloud image acquired by each low-orbit satellite 201-20n may also be, for example, an infrared image.

[0042] Subsequently, upon receiving the second cloud images corresponding to each of the second target areas, the high-orbit satellite 10 uses an image stitching module to stitch the second cloud images corresponding to each of the second target areas together and generate a first input image (S304). For example, Figure 4 This is a schematic diagram of various second cloud layer images and a first input image according to embodiments of this application. (Reference) Figure 4 As shown, the high-orbit satellite 10 receives signals from each of the second target regions s1 to s2. n The corresponding second cloud layer images l1~l n Then, the high-orbit satellite 10 uses an image stitching module to stitch together the images of each second cloud layer l1~l n By splicing, a result can be generated as follows: Figure 4 The first input image l0 is shown.

[0043] Furthermore, the high-orbit satellite 10 sends position coordinate request commands to each low-orbit satellite 201-20n. In response to the request commands sent by the high-orbit satellite 10, each low-orbit satellite 201-20n sends coordinates to the high-orbit satellite 10 corresponding to its position coordinates with the corresponding second target regions s1-s2. n The corresponding position coordinates. Furthermore, Figure 5 A schematic diagram showing a first cloud image and a second input image is provided. (Reference) Figure 5 As shown, the high-orbit satellite 10 will display images of the first cloud layer and the various second target regions s1 to s2. n The corresponding position coordinates are input to the image segmentation and combination module, which then performs the segmentation and combination based on the coordinates of each second target region s1 to s2. n The corresponding position coordinates are used to determine the relationship between the first cloud layer image H1 and each of the second target regions s1 to s2. n The corresponding image regions h1~h n Then the image segmentation and combination module performs the segmentation based on the determined image regions h1 to h2. n The first cloud layer image H1 is segmented to generate images l1~l1 corresponding to each of the second cloud layers. n The corresponding multiple image blocks p1 to p n Finally, the image segmentation and combination module combines multiple image blocks p1 to p2. n Combine the data and generate the second input image p0 (S306). This will be described in detail later, so it will not be repeated here.

[0044] Furthermore, Figure 6 This is a schematic diagram illustrating the connection relationship between the image stitching module, the image segmentation and combination module, and the discrimination and generation network according to embodiments of this application. (Reference) Figure 6 As shown, the high-orbit satellite 10 inputs a first input image, a second input image, and a first cloud image into a pre-trained discriminative generation network. The discriminator in the network then analyzes whether the image quality of the first and second input images is consistent and generates a corresponding first image quality analysis result (S308). Image quality may include, for example, resolution, noise, contrast, and sharpness. The first image quality analysis result indicates whether the image quality of the first input image is consistent with the image quality of the second input image, or whether the image quality of the first input image is inconsistent with the image quality of the second input image.

[0045] Furthermore, if the first image quality analysis result indicates that the image quality of the first input image and the second input image are inconsistent, the generator in the discriminative generation network enhances the first cloud image based on the first image quality analysis result and generates an enhanced cloud image corresponding to the first cloud image. The generator then sends the enhanced cloud image to the image segmentation and combination module, which segments and combines the enhanced cloud image to generate a third input image corresponding to the enhanced cloud image. Further, the high-orbit satellite 10 sends the third input image to the discriminator, which analyzes whether the image quality of the third input image is consistent with the first input image. If the first image quality analysis result indicates inconsistency, the generator enhances the enhanced cloud image based on the first image quality analysis result. This process is repeated until the second image quality analysis result generated by the discriminator shows that the image quality of the first input image and the enhanced cloud image is consistent. At this point, the generator uses the enhanced cloud image as the standard cloud image and outputs the standard cloud image (S310). The above will be described in detail later, so it will not be repeated here.

[0046] As described in the background section, cloud images can be acquired using both high-orbit and low-orbit satellites. However, because high-orbit satellites are 36,000 kilometers above the ground, while clouds are only 1 to 15 kilometers above the ground, the image quality of cloud images acquired by high-orbit satellites is far lower than that of cloud images acquired by low-orbit satellites due to the limitations of their orbital altitude. Therefore, it is impossible to accurately monitor weather changes based solely on cloud images from high-orbit satellites.

[0047] While low-Earth orbit (LEO) satellites produce high-quality cloud images, their altitude ranges from 200 to 2000 kilometers. Consequently, the cloud images captured by LEO satellites cover a smaller area compared to those captured by high-Earth orbit (HEO) satellites, making them insufficient to provide adequate data support for monitoring weather changes.

[0048] In view of this, this application provides an image enhancement method for cloud images from high-orbit satellites. Referring to the above description, to improve the quality of the first cloud image acquired by the high-orbit satellite, this application utilizes the characteristic that the image quality of cloud images acquired by multiple low-orbit satellites is much higher than that of cloud images acquired by high-orbit satellites, and acquires multiple second cloud images corresponding to the first cloud image. Then, the first input image corresponding to the multiple second cloud images and the second input image corresponding to the first cloud image are input into a pre-trained discriminative generation network. The discriminative generation network can then generate an enhanced cloud image corresponding to the first cloud image based on the first image quality analysis results of the first and second input images. Then, based on the first image quality analysis results of the generated enhanced cloud image and the first input image, the image quality of the enhanced cloud image is continuously enhanced. This process is repeated until a standard cloud image is finally generated.

[0049] The aforementioned standard cloud images not only cover a large area but also possess image quality consistent with the second cloud images acquired by low-orbit satellites. This achieves the technical effect of obtaining high-quality cloud images with broad coverage, and enabling accurate monitoring of weather changes based solely on the first cloud images from high-orbit satellites.

[0050] This solves the technical problem in existing technologies where the image quality of cloud images collected by high-orbit satellites is relatively poor compared to that of cloud images collected by low-orbit satellites, making it impossible to accurately monitor weather changes solely based on cloud images from high-orbit satellites.

[0051] Optionally, the operation of determining multiple image regions in the first cloud image corresponding to each second target region, and segmenting and combining the first cloud image based on each image region to generate a second input image includes: receiving position coordinates sent by each low-orbit satellite corresponding to each second target region; determining the image regions in the first cloud image corresponding to each second target region based on each position coordinate; segmenting the first cloud image according to the determined image regions to generate multiple image blocks corresponding to each second cloud image; and combining the multiple image blocks to generate a second input image.

[0052] Specifically, firstly, upon receiving second cloud images corresponding to each second target area transmitted by each low-orbit satellite 201-20n, the high-orbit satellite 10 further transmits a position request command to each low-orbit satellite 201-20n to request the position coordinates corresponding to each second target area. These position coordinates, for example, correspond to a geocentric coordinate system.

[0053] For example, each LEO satellite 201-20n can calculate its own position coordinates at the time of image capture based on its own ephemeris information. Then, each LEO satellite 201-20n converts the coordinates of each pixel in the acquired second cloud layer images into coordinates in the camera coordinate system corresponding to each LEO satellite 201-20n through a transformation relationship. The camera coordinate system indicates the coordinate system of the camera used to capture each second cloud layer image. Further, each LEO satellite 201-20n converts the coordinates of each second cloud layer image in the camera coordinate system into coordinates in its own body coordinate system through a transformation relationship. Finally, each LEO satellite 201-20n converts the coordinates of each second cloud layer image in its body coordinate system into position coordinates in the geocentric coordinate system through a transformation relationship. Thus, each LEO satellite 201-20n can determine the position coordinates corresponding to each second target area.

[0054] After determining the position coordinates corresponding to each second target area by each low-orbit satellite 201-20n, the position coordinates corresponding to each second target area are sent to the high-orbit satellite 10.

[0055] Furthermore, the high-orbit satellite 10 determines the image regions corresponding to each second target region in the first cloud image based on the coordinates of each position. For example, firstly, the high-orbit satellite 10 converts the position coordinates corresponding to each second target region into coordinates in the body coordinate system through a transformation relationship. Then, the high-orbit satellite 10 converts the coordinates corresponding to each second target region in the body coordinate system into coordinates corresponding to each second target region in the camera coordinate system through a transformation relationship. The camera coordinate system indicates the coordinate system of the camera used to capture the first cloud image. Further, the high-orbit satellite 10 converts the coordinates corresponding to each second target region in the camera coordinate system into coordinates corresponding to each second target region in the pixel coordinate system through a transformation relationship. Thus, the high-orbit satellite 10 inputs the determined coordinates and the first cloud image together into the image segmentation and combination module using the aforementioned coordinates of each second target region in the pixel coordinate system of the high-orbit satellite 10. The image segmentation and combination module can then determine the image regions corresponding to each second target region in the first cloud image.

[0056] Furthermore, after the image segmentation and combination module determines the image regions in the first cloud image that correspond to each of the second target regions, the first cloud image is segmented, and image blocks corresponding to each image region are generated. Then, the image segmentation and combination module combines the multiple image blocks corresponding to each image region to generate the second input image.

[0057] Optionally, the discriminative generation network includes a discriminator, and the operation of inputting the first input image, the second input image, and the first cloud image into the pre-trained discriminative generation network and generating a first image quality analysis result corresponding to the first input image and the second input image using the discriminative generation network includes: inputting the first input image and the second input image into the discriminator, and using the discriminator to output the first image quality analysis result, wherein the first image quality analysis result includes whether the image quality between the first input image and the second input image is consistent, or whether the image quality between the first input image and the second input image is inconsistent.

[0058] Specifically, refer to Figure 6 As shown, the discriminant-generative network includes a discriminator, which comprises a feature extraction layer, a feature difference analysis layer, and an output layer. The feature extraction layer includes dual-channel convolutional layers with equal weights, used to process the first input image and the second input image respectively, generating a first feature map corresponding to the first input image and a second feature map corresponding to the second input image. The feature difference analysis layer, for example, can be an MLP (Multilayer Perceptron), used to calculate the difference between the first and second feature maps, and analyzes the difference through a fully connected layer to output a feature vector. The output layer uses a logistic regression function and, based on the feature vector, outputs a value between 0 and 1 as the image quality analysis result.

[0059] In other words, when the image stitching module inputs the first input image and the image segmentation and combination module inputs the second input image to the discriminator, the discriminator can ultimately output a first image quality analysis result. This first image quality analysis result indicates whether the image quality of the first input image and the second input image is consistent, or whether their image quality is inconsistent.

[0060] Furthermore, in the embodiments of this application, the image quality of the second input image is considered consistent with that of the first input image only when the first image quality analysis result is greater than a preset threshold. For example, the preset threshold is 0.9. Thus, when the first image quality analysis result is greater than 0.9, it indicates that the image quality of the second input image is consistent with that of the first input image. Conversely, when the first image quality analysis result is less than or equal to 0.9, it indicates that the image quality of the second input image is inconsistent with that of the first input image.

[0061] Optionally, the discriminative generation network includes a generator, and the operation of using the discriminative generation network and based on the first image quality analysis result to output a standard cloud image corresponding to the first target region includes: multiplying the first cloud image and the first image quality analysis result and inputting the result into the generator; and if the first image quality analysis result indicates that the image quality between the first input image and the second input image is inconsistent, the generator generates an enhanced cloud image based on the first cloud image; segmenting and combining the enhanced cloud image based on each image region to generate a third input image; inputting the third input image into the discriminator and using the discriminator to output a second image quality analysis result, wherein the second image quality analysis result includes whether the image quality between the first input image and the enhanced cloud image is consistent, or whether the image quality between the first input image and the enhanced cloud image is inconsistent; and inputting the second image quality analysis result into the generator, and if the second image quality analysis result indicates that the image quality between the first input image and the enhanced cloud image is consistent, using the enhanced cloud image as the standard cloud image.

[0062] Specifically, the discriminative generator network includes a generator. The generator includes encoder 1, decoder 1, encoder 2, and decoder 2. The generator has two input branches: a first input branch for inputting a first cloud image or an enhanced cloud image, and a second input branch for inputting the first cloud image or the enhanced cloud image multiplied by the image quality analysis result.

[0063] In the scenario where the first image quality analysis result generated by the discriminator indicates an inconsistency in image quality between the first input image and the second input image, the generator's first input branch receives the first cloud image. The generator then encodes the first cloud image using encoder 1 and decodes it using decoder 1 to generate a reconstructed image. Encoder 1 and decoder 1 can employ architectures commonly used in existing technologies. For example, encoder 1 can consist of convolutional layers and pooling layers, and decoder 1 can consist of deconvolutional layers and upsampling layers corresponding to encoder 1.

[0064] Simultaneously, the first cloud image, multiplied by the first image quality analysis result, is input into the generator through the second input branch and fused with the reconstructed image output by decoder 1. In this embodiment, this fusion can be achieved, for example, by concatenation. The fused image is input to encoder 2 and decoder 2, thereby decoder 2 outputs an enhanced cloud image of the first cloud image. Encoder 2 and decoder 2 can adopt architectures commonly used in existing technologies. Further details are omitted here.

[0065] Therefore, when the first image quality analysis result generated by the discriminator indicates that the image quality between the first input image and the second input image is inconsistent, the generator generates an enhanced cloud image corresponding to the first cloud image based on the above method.

[0066] For reference later Figure 6 As shown, the generator generates an enhanced cloud image and sends it to the image segmentation and combination module. The image segmentation and combination module segments and combines the enhanced cloud image based on each image region to generate a third input image. The image segmentation and combination module then sends the third input image to the discriminator, which analyzes whether the image quality of the third input image is consistent with that of the first input image and generates a corresponding second image quality analysis result.

[0067] Furthermore, the discriminator sends the second image quality analysis result to the generator, and when the second image quality analysis result indicates that the image quality between the first input image and the third input image is consistent, the generation module uses the pre-stored enhanced cloud image as the standard cloud image and outputs the standard cloud image.

[0068] Furthermore, when the second image quality analysis result indicates that the image quality between the first input image and the third input image is inconsistent, the first input branch of the generator receives the enhanced cloud image from the previous iteration round. Thus, the generator encodes the enhanced cloud image from the previous iteration round using encoder 1 and then decodes it using decoder 1 to generate a reconstructed image.

[0069] Simultaneously, the enhanced cloud image from the previous iteration, multiplied by the second image quality analysis result, is input into the generator via the second input branch and fused with the reconstructed image output by decoder 1. In this embodiment, this fusion can be achieved, for example, by concatenation. The fused image is input to encoder 2 and decoder 2, so that decoder 2 outputs an enhanced cloud image corresponding to the enhanced cloud image from the previous iteration. Encoder 2 and decoder 2 can adopt a commonly used architecture in the prior art. Further details are omitted here. This process is repeated until the second image quality analysis result indicates that the image quality of the third input image at the current iteration number is consistent with that of the first input image. At this point, the generation module uses the enhanced cloud image corresponding to the current iteration number as the standard cloud image and outputs the standard cloud image.

[0070] Optionally, the method further includes: pre-training the discriminative generation network, wherein the operation of pre-training the discriminative generation network includes: crawling multiple first input image samples corresponding to multiple first sample regions taken by high-orbit satellites, and multiple second input image samples corresponding to multiple first sample regions taken by multiple low-orbit satellites; crawling standard cloud image samples corresponding to each first input image sample, wherein the image quality of the standard cloud image samples is consistent with the image quality of the second input image samples; and constructing the discriminative generation network and training the discriminative generation network using the multiple first input image samples, the multiple second input image samples, and the standard cloud image samples.

[0071] Specifically, before applying the discriminative generation network, it needs to be trained. First, the high-orbit satellite 10 crawls multiple first sample regions b1 to b2 from the network. m The corresponding multiple first input image samples c1~c m And images taken by various low-orbit satellites at 201-20m and multiple first sample regions b1-b m The corresponding multiple second input image samples d1~d m For example, low-Earth orbit satellite 201 acquires a second input image sample d1 corresponding to the first sample region b1, low-Earth orbit satellite 202 acquires a second input image sample d2 corresponding to the first sample region b2, low-Earth orbit satellite 203 acquires a second input image sample d3 corresponding to the first sample region b3, ..., low-Earth orbit satellite 20m acquires a second input image sample d3 corresponding to the first sample region b1. m The corresponding second input image sample d m It is worth noting that the coverage of each first input image sample collected by the high-orbit satellite 10 is the same as that of the corresponding second input image samples collected by each low-orbit satellite 201-20n, but the image quality is different.

[0072] Then, the high-orbit satellite 10 crawls and extracts each of the first input image samples c1 to c2. m The corresponding standard cloud image samples f1 to f2 m Among them, standard cloud image samples f1 to f2 are... m For example, with the second input image samples d1 to d2 m The image quality is consistent.

[0073] Furthermore, the high-orbit satellite 10 constructs a discriminative-generator network (i.e., a discriminator and a generator), and processes multiple first input image samples c1 to c2. m Multiple second input image samples d1~d m and standard cloud image samples f1 to f m The input is fed into the discriminant generation network, thereby training the discriminant generation network.

[0074] Thus, according to the first aspect of this embodiment, the technical effect of accurately monitoring weather changes based solely on first cloud images from high-orbit satellites is achieved.

[0075] In addition, refer to Figure 1 As shown, according to a second aspect of this embodiment, a storage medium is provided. The storage medium includes a stored program, wherein, when the program is executed, a processor performs any of the methods described above.

[0076] Thus, according to this embodiment, the technical effect of accurately monitoring weather changes based solely on the first cloud layer image from a high-orbit satellite is achieved.

[0077] It should be noted that for the aforementioned method embodiments, for simplicity of description, they are all expressed as a series of action combinations. However, those skilled in the art should be aware that the present invention is not limited by the order of the actions described, because according to the present invention, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in this specification are all preferred embodiments, and the actions and modules involved are not necessarily required by the present invention.

[0078] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods according to the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions to cause a terminal device (which may be a mobile phone, computer, server, or network device, etc.) to execute the methods described in the various embodiments of the present invention.

[0079] Example 2

[0080] Figure 7 An image enhancement apparatus 700 for high-orbit satellite cloud images according to this embodiment is shown, which corresponds to the method described according to Embodiment 1. Reference Figure 7As shown, the device 700 includes: a cloud image receiving module 710, used to acquire a first cloud image corresponding to a first target area, and receive second cloud images corresponding to each second target area transmitted by multiple low-orbit satellites, wherein each second target area is within the coverage area of ​​the first target area; an image stitching module 720, used to stitch together the various second cloud images and generate a first input image; an input image generation module 730, used to determine multiple image regions in the first cloud image corresponding to each second target area, and to segment and combine the first cloud image based on each image region to generate a second input image; an image analysis module 740, used to input the first input image, the second input image, and the first cloud image into a pre-trained discriminant generation network, and use the discriminant generation network to generate a first image quality analysis result corresponding to the first input image and the second input image; and a standard cloud image output module 750, used to use the discriminant generation network and based on the first image quality analysis result to output a standard cloud image corresponding to the first target area.

[0081] Optionally, the input image generation module 730 includes: a position coordinate receiving module for receiving position coordinates corresponding to each second target region sent by each low-orbit satellite; an image region determination module for determining image regions in the first cloud image corresponding to each second target region based on each position coordinate; an image patch generation module for segmenting the first cloud image according to the determined image regions to generate multiple image patches corresponding to each second cloud image; and an input image generation submodule for combining multiple image patches to generate a second input image.

[0082] Optionally, the discriminant generation network includes a discriminator, and the image analysis module 740 includes: a quality analysis result output module, used to input a first input image and a second input image to the discriminator, and use the discriminator to output a first image quality analysis result, wherein the first image quality analysis result includes whether the image quality between the first input image and the second input image is consistent, or whether the image quality between the first input image and the second input image is inconsistent.

[0083] Optionally, the discriminative generation network includes a generator, and the standard cloud image output module 750 includes: an enhanced cloud image generation module, used to input a first cloud image and a first image quality analysis result to the generator, and, when the first image quality analysis result indicates that the image quality between the first input image and the second input image is inconsistent, the generator generates an enhanced cloud image based on the first cloud image; a third input image generation module, used to segment and combine the enhanced cloud image based on each image region to generate a third input image, input the third input image to the discriminator, and use the discriminator to output a second image quality analysis result, wherein the second image quality analysis result includes whether the image quality between the first input image and the third input image is consistent, or whether the image quality between the first input image and the third input image is inconsistent; and a standard cloud image generation module, used to input the second image quality analysis result to the generator, and, when the second image quality analysis result indicates that the image quality between the first input image and the enhanced cloud image is consistent, use the enhanced cloud image as the standard cloud image.

[0084] Optionally, the device 700 further includes: a training module for pre-training the discriminative generation network, wherein the operation of pre-training the discriminative generation network includes: a first sample crawling module for crawling multiple first input image samples corresponding to multiple first sample regions taken by high-orbit satellites, and multiple second input image samples corresponding to multiple first sample regions taken by multiple low-orbit satellites; a second sample crawling module for crawling standard cloud image samples corresponding to each first input image sample, wherein the image quality of the standard cloud image samples is consistent with the image quality of the second input image samples; and a training submodule for constructing the discriminative generation network and training the discriminative generation network using multiple first input image samples, multiple second input image samples, and standard cloud image samples.

[0085] Thus, according to this embodiment, the technical effect of accurately monitoring weather changes based solely on the first cloud layer image from a high-orbit satellite is achieved.

[0086] Example 3

[0087] Figure 8 An image enhancement apparatus 800 for high-orbit satellite cloud images according to this embodiment is shown, which corresponds to the method described according to Embodiment 1. Reference Figure 8As shown, the device 800 includes: a processor 810; and a memory 820 connected to the processor 810, used to provide the processor 810 with instructions to process the following steps: acquiring a first cloud image corresponding to a first target area, and receiving second cloud images corresponding to each second target area respectively transmitted by multiple low-orbit satellites, wherein each second target area is within the coverage area of ​​the first target area; stitching together the second cloud images to generate a first input image; determining multiple image regions in the first cloud image corresponding to each second target area, and segmenting and combining the first cloud image based on each image region to generate a second input image; inputting the first input image, the second input image, and the first cloud image into a pre-trained discriminant generation network, using the discriminant generation network to generate a first image quality analysis result corresponding to the first input image and the second input image; and using the discriminant generation network and based on the first image quality analysis result to output a standard cloud image corresponding to the first target area.

[0088] Optionally, the operation of determining multiple image regions in the first cloud image corresponding to each second target region, and segmenting and combining the first cloud image based on each image region to generate a second input image includes: receiving position coordinates sent by each low-orbit satellite corresponding to each second target region; determining the image regions in the first cloud image corresponding to each second target region based on each position coordinate; segmenting the first cloud image according to the determined image regions to generate multiple image blocks corresponding to each second cloud image; and combining the multiple image blocks to generate a second input image.

[0089] Optionally, the discriminative generation network includes a discriminator, and the operation of inputting the first input image, the second input image, and the first cloud image into the pre-trained discriminative generation network and generating a first image quality analysis result corresponding to the first input image and the second input image using the discriminative generation network includes: inputting the first input image and the second input image into the discriminator, and using the discriminator to output the first image quality analysis result, wherein the first image quality analysis result includes whether the image quality between the first input image and the second input image is consistent, or whether the image quality between the first input image and the second input image is inconsistent.

[0090] Optionally, the discriminative generation network includes a generator, and the operation of using the discriminative generation network and based on the first image quality analysis result to output a standard cloud image corresponding to the first target region includes: inputting the first cloud image and the first image quality analysis result into the generator, and when the first image quality analysis result indicates that the image quality between the first input image and the second input image is inconsistent, the generator generates an enhanced cloud image based on the first cloud image; segmenting and combining the enhanced cloud image based on each image region to generate a third input image, inputting the third input image into the discriminator, and using the discriminator to output a second image quality analysis result, wherein the second image quality analysis result includes whether the image quality between the first input image and the third input image is consistent, or whether the image quality between the first input image and the third input image is inconsistent; and inputting the second image quality analysis result into the generator, and when the second image quality analysis result indicates that the image quality between the first input image and the enhanced cloud image is consistent, using the enhanced cloud image as the standard cloud image.

[0091] Optionally, the method further includes: pre-training the discriminative generation network, wherein the operation of pre-training the discriminative generation network includes: crawling multiple first input image samples corresponding to multiple first sample regions taken by high-orbit satellites, and multiple second input image samples corresponding to multiple first sample regions taken by multiple low-orbit satellites; crawling standard cloud image samples corresponding to each first input image sample, wherein the image quality of the standard cloud image samples is consistent with the image quality of the second input image samples; and constructing the discriminative generation network and training the discriminative generation network using the multiple first input image samples, the multiple second input image samples, and the standard cloud image samples.

[0092] Thus, according to this embodiment, the technical effect of accurately monitoring weather changes based solely on the first cloud layer image from a high-orbit satellite is achieved.

[0093] The sequence numbers of the above embodiments of the present invention are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0094] In the above embodiments of the present invention, the description of each embodiment has its own focus. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0095] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. The device embodiments described above are merely illustrative; for example, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the displayed or discussed mutual coupling, direct coupling, or communication connection may be through some interfaces; the indirect coupling or communication connection between units or modules may be electrical or other forms.

[0096] The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected to achieve the purpose of this embodiment according to actual needs.

[0097] In addition, the functional units in the various embodiments of the present invention may be integrated into a single processing unit, each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or software functional units.

[0098] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, read-only memory (ROM), random access memory (RAM), portable hard drives, magnetic disks, or optical disks.

[0099] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. An image enhancement method for cloud images from high-orbit satellites, characterized in that, include: Acquire a first cloud image corresponding to the first target area, and receive second cloud images corresponding to each second target area transmitted by multiple low-orbit satellites, wherein each second target area is within the coverage area of ​​the first target area; The images of each second cloud layer are stitched together to generate the first input image; Multiple image regions in the first cloud image that correspond to each of the second target regions are determined, and the first cloud image is segmented and combined based on each image region to generate a second input image; The first input image, the second input image, and the first cloud image are input into a pre-trained discriminative generation network, and the discriminative generation network is used to generate a first image quality analysis result corresponding to the first input image and the second input image; as well as Using the discriminative generation network and based on the first image quality analysis results, a standard cloud image corresponding to the first target region is output.

2. The method according to claim 1, characterized in that, The operation of determining multiple image regions in the first cloud image that correspond to each of the second target regions, and segmenting and combining the first cloud image based on each image region to generate a second input image includes: Receive the position coordinates corresponding to each of the second target areas transmitted by each of the low-orbit satellites; Based on the coordinates of each location, determine the image region in the first cloud image that corresponds to each of the second target regions; The first cloud image is segmented according to the determined image region to generate multiple image blocks corresponding to each of the second cloud images; and The multiple image blocks are combined to generate a second input image.

3. The method according to claim 2, characterized in that, The discriminative generation network includes a discriminator, and the operation of inputting the first input image, the second input image, and the first cloud image into the pre-trained discriminative generation network, and using the discriminative generation network to generate a first image quality analysis result corresponding to the first input image and the second input image, includes: The first input image and the second input image are input to the discriminator, and the discriminator outputs the first image quality analysis result, wherein the first image quality analysis result includes whether the image quality of the first input image and the second input image is consistent, or whether the image quality of the first input image and the second input image is inconsistent.

4. The method according to claim 3, characterized in that, The discriminative generation network includes a generator, and the operation of using the discriminative generation network and based on the first image quality analysis result to output a standard cloud image corresponding to the first target region includes: The generator inputs the first cloud image and the first image quality analysis result into the generator, and when the first image quality analysis result is used to indicate that the image quality between the first input image and the second input image is inconsistent, the generator generates an enhanced cloud image based on the first cloud image. The enhanced cloud image is segmented and combined based on the respective image regions to generate a third input image. This third input image is then input to the discriminator, which outputs a second image quality analysis result. This second image quality analysis result includes whether the image quality of the first input image and the third input image is consistent, or whether their image quality is inconsistent. The second image quality analysis result is input into the generator, and if the second image quality analysis result indicates that the image quality between the first input image and the enhanced cloud image is consistent, the enhanced cloud image is used as the standard cloud image.

5. The method according to claim 1, characterized in that, Also includes: The discriminant generation network is pre-trained, wherein the operation of pre-training the discriminant generation network includes: The system crawls multiple first input image samples corresponding to multiple first sample regions, which are captured by the high-orbit satellites, and multiple second input image samples corresponding to the multiple first sample regions, which are captured by the multiple low-orbit satellites. Crawling standard cloud image samples corresponding to each first input image sample, wherein the image quality of the standard cloud image samples is consistent with the image quality of the second input image samples; and The discriminative generation network is constructed, and trained using the plurality of first input image samples, the plurality of second input image samples, and the standard cloud image samples.

6. A storage medium, characterized in that, The storage medium includes a stored program, wherein, when the program is executed, the method described in any one of claims 1 to 5 is performed by a processor.

7. An image enhancement device for high-orbit satellite cloud images, characterized in that, include: The cloud image receiving module is used to acquire a first cloud image corresponding to a first target area, and to receive second cloud images corresponding to each second target area sent by multiple low-orbit satellites, wherein each second target area is within the coverage area of ​​the first target area. The image stitching module is used to stitch together images of various second cloud layers and generate the first input image; The input image generation module is used to determine multiple image regions in the first cloud image that correspond to each of the second target regions, and to segment and combine the first cloud image based on each image region to generate a second input image. The image analysis module is used to input the first input image, the second input image, and the first cloud image into a pre-trained discriminative generation network, and use the discriminative generation network to generate a first image quality analysis result corresponding to the first input image and the second input image; as well as The standard cloud image output module is used to output a standard cloud image corresponding to the first target area using the discrimination generation network and based on the first image quality analysis result.

8. The apparatus according to claim 7, characterized in that, The input image generation module includes: The position coordinate receiving module is used to receive the position coordinates corresponding to the respective second target areas transmitted by the respective low-orbit satellites; The image region determination module is used to determine the image region in the first cloud image that corresponds to each of the second target regions based on each location coordinate; An image patch generation module is used to segment the first cloud image according to a determined image region, generating multiple image patches corresponding to each of the second cloud images; and The input image generation submodule is used to combine the multiple image blocks and generate a second input image.

9. The apparatus according to claim 8, characterized in that, The discriminant generation network includes a discriminator, and the image analysis module includes: The quality analysis result output module is used to input the first input image and the second input image to the discriminator, and use the discriminator to output the first image quality analysis result, wherein the first image quality analysis result includes whether the image quality between the first input image and the second input image is consistent, or whether the image quality between the first input image and the second input image is inconsistent.

10. An image enhancement device for high-orbit satellite cloud images, characterized in that, include: processor; as well as A memory, connected to the processor, for providing the processor with instructions to perform the following processing steps: Acquire a first cloud image corresponding to the first target area, and receive second cloud images corresponding to each second target area transmitted by multiple low-orbit satellites, wherein each second target area is within the coverage area of ​​the first target area; The images of each second cloud layer are stitched together to generate the first input image; Multiple image regions in the first cloud image that correspond to each of the second target regions are determined, and the first cloud image is segmented and combined based on each image region to generate a second input image; The first input image, the second input image, and the first cloud image are input into a pre-trained discriminative generation network, and the discriminative generation network is used to generate a first image quality analysis result corresponding to the first input image and the second input image; as well as Using the discriminative generation network and based on the first image quality analysis results, a standard cloud image corresponding to the first target region is output.

Images