Deep adversarial network super-resolution reconstruction method based on genetic optimization

Abstract

The invention provides a deep adversarial network super-resolution reconstruction method based on genetic optimization, and belongs to the technical field of image super-resolution reconstruction. According to the method, a generator structure of an adversarial network is converted into an evolvable gene sequence, and an optimal network topology is automatically searched in evolution through module-level gene coding, tournament selection, crossing of modules of the same type and parameter disturbance variation; meanwhile, a multi-target fitness function fusing PSNR and LPIPS is constructed, dynamic weight adjustment takes pixel precision and perception quality into consideration, and after evolution, an optimal individual is decoded to obtain a light-weight and high-reconstruction-quality remote sensing image super-resolution generator. According to the method, aiming at the characteristics of large scale difference and complex texture of the remote sensing image, the network depth, the convolution channel number and the module combination are automatically optimized, and architecture search with strong pertinence and good generalization ability is realized. The processing process of the method is not approximate, the quality of the reconstructed image can be effectively improved, the model complexity is reduced, and the method is suitable for remote sensing scenes such as ground feature recognition and target detection.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image super-resolution reconstruction, and particularly relates to a remote sensing image super-resolution reconstruction method combining a genetic algorithm adaptive evolution adversarial network generator. BACKGROUND

[0002] Image super-resolution technology aims to restore low-resolution images to high-resolution images to improve image details and overall quality, and is widely used in remote sensing monitoring, medical imaging, security monitoring and other scenarios. Traditional super-resolution methods mainly rely on interpolation algorithms, although the computational overhead is low, but they generally have problems such as difficulty in restoring high-frequency details, limited image quality, etc., especially when facing complex textures or high magnification, traditional methods are difficult to meet the fine needs.

[0003] In recent years, super-resolution methods based on deep learning have made significant progress. Since Dong et al. proposed the convolutional neural network SRCNN (Super-Resolution Convolutional Neural Network) (reference: Dong C, Loy C C, He K, et al. Learning a deep convolutional network for image super-resolution [C] / / Computer Vision–ECCV 2014: 13th European Conference, Zurich, Switzerland, September 6-12, 2014, Proceedings, Part IV 13. Springer International Publishing, 2014: 184-199.), research has gradually developed from shallow networks to deep networks, and representative methods include the introduction of residual structure VDSR (Very Deep Super-Resolution) (reference: J. Kim, J. K. Lee, and K. M. Lee, “Accurate image super-resolution using very deep convolutional networks,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2016, pp. 1646-1654.) and LapSRN (Laplacian Pyramid SR Network) (reference: W.-S. Lai, J.-B. Huang, N. Ahuja, and M.-H. Yang, “Fast and accurate image super-resolution with deep laplacian pyramid networks,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 41, no. 11, pp. 2599-2613, 2019.) using pyramid architecture, which improves the reconstruction ability and training efficiency of the model.The introduction of SRGAN (Super-Resolution Generative Adversarial Network) (reference: C. Ledig et al., “Photo-realistic single image super-resolution using a generative adversarial network,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2017, pp. 4681-4690.) brings in perceptual loss and adversarial learning mechanism, making the generated image closer to human eye perception, significantly improving the visual quality, marking the super-resolution task from “pixel-driven” to “perception-driven”. Since then, advanced architectures such as Transformer and diffusion model have also been applied to SR tasks, continuously improving model performance. Although the architecture of deep models is constantly developing, existing SR methods generally rely on static structures set by humans, and the design is often based on experience or exhaustive experiments, lacking flexibility and adaptability. In applications such as remote sensing images, where the scale difference of ground objects is large and the texture structure is complex, fixed architecture is difficult to effectively cope with, limiting the improvement of reconstruction performance.

[0004] Evolutionary deep learning introduces optimization mechanisms from evolutionary algorithms to automatically explore and adjust the structure, weights, and hyperparameters of deep networks, thereby enhancing model performance and significantly reducing the cost of manual design and trial-and-error. Among them, genetic algorithm (GA) as a classic evolutionary strategy (Reference: S. Mirjalili and S. Mirjalili, “Genetic algorithm,” Evol. Algorithms Neural Netw.: Theory Appl., pp. 43–55, 2019.) has shown strong optimization ability in various complex tasks. Insik Jung used GA to complete the orbit configuration optimization of multi-source heterogeneous satellite constellation, enabling non-cooperative satellites to operate efficiently (Reference: I. Jung and D. Chung, “Geneticalgorithm-based approach for improving temporal resolution in constellationoperation of national satellites,” Int. J. Aeronaut. Space Sci., vol. 26, no. 1, pp. 314–326, Jan. 2025, doi: 10.1007 / s42405-024-00761-1.). Hao Zhang applied GA to the design of laser ranging system, breaking through the dependence on artificial experience in traditional design and significantly improving the consistency of system performance (Reference: H. Zhang, S. Wang, and J. Wang, “Automatic optimization design of laser triangulation ranging sensors using an improved genetic algorithm,” Measurement, vol. 241, p. 115739, Feb. 2025, doi: 10.1016 / j.measurement.2024.115739.).Tongpo Zhang improved the quality of PCB image fusion and the effect of detail restoration by introducing a genetic algorithm combined with an elite strategy (ESGA) (Ref: T. Zhang, Q. Yin, S. Li, T. Guo, and Z. Fan, “An optimized genetic algorithm-based wavelet image fusion technique for PCB detection,” Appl. Sci., vol. 15, no. 6, p. 3217, Mar. 2025, doi: 10.3390 / app15063217.). These studies show that genetic algorithms can effectively overcome the limitations of manual design and have good structure optimization capabilities.

[0005] Inspired by this, the present application plans to introduce genetic algorithms into the architecture optimization task of image super-resolution networks. However, how to combine genetic algorithms, remote sensing image features, and super-resolution networks, use evolutionary algorithms to automatically optimize model structures, and reduce model complexity while ensuring reconstruction accuracy, is still a key problem to be solved. SUMMARY

[0006] To solve the above technical problems, the purpose of the present application is to provide a deep generative adversarial network super-resolution reconstruction method based on genetic optimization, which converts the structure design process of the generative adversarial network generator into a modular combination task, uses a module-level gene coding strategy to express the network structure flexibly, balances pixel accuracy and perceptual quality by constructing a multi-objective fitness function that combines PSNR and LPIPS, and introduces module-level selection, crossover, and mutation operations during the evolution process to guide continuous optimization of the architecture and gradually improve the reconstruction ability and visual performance.

[0007] To achieve the above purpose, the present application proposes a deep generative adversarial network super-resolution reconstruction method based on genetic optimization, comprising the following steps:

[0008] S1: Define the gene structure of the individual, each individual is composed of several basic modules, including convolutional layers (C), multi-scale convolutional modules (MS), residual modules (RES), PReLU activation layers (P), dynamic feature fusion modules (DF), static feature fusion modules (SF), up-sampling modules (US), and batch normalization layers (BN), which are used to construct network individuals with variable structures;

[0009] S2: Construct an initial generative adversarial network generator individual, combine different modules to generate an initial network architecture, provide a variety of structural seeds for evolution, and the example of the generative adversarial network generator individual is S21;

[0010] S3: initialize the population, randomly generate a plurality of individuals, each individual corresponds to a trainable network with different module number, combination, order and parameter;

[0011] S4: pre-train each individual with MSE, use mean square error (MSE) as the loss function, preliminarily train the generator, so that it realizes high-fidelity reconstruction in the low-resolution to high-resolution mapping process, and obtains initial weight parameters that are numerically stable and close to the real distribution, providing a good starting point for subsequent adversarial training;

[0012] S5: perform lightweight adversarial training on the generator individual pre-trained with MSE and the pre-defined discriminator shown in S51, fine-tune the generator using the generative adversarial mechanism, guide the generated image to approach the real image manifold, and improve the image perception quality, while avoiding mode collapse phenomenon in the training process;

[0013] S6: evaluate the fitness of each individual, use PSNR and LPIPS on the validation set as fitness indicators, retain individuals with low fitness and discard individuals with high fitness;

[0014] S7: perform cross operation on the retained individuals, generate new sub-individuals using a random exchange method based on module index;

[0015] S8: perform mutation operation on the sub-individuals, generate new mutation individuals using parameter perturbation method. The channel number parameter of the convolution layer module in the individual is multiplied by a random factor in the range of [0.8, 1.2]; the branch number parameter of the multi-scale convolution module is a random integer in the range of [2, 3], when the branch number is 2, the convolution kernel configuration parameter is a random integer in the range of [0, 2], and when the branch number is 3, the convolution kernel configuration can only be 3; the jump interval parameter of the dynamic fusion module is a random integer in the range of [2, 4]; the feature fusion strategy parameter of the static fusion module is a random integer in the range of [0, 1];

[0016] S9: the generated new generation of individuals is updated to a new generation of population;

[0017] S10: repeat steps S4 to S9 for evolutionary iteration until the iteration number reaches the preset upper limit;

[0018] S11: output the adversarial network generator architecture with the optimal fitness.

[0019] Based on the above, the deep adversarial network super-resolution reconstruction method based on genetic optimization of the present application dynamically guides the search direction by optimizing the objective function during the evolution process, realizes the automatic optimization of the super-resolution generator architecture without relying on artificial experience, and effectively improves the reconstruction quality and perception performance of the generated image. BRIEF DESCRIPTION OF DRAWINGS

[0020] Figure 1 is a flowchart of the genetic optimization-based deep adversarial network super-resolution reconstruction method of the present application;

[0021] Figure 2 is a comparison chart of the image super-resolution results of the present application and the super-resolution results of Bicubic, SRRESNET, SRGAN, ESRGAN, SRCNN, and RRDBNET (NWPU-RESISC45 simulation data). DETAILED DESCRIPTION

[0022] The following will be described Figure 1 , Figure 2 The genetic optimization-based deep adversarial network super-resolution reconstruction method of the present application will be described in detail.

[0023] As shown in Figure 1 , in order to obtain better image super-resolution results, the present application provides a genetic optimization-based deep adversarial network super-resolution reconstruction method, which comprises the following steps:

[0024] S1: defining the gene structure of an individual, each individual being composed of a plurality of basic modules, including a convolution layer (C), a multi-scale convolution module (MS), a residual module (RES), a PReLU activation layer (P), a dynamic feature fusion module (DF), a static feature fusion module (SF), an up-sampling module (US), and a batch normalization layer (BN), which are used to construct a network individual with variable structure;

[0025] S2: constructing an initial adversarial network generator individual, combining different modules to generate an initial network architecture, providing a diversified structure seed for evolution, and an example of an adversarial network generator individual is shown in S21;

[0026] S3: initializing a population and randomly generating a plurality of individuals, each individual corresponding to a trainable network with different numbers, combinations, sequences, and parameters of modules;

[0027] S4: pre-training each individual by MSE, taking the mean square error (MSE) as a loss function, and preliminarily training the generator to achieve high-fidelity reconstruction in the low-resolution to high-resolution mapping process, obtaining initial weight parameters that are numerically stable and close to the real distribution, and providing a good starting point for subsequent adversarial training;

[0028] S5: performing lightweight adversarial training on the generator individual pre-trained by MSE and the pre-defined discriminator shown in S51, using a generative adversarial mechanism to fine-tune the generator, guiding the generated image to approach the real image manifold, improving the image perception quality, and avoiding the pattern collapse phenomenon in the training process;

[0029] S6: Perform fitness evaluation on each individual, use PSNR and LPIPS on the validation set as fitness indicators, retain individuals with low fitness, discard individuals with high fitness;

[0030] S7: Perform crossover operation on the retained individuals, use a random exchange method based on module index to generate new sub-individuals;

[0031] S8: Perform mutation operation on the sub-individuals, use parameter perturbation method to generate new mutation individuals. The channel number parameter of the convolution layer module in the individual is multiplied by a random factor in the range of [0.8, 1.2]; the branch number parameter of the multi-scale convolution module is a random integer in the range of [2, 3], when the branch number is 2, the convolution kernel configuration parameter is a random integer in the range of [0, 2], and when the branch number is 3, the convolution kernel configuration can only be 3; the jump interval parameter of the dynamic fusion module is a random integer in the range of [2, 4]; the feature fusion strategy parameter of the static fusion module is a random integer in the range of [0, 1];

[0032] S9: The generated new generation of individuals is updated to the new generation population;

[0033] S10: Repeat steps S4 to S9 for evolution iteration until the iteration number reaches the preset upper limit;

[0034] S11: Output the adversarial network generator architecture with the optimal fitness.

[0035] The method of the application will be further described below through an implementation example.

[0036] Dataset: NWPU-RESISC45 simulation data.

[0037] Evaluation index: For simulation data, peak signal-to-noise ratio (PSNR), structural similarity index (SSIM), and perceptual similarity (LPIPS) are used. Higher scores of PSNR and SSIM indicate better image quality of super-resolution, and lower scores of LPIPS indicate better image quality and visual effect.

[0038] Implementation steps:

[0039] The NWPU-RESISC45 simulation data is processed using the method of the application and Bicubic, SRRESNET, SRGAN, ESRGAN, SRCNN, and RRDBNET methods respectively. In the experiment, the parameters of the method of the application are as follows: the maximum population size is set to 100, the evolution generation is set to 20, the crossover rate is set to 0.7, and the mutation rate is set to 0.4. The crossover and mutation methods of genetic algorithm are based on layer index exchange method and parameter perturbation mutation method respectively.

[0040] For the NWPU-RESISC45 simulation data, the image super-resolution results are as shown in Figure 2 Fig. 4 (h), and the four times super-resolution results (see Figure 2 (h) ) of the method of the present application are compared with the four times super-resolution results (see Figure 2 (b), Figure 2 (c), Figure 2 (d), Figure 2 (e), Figure 2 (f), Figure 2 (g) ) of Bicubic, SRRESNET, SRGAN, ESRGAN, SRCNN and RRDBNET, it can be found that the method of the present application is more real in restoring details, and the overall linear distribution and spatial structure of the parking lot are better restored in vision, and the super-resolution results are closer to clear images. In order to further prove that the framework optimized by the method of the present application has good super-resolution effect, the PSNR, SSIM and LPIPS values of the super-resolution images are calculated. As shown in the comparison results in Figure 2 Fig. 5, the PSNR values of the super-resolution results (see Figure 2 (b), Figure 2 (c), Figure 2 (d), Figure 2 (e), Figure 2 (f), Figure 2 (g) ) of Bicubic, SRRESNET, SRGAN, ESRGAN, SRCNN and RRDBNET are 21.79, 21.90, 23.47, 25.51, 23.25, 23.76 respectively, the SSIM values are 0.5247, 0.5298, 0.6843, 0.7730, 0.5426, 0.5992 respectively, and the LPIPS values are 0.3785, 0.4015, 0.2274, 0.2014, 0.4086, 0.3672 respectively; the PSNR, SSIM and LPIPS values of the super-resolution results (see ​ (h) ) of the method of the present application are 26.01, 0.7942 and 0.1686 respectively. The PSNR, SSIM and LPIPS values of the super-resolution results of other methods are all less than the PSNR, SSIM and LPIPS values of the image super-resolution results of the present application, so the framework optimized by the method of the present application indeed has good super-resolution effect.

[0041] The above only describes the preferred embodiments of the present application, and is not intended to limit the present application, and any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. A super-resolution reconstruction method based on deep adversarial networks using genetic optimization, characterized in that, Includes the following steps: S1: Defines the genetic structure of an individual. Each individual consists of several basic modules, including convolutional layers (C), multi-scale convolutional modules (MS), residual modules (RES), PReLU activation layers (P), dynamic feature fusion modules (DF), static feature fusion modules (SF), upsampling modules (US), and batch normalization layers (BN), used to construct network individuals with variable structures. S2: Construct the initial adversarial network generator individual, combine different modules to generate the initial network architecture, and provide diverse structural seeds for evolution. An example of the adversarial network generator individual is S21. S3: Initialize the population and randomly generate multiple individuals, each individual corresponding to a trainable network with different number, combination, order and parameters of modules; S4: Perform MSE pre-training on each individual, using mean squared error (MSE) as the loss function to initially train the generator, enabling it to achieve high-fidelity reconstruction during the low-resolution to high-resolution mapping process, and obtain numerically stable initial weight parameters that are close to the real distribution, providing a good starting point for subsequent adversarial training. S5: Perform lightweight adversarial training between the generator individuals pre-trained by MSE and the discriminator predefined in S51. Use the generative adversarial mechanism to fine-tune the generator, guide the generated images to move closer to the real image manifold, improve the image perception quality, and avoid the mode collapse phenomenon during training. S6: Evaluate the fitness of each individual, using PSNR and LPIPS on the validation set as fitness indicators, retaining individuals with low fitness and discarding individuals with high fitness; S7: Perform crossover operations on the remaining individuals, using a random swapping method based on module indexes to generate new sub-individuals; S8: Perform a mutation operation on the sub-individuals, generating new mutated individuals using parameter perturbation. The channel number parameter of the convolutional layer module within each individual is multiplied by a random factor ranging from [0.8, 1.2]; the branch number parameter of the multi-scale convolutional module ranges from a random integer between [2, 3]. When the number of branches is 2, the convolutional kernel configuration parameter ranges from a random integer between [0, 2]; when the number of branches is 3, the convolutional kernel configuration can only be 3; the jump interval parameter of the dynamic fusion module ranges from a random integer between [2, 4]; and the feature fusion strategy parameter of the static fusion module ranges from a random integer between [0, 1]. S9: The generated new generation of individuals is updated into a new generation of the population; S10: Repeat steps S4 to S9 for evolutionary iteration until the number of iterations reaches the preset limit; S11: Output the adversarial network generator with optimal fitness.

Images