Image super-resolution reconstruction method, system, equipment and medium

Abstract

The invention relates to the technical field of image super-resolution under a power system, and discloses an image super-resolution reconstruction method, system and device and a medium, and the method comprises the steps: presetting a plurality of test stages which are connected in sequence, obtaining a target test image, and inputting the target test image; establishing an image super-resolution reconstruction model, and inputting optimal enhanced view subsets and learnable prompts in each stage; and carrying out image super-resolution reconstruction according to the trained model. Firstly, the image quality can be gradually optimized by presetting multiple stages, and the super-resolution effect is improved. Secondly, the optimal enhanced view subset is selected as input, redundancy and noise can be reduced, and the reconstruction quality is improved. Moreover, learnable prompts are introduced as input, so that the generalization ability of the model can be improved. Finally, the method combines the advantages of image enhancement and super-resolution reconstruction, can guarantee the detail texture of the image, improves the resolution and definition, and provides a new scheme for related fields.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of image super-resolution technology in power systems, and in particular to an image super-resolution reconstruction method, system, device and medium. BACKGROUND

[0002] In the daily operation and fault troubleshooting of power systems, high-altitude operation robots, mobile inspection platforms or remote monitoring devices are often used to obtain image information of key facilities such as substations, towers and switch cabinets. However, due to the uncontrollability of the on-site environment, such as strong light interference, equipment vibration, low-illumination night operation and high-temperature and high-humidity weather conditions, the collected images often have various degradation phenomena such as insufficient brightness, missing details and compression artifacts. These low-quality images not only affect the judgment efficiency of the operation personnel, but also interfere with subsequent intelligent recognition algorithms such as target detection and defect classification. Therefore, image enhancement and super-resolution reconstruction technology has become a key support means to improve the perception ability of power systems.

[0003] Image super-resolution reconstruction aims to recover a high-resolution image from a low-resolution image. However, most existing methods are trained on synthetic data sets, in which HR images are generated by down-sampling LR images using a pre-defined down-sampling model, such as bicubic down-sampling. Although effective in limited test environments, these models often cannot be generalized to real complex power operation scenarios. And models trained based on synthetic data often produce results comparable to traditional interpolation techniques, and are difficult to generalize to unseen degradation cases. In order to cope with these limitations, some studies have compiled real-world data sets for training and evaluating SR models, but these methods usually require a large amount of resources and are costly. In addition, images taken from different cameras often exhibit significant differences in degradation kernels. These inconsistencies can negatively affect the performance and stability of SR models, further increasing the complexity of adapting to real power operation scenarios. In addition, some studies attempt to use second-order degradation to construct pseudo-data pairs, thereby enhancing the generalization ability of the model to some extent. However, this fixed degradation assumption is often different from the complex and variable degradation in reality, thereby limiting the performance of the model on real power operation scenario data sets. SUMMARY

[0004] In view of the above existing problems, the present application is proposed.

[0005] Therefore, the present application provides an image super-resolution reconstruction method, system, device and medium, which can solve the problem of severe image degradation and limited model generalization ability in real power operation scenarios.

[0006] To solve the above technical problems, the present application provides the following technical solutions:

[0007] In a first aspect, the present application provides an image super-resolution reconstruction method, comprising:

[0008] A plurality of test stages are preset, and the plurality of test stages are sequentially connected;

[0009] The plurality of test stages are used for enhancing, screening and distribution alignment of a target test image;

[0010] A target test image is obtained, and the target test image is input into the plurality of test stages;

[0011] An image super-resolution reconstruction model is established, and an input of the image super-resolution reconstruction model is an optimal enhanced view subset of each stage and a learnable hint;

[0012] The optimal enhanced view subset is an optimal enhanced view of an enhanced view of the target test image after enhancement in each test stage;

[0013] The learnable hint is an adjustable variable, which is used for aligning a distribution of test data of the image super-resolution reconstruction model with a distribution of training data;

[0014] Image super-resolution reconstruction is performed according to the image super-resolution reconstruction model after training is completed.

[0015] As a preferred scheme of the image super-resolution reconstruction method, the plurality of test stages are sequentially connected, comprising:

[0016] The plurality of test stages comprise a simple stage, a medium stage and a complex stage;

[0017] An output of the simple stage is an input of the medium stage, and an output of the medium stage is an input of the complex stage.

[0018] This preferred scheme can gradually enhance, screen and align the distribution of the target test image, and gradually improve the resolution and quality of the image through processing in different stages. In the simple stage, the target test image can be preliminarily enhanced and screened to remove some obvious noise and interference, thereby laying a foundation for subsequent processing. In the medium stage, the image is further enhanced and screened to improve the details and clarity of the image. In the complex stage, the image is enhanced and screened at a higher level to ensure that the super-resolution reconstruction effect of the image reaches the best. In this way, the present application can effectively solve the problem of serious image degradation in a real power operation scene and the limited model generalization ability, and improve the accuracy and efficiency of image super-resolution reconstruction.

[0019] As a preferred scheme of the image super-resolution reconstruction method, the establishment of the image super-resolution reconstruction model comprises:

[0020] An initial image super-resolution reconstruction model is obtained, and a target test image is input to obtain an initial high-resolution image;

[0021] A first enhanced view is obtained by performing a first preprocessing on the target test image and the initial high-resolution image in a simple stage;

[0022] The first enhanced view includes a first simple test image and a first simple high-resolution image.

[0023] As a preferred scheme of the image super-resolution reconstruction method, the method further comprises:

[0024] A second enhanced view is obtained by performing a second preprocessing on the first enhanced view in a medium stage;

[0025] The first enhanced view includes a second medium test image and a second medium high-resolution image.

[0026] As a preferred scheme of the image super-resolution reconstruction method, the method further comprises:

[0027] A third enhanced view is obtained by performing a third preprocessing on the second enhanced view in a medium stage;

[0028] The third enhanced view includes a third complex test image and a third complex high-resolution image.

[0029] An optimal enhanced view subset of the first enhanced view, the second enhanced view and the third enhanced view is obtained;

[0030] A preset learnable hint is combined with the optimal enhanced view subset of each stage to retrain the image super-resolution reconstruction model.

[0031] As a preferred scheme of the image super-resolution reconstruction method, the optimal enhanced view subset comprises:

[0032] An evaluation index of the enhanced view of each stage is obtained;

[0033] The evaluation index of the enhanced view of each stage is sorted;

[0034] The optimal enhanced view of each stage is selected as the optimal enhanced view subset.

[0035] As a preferred solution of the image super-resolution reconstruction method, the distribution of the test data is aligned with the distribution of the training data by establishing a loss function, wherein the loss function comprises a learnable hint, and the distribution of the test data is aligned with the distribution of the training data by adjusting the learnable hint.

[0036] In a second aspect, the present application provides an image super-resolution reconstruction system, comprising:

[0037] a stage design module configured to preset a plurality of test stages, wherein the plurality of test stages are sequentially connected;

[0038] The plurality of test stages are configured to enhance, screen and align the distribution of the target test image;

[0039] a running module configured to obtain a target test image and input the target test image into the plurality of test stages;

[0040] a model establishing module configured to establish an image super-resolution reconstruction model, wherein the input of the image super-resolution reconstruction model is an optimal enhanced view subset of each stage and a learnable hint;

[0041] The optimal enhanced view subset is an optimal enhanced view of the enhanced view of the target test image in each test stage;

[0042] The learnable hint is an adjustable variable, which is used to align the distribution of the test data of the image super-resolution reconstruction model with the distribution of the training data;

[0043] a reconstruction module configured to perform image super-resolution reconstruction according to the image super-resolution reconstruction model after training.

[0044] In a third aspect, the present application provides an electronic device comprising a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to realize the steps of the method described above.

[0045] In a fourth aspect, the present application provides a computer readable storage medium, which stores a computer program, wherein the computer program is executed by a processor to realize the steps of the method described above.

[0046] Compared with the prior art, the present application has the beneficial effects that: the present application provides an image super-resolution reconstruction method, a plurality of test stages are preset and connected in sequence; a target test image is obtained and input into the plurality of test stages; an image super-resolution reconstruction model is established, the input of the image super-resolution reconstruction model is an optimal enhanced view subset of each stage and a learnable hint; and image super-resolution reconstruction is performed according to the image super-resolution reconstruction model after training. First, by presetting a plurality of test stages and connecting them in sequence, the method of the present application can gradually optimize the image quality, ensure that each stage can effectively enhance the image, and thus improve the super-resolution effect of the final reconstructed image. Second, by selecting the optimal enhanced view subset of each stage as the input, the present application can maximize the use of the information of the enhanced view, reduce the interference of redundancy and noise, and further improve the quality of image reconstruction. Third, the introduction of the learnable hint as part of the model input enables the method of the present application to better adapt to different test data distributions and improve the generalization ability of the model. Finally, the method of the present application combines the advantages of image enhancement and super-resolution reconstruction, can significantly improve the resolution and clarity of the image while ensuring the details and texture of the image, and provides a new solution for the image processing and computer vision fields. BRIEF DESCRIPTION OF DRAWINGS

[0047] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings needed in the embodiment description will be briefly introduced as follows. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor.

[0048] Figure 1 A method flowchart of an image super-resolution reconstruction method provided by an embodiment of the present application.

[0049] Figure 2 A whole architecture schematic diagram of an image super-resolution reconstruction method provided by an embodiment of the present application.

[0050] Figure 3 A generation and screening schematic diagram of an enhanced view of an image super-resolution reconstruction method provided by an embodiment of the present application.

[0051] Figure 4 A hint updating, i.e., distribution alignment, schematic diagram of an image super-resolution reconstruction method provided by an embodiment of the present application.

[0052] Figure 5 A training data and test sample distribution alignment schematic diagram of an image super-resolution reconstruction method provided by an embodiment of the present application.

[0053] Figure 6 An internal structure diagram of an electronic device providing an image super-resolution reconstruction method according to an embodiment of the present application is shown. DETAILED DESCRIPTION

[0054] In order to make the above objectives, features and advantages of the present application more obvious and comprehensible, specific embodiments of the present application will be described in detail below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative work should belong to the protection scope of the present application.

[0055] Embodiment 1, Reference Figures 1-6 For the first embodiment of the present application, the embodiment provides an image super-resolution reconstruction method, comprising:

[0056] In the prior related art, there are some problems, for example, image degradation is serious in real power operation scene, model generalization ability is limited, etc.

[0057] The present application provides a method that can effectively solve the above-mentioned problems. Next, how to implement the image super-resolution reconstruction method will be described in detail in combination with multiple embodiments.

[0058] Figure 1 A method flowchart of an image super-resolution reconstruction method is shown, comprising:

[0059] S101, a plurality of test stages are preset, and the plurality of test stages are connected in sequence;

[0060] It should be noted that image super-resolution reconstruction is needed to perform image super-resolution reconstruction on the input image to obtain a higher resolution and clearer image.

[0061] In an optional embodiment, the input image can be feature extracted or enhanced by image enhancement technology, and a new image super-resolution reconstruction model is retrained or established according to the feature extracted or enhanced data, so as to realize higher precision super-resolution reconstruction of the input image.

[0062] In an optional embodiment, the image super-resolution reconstruction model can be constructed by a deep learning algorithm, such as a convolutional neural network (CNN) or a generative adversarial network (GAN). Specifically, when using a convolutional neural network (CNN) or a generative adversarial network (GAN), the specific steps can be as follows:

[0063] First, the input degraded image is preprocessed, such as denoising, deblurring, etc., to improve the image quality.

[0064] Then, the pre-processed images are input into the constructed deep learning model for feature extraction and image reconstruction. In the feature extraction phase, the deep learning model can automatically learn useful features in the images, such as edges, textures, etc., and use these features for subsequent image reconstruction.

[0065] Finally, in the image reconstruction phase, the deep learning model uses the extracted feature information to generate higher resolution and clearer images.

[0066] Through continuous iteration and optimization, the deep learning model can gradually improve the effect of image super-resolution reconstruction, achieving higher precision super-resolution reconstruction of input images.

[0067] In an optional embodiment, the image super-resolution reconstruction model can also be constructed using other advanced deep learning algorithms, such as Transformer networks or recurrent neural networks (RNN). The specific steps can be as follows:

[0068] First, the system receives the input degraded image, which may be due to limitations of the acquisition device, loss during transmission or other factors resulting in reduced resolution.

[0069] Then, the received degraded image is pre-processed, which includes but is not limited to denoising, contrast enhancement and possible geometric correction to ensure the quality of the image suitable for subsequent deep learning processing.

[0070] Next, the pre-processed image is sent to a deep learning model constructed using a Transformer network or a recurrent neural network (RNN). In the Transformer network, the image is divided into multiple small blocks, each of which is treated as a sequence element for processing, and the model learns the relationship between these elements to reconstruct a higher resolution image. In the recurrent neural network, image information is input into the network pixel by pixel or block by block, and the network uses previous information to predict the value of the current pixel or image block, gradually building the entire high-resolution image.

[0071] In the feature extraction phase, these deep learning models can capture complex features and patterns in the image, such as the shape of objects, color distribution, and texture details. These features are then used in the image reconstruction phase, where the model uses the extracted feature information to generate images with higher resolution and finer details.

[0072] Finally, the entire system iterates and optimizes the parameters of the deep learning model to improve the effect of image super-resolution reconstruction. This includes adjusting the network structure, changing the loss function, and using more training data, etc. strategies, aiming to make the reconstructed image closer to the real high-resolution image.

[0073] But it is impossible to really improve the image super-resolution reconstruction model according to a simple image processing or image enhancement, therefore the application designs several test stages as follows to realize the final image super-resolution reconstruction model improvement.

[0074] In the embodiment of the application, the several test stages are used for enhancement, screening and distribution alignment of the target test image;

[0075] In the embodiment of the application, the preset several test stages are sequentially connected and include:

[0076] The several test stages include a simple stage, an intermediate stage and a complex stage.

[0077] The output of the simple stage is the input of the intermediate stage, and the output of the intermediate stage is the input of the complex stage.

[0078] Specifically, input a test image X test , use a pre-trained super-resolution model to generate an initial high-resolution image Y test from X test as a pseudo reference.

[0079] The test stages are divided into three stages, namely, a simple stage, an intermediate stage and a complex stage.

[0080] In an optional embodiment, in the simple stage, use large-size cropping to generate an enhanced view set from X test and Y test . That is, the enhanced view set The cropping size ranges from 0.7 to 1 times the image size.

[0081] In an optional embodiment, similarly, in the intermediate stage, use mixed large-size and medium-size cropping to generate an enhanced view set, and the cropping size ranges from 0.5 to 0.7 times the image size.

[0082] In an optional embodiment, in the complex stage, use mixed medium-size and small-size cropping to generate an enhanced view set, and the cropping size ranges from 0.3 to 0.5 times the image size.

[0083] It should be noted that a person skilled in the art can add other stages according to actual needs, for example, an extremely complex stage, which uses mixed small-size and extremely small-size cropping to generate an enhanced view set, and the cropping size ranges from 0.1 to 0.3 times the image size. The cropping of each stage aims to simulate image degradation of different scales, thereby enhancing the adaptability of the model to images of different degradation degrees.

[0084] It should be noted that a plurality of test stages are preset, and the plurality of test stages are sequentially connected to gradually enhance the target test image, so that more useful image features are gradually extracted and screened in different test stages. This phased method ensures that each stage can effectively enhance and optimize the image, avoiding the limitations that may be brought by single simple image processing or image enhancement. At the same time, by taking the output of different stages as the input of the next stage, the method of the present application can make full use of the information of the previous stage, further improving the quality of image reconstruction. In addition, the preset plurality of test stages and the sequential connection also provide more stable and reliable input data for subsequent model training and reconstruction process, which helps to improve the performance and accuracy of the entire image super-resolution reconstruction system.

[0085] S102, acquiring a target test image, and inputting the target test image into a plurality of test stages;

[0086] It should be noted that after establishing a plurality of test stages, the target test image needs to be input and output for subsequent image super-resolution reconstruction processing. The target test image can be any image that needs to be super-resolution reconstructed, such as low-resolution monitoring images, blurred medical images, etc. Inputting the target test image into the preset plurality of test stages can make the image gradually undergo enhancement, screening, and distribution alignment in different test stages, thereby extracting more useful image features and providing more stable and reliable input data for subsequent image super-resolution reconstruction.

[0087] In an optional embodiment, after inputting the target test image into the plurality of test stages, each test stage will perform enhancement processing on the image to generate a series of enhanced views. These enhanced views are generated by different image enhancement techniques, aiming to simulate different image degradation conditions, thereby enhancing the adaptability of the model to images with different degradation degrees. Next, the system will evaluate the enhanced views generated by each test stage and select the optimal enhanced view of each stage as the optimal enhanced view subset. This step is achieved by calculating the evaluation indicators of each enhanced view, such as peak signal-to-noise ratio (PSNR), structural similarity (SSIM), etc., and then sorting the evaluation indicators to select the optimal enhanced view. By selecting the optimal enhanced view subset as the input, the method of the present application can maximize the use of enhanced view information, reduce the interference of redundancy and noise, and further improve the quality of image reconstruction.

[0088] In an optional embodiment, after the screening of the enhanced views is completed, the system also needs to align the distribution of the test data to ensure that the distribution of the test data is consistent with the distribution of the training data. This step is achieved by establishing a loss function that includes a learnable hint. By adjusting the learnable hint, the distribution of the test data can gradually approach the distribution of the training data, thereby improving the generalization ability of the model. In the embodiments of the present application, the learnable hint is an adjustable variable used to align the distribution of the test data of the image super-resolution reconstruction model with the distribution of the training data. By introducing the learnable hint as part of the model input, the method of the present application can better adapt to different test data distributions and improve the generalization ability of the model.

[0089] For example, at each stage, the obtained enhanced views are The trained image super-resolution reconstruction model obtained

[0090] Using and The evaluation index WeightedPSNR of each enhanced view is calculated, i.e. WeightedPSNR = PSNR w(r), where r is the crop size of the enhanced view. The enhanced views of each stage are sorted according to WeightedPSNR, and the optimal enhanced view subset of each stage is selected PSNR is an important index for measuring the similarity between the reconstructed image and the real image.

[0091] Further, the specific content of calculating the WeightedPSNR of each enhanced view is:

[0092] In different stages, w(r) is set differently. In the simple stage, w(r) = 1. In the medium stage, w(r) = 0.8 + 0.2r. In the complex stage, w(r) = 0.5 + 0.5r.

[0093] It should be noted that the target test image is obtained and input into several test stages, so that the image can be gradually subjected to fine processing such as enhancement, screening, and distribution alignment in different test stages, thereby fully utilizing useful information in the image and reducing the interference of redundancy and noise. The method of processing in stages and gradually can ensure that each stage can effectively optimize the image, and provide more stable and reliable input data for subsequent image super-resolution reconstruction. In addition, by inputting the target test image into the preset several test stages, different image degradation conditions can be simulated, the adaptability of the model to images of different degradation degrees can be enhanced, and the performance and accuracy of the entire image super-resolution reconstruction system can be further improved. In the subsequent step, the system will perform higher-precision image super-resolution reconstruction based on the optimized image data to meet the needs of actual applications.

[0094] S103, an image super-resolution reconstruction model is established, and the input of the image super-resolution reconstruction model is the optimal enhanced view subset of each stage and a learnable hint;

[0095] It should be noted that the image super-resolution reconstruction model established here is actually a final improved image super-resolution reconstruction model, and the final improved image super-resolution reconstruction model is obtained by continuously evolving the initial image super-resolution reconstruction model.

[0096] In the embodiment of the application, the optimal enhanced view subset is the optimal enhanced view of the enhanced view of the target test image in each test stage.

[0097] In the embodiment of the application, the learnable hint is an adjustable variable, which is used to align the distribution of the test data of the image super-resolution reconstruction model with the distribution of the training data.

[0098] In an optional embodiment, the learnable hint can include but is not limited to feature information such as style, color, and texture of the image. These hint information can guide the model to better restore the real details and features of the image in the reconstruction process by being part of the model input, and further improve the effect of image super-resolution reconstruction.

[0099] In the specific implementation process, the learnable hint can be trained and optimized through a deep learning algorithm, so that it can adaptively adjust its own parameters to best match the distribution of the test data. In this way, the method of the application can realize adaptive processing of different test data, and improve the generalization ability and reconstruction accuracy of the model.

[0100] In the embodiment of the application, the optimal enhanced view subset includes:

[0101] An evaluation index of the enhanced view of each stage is obtained.

[0102] ranking the evaluation indexes of the enhanced views of each stage;

[0103] selecting the optimal enhanced views of each stage as an optimal enhanced view subset.

[0104] In an optional embodiment, the enhanced views can be evaluated according to the image enhancement strategy in step S102. The evaluation indexes can include, but are not limited to, the definition, the detail richness, the color restoration degree, and the like of the image. Through quantitative analysis of these evaluation indexes, the quality of the enhanced views can be objectively reflected. In the ranking process, the enhanced views can be ranked according to the evaluation indexes, so as to select the optimal enhanced views of each stage. These optimal enhanced views constitute the optimal enhanced view subset, which is used for the subsequent image super-resolution reconstruction process. In this way, the image super-resolution reconstruction system provided by the present application can automatically select the optimal enhanced view subset, further improving the effect and accuracy of image reconstruction.

[0105] In the embodiment of the present application, establishing the image super-resolution reconstruction model comprises:

[0106] obtaining an initial image super-resolution reconstruction model, and inputting a target test image to obtain an initial high-resolution image;

[0107] performing first preprocessing on the target test image and the initial high-resolution image in the simple stage to obtain a first enhanced view;

[0108] The first enhanced view comprises a first simple test image and a first simple high-resolution image.

[0109] In the embodiment of the present application, establishing the image super-resolution reconstruction model further comprises:

[0110] performing second preprocessing on the first enhanced view in the medium stage to obtain a second enhanced view;

[0111] The first enhanced view comprises a second medium test image and a second medium high-resolution image.

[0112] In the embodiment of the present application, establishing the image super-resolution reconstruction model further comprises:

[0113] performing third preprocessing on the second enhanced view in the medium stage to obtain a third enhanced view;

[0114] The third enhanced view comprises a third complex test image and a third complex high-resolution image.

[0115] obtaining an optimal enhanced view subset of the first enhanced view, the second enhanced view, and the third enhanced view;

[0116] The preset learnable prompt is input into the image super-resolution reconstruction model combined with the optimal enhanced view subset of each stage to retrain the image super-resolution reconstruction model. As shown in Figure 2 , Figure 3

[0117] It should be noted that the first preprocessing, the second preprocessing and the third preprocessing are operations of image enhancement, screening and distribution alignment of the target test image. These preprocessing operations are aimed at simulating different image degradation conditions to generate enhanced views with different degradation degrees. By gradually enhancing and screening the images at different stages, the system can extract more useful image features and provide more stable and reliable input data for subsequent image super-resolution reconstruction. In the specific implementation process, the specific operations of the first preprocessing, the second preprocessing and the third preprocessing can be flexibly set according to actual needs, for example, can include image cropping, rotation, scaling, color adjustment and other image enhancement technologies. These operations are aimed at simulating different image degradation conditions to enhance the adaptability of the model to images with different degradation degrees.

[0118] For example, at three different stages, the optimal enhanced view subset obtained is input into the encoder together with the learnable prompt p. The average value and variance statistics of tokens embedding are calculated in the output of each transformer layer of the model encoder, as shown in Figure 4 .

[0119]

[0120] wherein μ l (T; p), is the mean and variance of the l-th layer, and T is the test data distribution.

[0121] Similarly, the statistics of the training data are calculated, θ is the pre-training model encoder parameter, and D is the training data distribution.

[0122] In an optional embodiment, the distribution alignment loss l (T; p), between the training data and the test sample is calculated by the obtained μ .

[0123]

[0124] Down-sampling is applied to the generated high-resolution image, and a down-sampling reconstruction loss

[0125] ​

[0126] In an optional embodiment, the obtained distribution alignment loss and the obtained down-sampling reconstruction loss to generate a final loss The learnable hint p is updated by minimizing the feature distribution difference, so that the test data distribution is aligned with the training data.

[0127] In an optional embodiment, the specific content of generating the final loss is:

[0128] the obtained distribution alignment loss and the obtained down-sampling reconstruction loss are combined with a certain proportion β(t). β(t) is set differently in different stages. In the simple stage, β = 5. In the medium stage, β = 10. In the complex stage, β = 15.

[0129] It should be noted that the image super-resolution reconstruction model can be established to perform higher-precision super-resolution reconstruction of images based on the optimal enhanced view subset and the learnable hint. By taking the optimal enhanced view subset as the input of the model, the image data that has been optimized can be fully utilized to extract more useful image features, thereby improving the accuracy and effect of image reconstruction. At the same time, the introduction of the learnable hint as part of the model input can further guide the model to better restore the real details and features of the image during the reconstruction process, improve the generalization ability and reconstruction accuracy of the model. This method of combining the optimal enhanced view subset and the learnable hint can realize adaptive processing of different test data and meet the needs of different application scenarios. In the subsequent steps, the system will perform higher-precision image super-resolution reconstruction based on the established image super-resolution reconstruction model to obtain clearer and more detailed image results.

[0130] S104, image super-resolution reconstruction is performed according to the image super-resolution reconstruction model after training is completed.

[0131] In the embodiment of the present application, aligning the distribution of the test data with the distribution of the training data includes establishing a loss function, the loss function including a learnable hint, and adjusting the learnable hint to align the distribution of the test data with the distribution of the training data, as shown in Figure 5 .

[0132] Specifically, the test image X test is combined with the updated hint p, and the trained image super-resolution reconstruction model obtains a high-resolution image Y.

[0133] In summary, the present application proposes an image super-resolution reconstruction method, a plurality of test stages are preset, and the plurality of test stages are connected in sequence; a target test image is obtained, and the target test image is input into the plurality of test stages; an image super-resolution reconstruction model is established, the input of the image super-resolution reconstruction model is an optimal enhanced view subset of each stage and a learnable hint; and image super-resolution reconstruction is performed according to the image super-resolution reconstruction model after training. First, by presetting a plurality of test stages and connecting them in sequence, the method of the present application can gradually optimize the image quality, ensure that each stage can effectively enhance the image, and thus improve the super-resolution effect of the final reconstructed image. Secondly, by selecting the optimal enhanced view subset of each stage as the input, the present application can maximize the use of the information of the enhanced view, reduce the interference of redundancy and noise, and further improve the quality of image reconstruction. Thirdly, the introduction of the learnable hint as part of the model input enables the method of the present application to better adapt to different test data distributions and improve the generalization ability of the model. Finally, the method of the present application combines the advantages of image enhancement and super-resolution reconstruction, can significantly improve the resolution and clarity of the image while ensuring the details and texture of the image, and provides a new solution for the image processing and computer vision fields.

[0134] In a preferred embodiment, Example 2, the present application is implemented based on SwinIR, solving the distribution alignment problem of test stages. For data augmentation, in each stage, the test image is cropped and randomly horizontally flipped to generate 12 enhanced views. In the simple stage, the cropping size ranges between [0.7, 1]; in the medium stage, the cropping size ranges between [0.5, 0.7]; and in the complex stage, the cropping size ranges between [0.3, 0.5]. In each stage, from the 12 enhanced views, 8 views are selected by applying a selection strategy, with K = 12 and N = 8. The hint is optimized using the AdamW optimizer to minimize the final combined loss, which consists of the down-sampling reconstruction loss and the distribution alignment loss. The learning rate is set to 0.004. In the simple stage, β is set to 5; in the medium stage, β is set to 10; and in the complex stage, β is set to 15. The number of hint update steps is set to 2.

[0135] The application evaluates the method using four widely recognized benchmark datasets: SET5, SET14, BSDS100, and URBAN100. SET5 contains 5 high-resolution natural images and is mainly used to evaluate the performance of super-resolution on simple and clear images; SET14 contains 14 diversified natural images, representing various scenes, providing a more challenging test for super-resolution models; BSDS100 consists of 100 images with detailed and real labels, known for its complex texture and fine structure, suitable for evaluating the performance of super-resolution on highly detailed content; URBAN100 contains 100 high-resolution images of urban scenes, with complex structures such as buildings and streets, for testing the performance of super-resolution methods in structured environments.

[0136] In particular, to verify the generalization ability and repair effect of the method in the power scene, the application introduces 218 image samples from the real power operation and maintenance environment in the experiment, combined with synthetic degradation for extended evaluation, including motion blur, Gaussian blur, JPEG compression artifacts, and extreme super-resolution requirements, etc.

[0137] The application evaluates the performance of the model using objective evaluation method. Specifically, the application uses peak signal-to-noise ratio (PSNR) and structural similarity (SSIM) indicators, and the statistical results are shown in Table 1.

[0138] Table 1: Objective evaluation results on the verification dataset

[0139]

[0140]

[0141] PSNR is mainly used to measure the pixel difference between the reconstructed image and the reference image. Its essence is a metric based on mean square error (MSE), which can reflect the degree of image distortion. The higher the value of PSNR, the smaller the difference between the reconstructed image and the reference image, and the better the image quality. SSIM, on the other hand, evaluates the similarity between images from the perspective of human visual perception. It considers changes in brightness, contrast, and structure to measure the perceived quality of images. The value of SSIM is between 0 and 1, and the closer the value is to 1, the closer the reconstructed image is to the reference image in structure and visual quality.

[0142] In all data sets, the method proposed in the present application obtains the highest or second highest PSNR and SSIM values compared with the most advanced models, proving that it can effectively face the super-resolution restoration problem of complex real-world degraded images. It is worth mentioning that in the real images of the power scene, the method proposed in the present application still achieves the best effect, further verifying the generalization ability and application value of the method proposed in the present application in the power scene.

[0143] In this embodiment, an image super-resolution reconstruction system is also provided, comprising:

[0144] A stage design module is configured to preset a plurality of test stages, and the plurality of test stages are connected in sequence.

[0145] The plurality of test stages are configured to enhance, filter and align the distribution of the target test image.

[0146] A running module is configured to obtain the target test image and input the target test image into the plurality of test stages.

[0147] A model establishing module is configured to establish an image super-resolution reconstruction model, and the input of the image super-resolution reconstruction model is an optimal enhanced view subset of each stage and a learnable hint.

[0148] The optimal enhanced view subset is an optimal enhanced view of the enhanced view of the target test image in each test stage.

[0149] The learnable hint is an adjustable variable configured to align the distribution of the test data of the image super-resolution reconstruction model with the distribution of the training data.

[0150] A reconstruction module is configured to perform image super-resolution reconstruction according to the image super-resolution reconstruction model after training.

[0151] The above-mentioned various unit modules can be embedded in or independent of the processor in the electronic device in hardware form, or can be stored in the memory in the electronic device in software form, so as to be called and executed by the processor to perform the operations corresponding to the above-mentioned various modules.

[0152] The present embodiment also provides an electronic device, which can be a terminal, and the internal structure diagram thereof can be as shown in Figure 6As shown in the figure. The electronic device includes a processor, a memory, a communication interface, a display screen and an input device connected by a system bus. Among them, the processor of the electronic device is used to provide computing and control capability. The memory of the electronic device includes non-volatile storage medium, internal memory. The non-volatile storage medium stores the operating system and the computer program. The internal memory provides an environment for the operating system and the computer program in the non-volatile storage medium to run. The communication interface of the electronic device is used for wired or wireless communication with external terminals. Wireless mode can be achieved through WIFI, operator network, NFC (near field communication) or other technologies. The computer program is executed by the processor to implement an image super-resolution reconstruction method. The display screen of the electronic device can be a liquid crystal display screen or an electronic ink display screen. The input device of the electronic device can be a touch layer overlaid on the display screen, or a key, trackball or touchpad provided on the shell of the electronic device, or an external keyboard, touchpad or mouse, etc.

[0153] The embodiment also provides a computer readable storage medium, which stores a computer program, and the computer program is executed by a processor to implement the following steps:

[0154] A plurality of test stages are connected in sequence;

[0155] The plurality of test stages are used for enhancement, screening and distribution alignment of the target test image;

[0156] The target test image is obtained and input into the plurality of test stages;

[0157] An image super-resolution reconstruction model is established, and the input of the image super-resolution reconstruction model is the optimal enhanced view subset of each stage and the learnable hint;

[0158] The optimal enhanced view subset is the optimal enhanced view of the enhanced view after the target test image is enhanced in each test stage;

[0159] The learnable hint is an adjustable variable, which is used to align the distribution of the test data of the image super-resolution reconstruction model with the distribution of the training data;

[0160] Image super-resolution reconstruction is performed according to the image super-resolution reconstruction model after training.

[0161] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present application and are not limiting. Although the present application has been described in detail with reference to the preferred embodiments, it should be understood by those skilled in the art that the technical solutions of the present application can be modified or replaced equivalently without departing from the spirit and scope of the technical solutions of the present application, which should be covered in the scope of the claims of the present application.

[0162] Those skilled in the art will appreciate that embodiments of the present application can be readily used as software, hardware, or a combination of software and hardware. In one embodiment, the present application can be implemented in software and can be stored on a computer readable medium, which can include random access memory (RAM), read only memory (ROM), magnetic disk or optical disk, or the like. The software implementation of the present application files can further be transmitted or received over a modem or network connection.

[0163] The present application is described in reference to the flowchart illustrations and / or block diagrams of methods, apparatus (systems) and computer program products according to embodiments of the application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general purpose computer, special purpose computer, embedded processing system or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart illustrations and / or block diagrams block or blocks. Figure 1 one or more functions specified in the flowchart illustrations and / or block diagrams block or blocks. Figure 1 means for performing the function of one or more of the steps in the flowchart illustrations and / or block diagrams.

[0164] These computer program instructions can also be stored in a computer readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer readable memory produce an article of manufacture including instructions which implement the flowchart illustrations and / or block diagrams block or blocks. Figure 1 one or more functions specified in the flowchart illustrations and / or block diagrams block or blocks. Figure 1 means for performing the function of one or more of the steps in the flowchart illustrations and / or block diagrams.

[0165] These computer program instructions can also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the flowchart illustrations and / or block diagrams block or blocks. Figure 1 one or more functions specified in the flowchart illustrations and / or block diagrams block or blocks. Figure 1 means for performing the function of one or more of the steps in the flowchart illustrations and / or block diagrams.

[0166] While preferred embodiments of the application have been described, modifications and alterations thereto can occur to those skilled in the art upon reading the preceding description. It is intended to include all such modifications and alterations insofar as they come within the scope of the basic inventive concepts disclosed herein.

[0167] Obviously, many modifications and variations of the present application are possible in light of the above teachings. It is, therefore, to be understood that within the scope of the appended claims and their equivalents, the application can be practiced otherwise than as specifically described.

Claims

1. A method for image super-resolution reconstruction, characterized in that: include: Presetting a plurality of test phases, wherein the plurality of test phases are connected in sequence; The several test stages are all used for enhancing, screening and distribution alignment of target test images; Acquire a target test image, and input the target test image into the plurality of test stages; Establishing an image super-resolution reconstruction model, wherein the input of the image super-resolution reconstruction model is the optimal enhanced view subset and learnable hints at each stage; The optimal enhanced view subset is the optimal enhanced view of the enhanced view after the target test image is enhanced in each test phase; The learnable hint is an adjustable variable used to align the distribution of test data of the image super-resolution reconstruction model with the distribution of training data; Image super-resolution reconstruction is performed based on the trained image super-resolution reconstruction model.

2. The image super-resolution reconstruction method according to claim 1, wherein: The preset test phases are connected in sequence and include: The plurality of test stages include a simple stage, a medium stage and a complex stage; The output of the simple stage is the input of the medium stage, and the output of the medium stage is the input of the complex stage.

3. The image super-resolution reconstruction method according to claim 2, wherein: The establishing of the image super-resolution reconstruction model comprises: Obtain an initial image super-resolution reconstruction model and input the target test image to obtain an initial high-resolution image; Performing a first preprocessing on the target test image and the initial high-resolution image in a simple stage to obtain a first enhanced view; The first enhanced view includes a first simple test image and a first simple high-resolution image.

4. The image super-resolution reconstruction method according to claim 3, wherein: The establishing of the image super-resolution reconstruction model further comprises: performing a second preprocessing on the first enhanced view at the intermediate stage to obtain a second enhanced view; The first enhanced view includes a second medium test image and a second medium high-resolution image.

5. The image super-resolution reconstruction method according to claim 4, wherein: The establishing of the image super-resolution reconstruction model further comprises: performing a third pre-processing on the second enhanced view at the intermediate stage to obtain a third enhanced view; The third enhanced view includes a third complex test image and a third complex high-resolution image; Acquire an optimal enhanced view subset of the first enhanced view, the second enhanced view, and the third enhanced view; Preset learnable cues and retrain the image super-resolution reconstruction model based on the optimal enhanced view subset at each stage.

6. The image super-resolution reconstruction method according to claim 5, wherein: The optimal enhanced view subset includes: Get evaluation metrics for the enhanced views at each stage; Ranking the evaluation metrics of the enhanced views at each stage; The optimal enhanced view in each stage is selected as the optimal enhanced view subset.

7. The image super-resolution reconstruction method according to claim 6, wherein: The aligning of the distribution of the test data with the distribution of the training data includes establishing a loss function, wherein the loss function includes a learnable hint, and the distribution of the test data is aligned with the distribution of the training data by adjusting the learnable hint.

8. An image super-resolution reconstruction system, applying the method according to any one of claims 1 to 7, characterized in that: include: A stage design module, used for presetting a plurality of test stages, wherein the plurality of test stages are connected in sequence; The several test stages are all used for enhancing, screening and distribution alignment of target test images; An operating module, configured to obtain a target test image and input the target test image into the plurality of test stages; A model building module, configured to build an image super-resolution reconstruction model, wherein the input of the image super-resolution reconstruction model is the optimal enhanced view subset and learnable hints at each stage; The optimal enhanced view subset is the optimal enhanced view of the enhanced view after the target test image is enhanced in each test phase; The learnable hint is an adjustable variable used to align the distribution of test data of the image super-resolution reconstruction model with the distribution of training data; The reconstruction module is used to perform image super-resolution reconstruction based on the trained image super-resolution reconstruction model.

9. An electronic device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the image super-resolution reconstruction method according to any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of an image super-resolution reconstruction method according to any one of claims 1 to 7 are implemented.

Images