Super-division method and system based on hierarchical fusion residual connection network

Abstract

The invention provides a super-division method and system based on a hierarchical fusion residual connection network, and the method comprises the steps: carrying out the dedegradation of original image information, extracting features, employing a U-Net network as a backbone network, and introducing the hierarchical fusion residual connection network; according to the method, information fusion of original features is effectively carried out on feature extraction in a backbone network coding stage and feature reduction in a decoding stage, enhanced details of original textures are supplemented for high-quality image generation of a super-resolution network, and super-resolution improvement is carried out by better fitting an original image; therefore, the natural degree, the texture details, the authenticity and the image quality effect of the super-resolution image are better improved. Through obtaining the image from the open source data set and carrying out a contrast experiment, it is proved that the super-resolution image obtained through the method is clearer and more natural, and the texture details, authenticity and image quality effect of the image are all obviously improved.

Description

Technical Field

[0001] The present invention belongs to the technical field of computer vision. Specifically, it relates to a super-resolution method and system based on a hierarchical fusion residual connection network. Background Art

[0002] The high-quality image super-resolution algorithm is an important technology in the fields of computer low-level vision and image enhancement. Its main purpose is to generate high-quality and high-resolution images, and at the same time improve the picture quality details and image information of the original low-resolution images. The high-quality image super-resolution technology mainly includes low-resolution image restoration, old image restoration, video restoration, etc. Image super-resolution has become the core application in the field of image enhancement. However, in low-quality real-world images, pictures may not be recognizable due to too low resolution. Through the high-quality image super-resolution technology, low-quality images can be upgraded to high-quality ones, restoring more detailed information to obtain a more accurate texture effect of the images. At the same time, in the super-resolution application in image restoration, in the fields of image restoration and picture quality enhancement, super-resolution processing needs to be performed on low-quality images. The super-resolution technology can increase the picture resolution and reconstruct detailed information, improving the authenticity and accuracy of the model to achieve better effects. However, the naturalness of super-resolved images, the texture details, authenticity, and picture quality effects of the existing technologies still need to be improved.

[0003] It should be noted that the information disclosed in the above background art section is only used to enhance the understanding of the background of the present disclosure. Summary of the Invention

[0004] In order to overcome the shortcomings and deficiencies of the existing technologies, the purpose of the present invention is to provide a super-resolution method and system based on a hierarchical fusion residual connection network to improve the naturalness, texture details, authenticity, and picture quality effects of super-resolved images.

[0005] The purpose of the present invention is achieved by the following technical solutions: According to one aspect of the present invention, there is provided a super-resolution method based on a hierarchical fusion residual connection network, the method comprising: Calculating the mean value of the original input features of the image, performing mean subtraction processing, and obtaining controllable output features by weighting the control factor values; The encoder of the backbone network performs multiple downsampling operations on the controllable output features to obtain the highest-level semantic features; The hierarchical fusion residual connection network takes the original input features as input and performs downsampling operations synchronized with the backbone network; The decoder of the backbone network starts from the highest-level semantic features and restores the semantic feature information through corresponding multiple upsampling operations; Based on the decoder stage, the hierarchical fusion residual connection network starts from the highest-level semantic features and performs upsampling operations synchronized with the backbone network; in each upsampling operation, the semantic features of the backbone network are first added to the downsampling features of the corresponding layer of the hierarchical fusion residual connection network in the encoder stage, and then after the upsampling operation, they are added to the upsampling features of the corresponding layer for the second time. After completing the upsampling operation layer by layer, the semantic features of the original image size are obtained; Use convolution operations to restore the number of feature channels of the semantic features of the original size to three channels, and improve the resolution by a multiple of the size through upsampling operations to obtain the restored multiple super-resolution processed image.

[0006] Based on the foregoing solution, the method further includes: connecting the semantic features obtained by each downsampling operation of the hierarchical fusion residual connection network with the semantic features of the corresponding layer of the backbone network, and using the connected fusion features as the input of the next layer of the backbone network.

[0007] Based on the foregoing solution, after obtaining the restored multiple super-resolution processed image, it further includes: Improve the resolution by a multiple of the size of the original input features through upsampling operations to obtain the multiple input processed image; Add the multiple super-resolution processed image and the multiple input processed image, and then add the mean value calculated by the mean value to finally obtain the optimized multiple super-resolution processed image.

[0008] Based on the foregoing solution, the upsampling operation is specifically a bilinear interpolation upsampling operation.

[0009] Based on the foregoing solution, the multiple is 2 times or 4 times.

[0010] Based on the foregoing solution, the multiple specifically is 5 times.

[0011] Based on the foregoing solution, the downsampling operation is specifically an average pooling downsampling operation.

[0012] Based on the foregoing solution, the backbone network is a U-Net network.

[0013] Based on the foregoing solution, calculating the mean value of the original input features of the image, performing mean subtraction processing, and controlling the numerical weighting of the factor to obtain the controllable output features are performed by the degradation control factor module, and specifically include: For the input image array X in , whose feature map size is H x W x C in , calculate the channel mean value for each of the RGB three channels and subtract the channel mean value to obtain the standardized result of the input data; Use a degradation factor Create a one-dimensional matrix the same size as the input array, with the value of the entire matrix being the preset degradation factor; Through different degradation factors V 1 、V 2 、V 3 Affect the output X of the degradation control factor module for different types of noise DFCM ; The output X of the degradation control factor module DFCM The calculation formula is as follows:

[0014] In the formula, X in is the input image array, mean_reduce() is the mean subtraction process, F 1 (), F 2 (), F 3 () represent the control of the noise level for deblurring, denoising, and decompression respectively. The degradation factors V 1 、V 2 、V 3 are the scalar values of the true degradation levels corresponding to blurring, noise, and compression respectively; In the formula: , where a is a heuristic numerical scalar, fixed through training, and has the same size as X in ;

[0015] According to another aspect of the present invention, a super-resolution system based on a hierarchical fusion residual connection network is provided, applicable to the above-mentioned super-resolution method based on a hierarchical fusion residual connection network, including the following modules: A degradation control factor module, used to calculate the mean value of the original input features of the image, perform mean subtraction processing, and obtain controllable output features through numerical weighting of the control factor; A backbone network module, including an encoder and a decoder. The encoder performs multiple downsampling operations on the controllable output features to obtain the highest-level semantic features; the decoder starts from the highest-level semantic features and restores the semantic feature information through corresponding multiple upsampling operations; A hierarchical fusion residual connection network module, used to take the original input features as input and perform downsampling operations synchronized with the backbone network; based on the decoder stage, the hierarchical fusion residual connection network starts from the highest-level semantic features and performs upsampling operations synchronized with the backbone network; in each upsampling operation, the semantic features of the backbone network are first added to the downsampling features of the corresponding layer of the hierarchical fusion residual connection network in the encoder stage, and then after upsampling, they are added to the upsampling features of the corresponding layer for the second time, and the semantic features of the original size of the image are obtained layer by layer after completing the upsampling operations; The multiple upsampling module is used to restore the number of feature channels of the semantic features of the original size to three channels using a convolution operation, and improve the multiple resolution of the size through an upsampling operation to obtain a restored multiple super-resolution processed image.

[0016] Further, the system further includes a result optimization module, which is used to improve the multiple resolution of the size of the original input features through an upsampling operation to obtain a multiple input processed image; add the multiple super-resolution processed image and the multiple input processed image, and then add the mean value calculated by the mean value to finally obtain an optimized multiple super-resolution processed image.

[0017] The present invention has the following advantages and effects compared with the prior art: The image super-resolution method based on the hierarchical fusion residual connection network of the present invention degrades and extracts features from the original picture information, uses the U-Net network as the backbone network and introduces the hierarchical fusion residual connection network, effectively fuses the original feature information for the feature extraction in the encoding stage and the feature restoration in the decoding stage of the backbone network, supplements the enhanced details of the original texture for the high-quality image generation of the super-resolution network, and is more suitable for the original image for super-resolution improvement, so as to better improve the naturalness, texture details, authenticity and picture quality effect of the super-resolution image. Through comparative experiments on images obtained from open-source data sets, it is proved that the super-resolution images obtained by this method are clearer and more natural, and the texture details, authenticity and picture quality effect of the images are significantly improved.

[0018] It should be understood that the above general description and the following detailed description are only exemplary and explanatory, and cannot limit the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] The accompanying drawings herein are incorporated into the specification and form a part of the specification, showing embodiments consistent with the present disclosure, and are used together with the specification to explain the principles of the present disclosure. Obviously, the accompanying drawings in the following description are only some embodiments of the present disclosure, and those of ordinary skill in the art can obtain other drawings based on these drawings without creative efforts. In the drawings: Figure 1It shows a partial process of the super-resolution method based on the hierarchical fusion residual connection network in Embodiment 3; in the figure: Concat is the connection operation, Convolution is the convolution operation, Add is the summation operation, Flow is the flow direction, AvgPool is the average pooling, Bilinear Upsample is the bilinear interpolation upsampling, Hierarchical fusion residual module is the hierarchical fusion residual connection network, Degradation factor control module is the degradation control factor module, and 2x upsampling is the 2-fold resolution improvement.

[0020] Figure 2 It shows the overall process of the super-resolution method based on the hierarchical fusion residual connection network in Embodiment 3.

[0021] Figure 3 It shows one of the comparisons of the super-resolution processing results between the present invention and the Real-ESRGAN model in Embodiment 5, where A is the original image, B is the super-resolution processing result of this method, and C is the super-resolution processing result of the Real-ESRGAN model.

[0022] Figure 4 It shows another comparison of the super-resolution processing results between the present invention and the Real-ESRGAN model in Embodiment 5, where A is the original image, B is the super-resolution processing result of this method, and C is the super-resolution processing result of the Real-ESRGAN model. Detailed implementation manners

[0023] Next, the implementation schemes of the present invention will be described clearly and completely in conjunction with the embodiments. The described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. The following description of at least one exemplary embodiment is actually only illustrative and in no way restricts the present invention and its application or use. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts fall within the scope of protection of the present invention.

[0024] The flowcharts shown in the drawings are only illustrative and not necessarily include all the contents and operations / steps, nor are they necessarily executed in the described order. For example, some operations / steps can be decomposed, and some operations / steps can be combined or partially combined. Therefore, the actual execution order may change according to the actual situation.

[0025] Embodiment 1

[0026] This embodiment provides a super-resolution method based on the hierarchical fusion residual connection network, and the method includes the following steps: S1. Calculate the mean value of the original input features of the image, perform mean subtraction processing, and weight the control factor values to obtain controllable output features.

[0027] S2. The encoder of the backbone network performs 5 downsampling operations on the controllable output features to obtain the highest-level semantic features.

[0028] S3. The hierarchical fusion residual connection network takes the original input features as input and performs downsampling operations synchronized with the backbone network.

[0029] S4. The decoder of the backbone network starts from the highest-level semantic features and restores the semantic feature information through corresponding 5 upsampling operations.

[0030] S5. Based on the decoder stage, the hierarchical fusion residual connection network starts from the highest-level semantic features and performs upsampling operations synchronized with the backbone network; in each upsampling operation, the semantic features of the backbone network are first added to the downsampling features of the corresponding layer of the hierarchical fusion residual connection network in the encoder stage, then an upsampling operation is performed and then added to the upsampling features of the corresponding layer for the second time. After completing the upsampling operations layer by layer, semantic features of the original size of the image are obtained.

[0031] S6. Use convolution operations to restore the number of feature channels of the semantic features of the original size to three channels, and improve the resolution by a multiple through upsampling operations to obtain the restored multiple super-resolution processed image.

[0032] In this embodiment, the backbone network used is specifically the U-Net network, the downsampling operation is specifically the average pooling downsampling operation, the upsampling operation is specifically the bilinear interpolation upsampling operation, and the multiple is 2 times or 4 times.

[0033] Embodiment 2

[0034] This embodiment provides a super-resolution method based on a hierarchical fusion residual connection network, and the method includes the following steps: S1. Calculate the mean value of the original input features of the image, perform mean subtraction processing, and weight the control factor values to obtain controllable output features.

[0035] S2. The encoder of the backbone network performs 5 downsampling operations on the controllable output features to obtain the highest-level semantic features.

[0036] S3. The hierarchical fusion residual connection network takes the original input features as input and performs downsampling operations synchronized with the backbone network; the semantic features obtained by each downsampling operation of the hierarchical fusion residual connection network are connected to the semantic features of the corresponding layer of the backbone network, and the connected fusion features are used as the input of the next layer of the backbone network.

[0037] S4. The decoder of the backbone network starts from the highest-level semantic features and restores the semantic feature information through corresponding 5 upsampling operations.

[0038] S5. Based on the decoder stage, the hierarchical fusion residual connection network starts from the highest-level semantic features and performs upsampling operations synchronized with the backbone network; in each upsampling operation, the semantic features of the backbone network are first added to the downsampling features of the corresponding layer of the hierarchical fusion residual connection network in the encoder stage, and then after the upsampling operation, they are added to the upsampling features of the corresponding layer for the second time. After completing the upsampling operations layer by layer, the semantic features of the original image size are obtained.

[0039] S6. Use convolution operations to restore the number of feature channels of the semantic features of the original size to three channels, and improve the resolution by 2 times through upsampling operations to obtain the restored 2x super-resolution processed image.

[0040] In this embodiment, the backbone network used is specifically the U-Net network, the downsampling operation is specifically the average pooling downsampling operation, and the upsampling operation is specifically the bilinear interpolation upsampling operation.

[0041] Based on Embodiment 1, in this embodiment, the hierarchical fusion residual connection network performs information fusion of the original features for both feature extraction in the encoding stage and feature restoration in the decoding stage of the backbone network, supplements enhanced details of the original texture for high-quality image generation of the super-resolution network, and is more suitable for improving the super-resolution close to the original image.

[0042] Embodiment 3

[0043] This embodiment provides a super-resolution method based on a hierarchical fusion residual connection network, and the method includes the following steps: S1. Calculate the mean value of the original input features of the image, perform mean subtraction processing, and weight the control factor values to obtain controllable output features.

[0044] In this embodiment, the calculation of the mean value of the original input features of the image, performing mean subtraction processing, and weighting the control factor values to obtain controllable output features are carried out by the degradation control factor module, specifically including S11 to S13: S11. For the input image array X in , whose feature map size is H x W x C in , calculate the channel mean value for each of the RGB three channels and subtract the channel mean value to obtain the normalized result of the input data; S12. Use the degradation factor to create a one-dimensional matrix of the same size as the input array, and the values of the entire matrix are all the preset degradation factors; S13. Through different degradation factors V1 , V 2 , V 3 affects the output X of the degradation control factor module for different types of noise DFCM ; The output X of the degradation control factor module DFCM has the following calculation formula:

[0045] In the formula, X in is the input image array, mean_reduce() is the mean subtraction process, F 1 (), F 2 (), F 3 (), respectively, represent the control of the noise levels for deblurring, denoising, and decompression. The degradation factors V 1 , V 2 , V 3 are the scalar values corresponding to the true degradation levels of blurring, noise, and compression, respectively; In the formula: , where a is a heuristic numerical scalar, fixed through training, and has the same size as X in ;

[0046] When the input image is fixed, that is, the characteristic value of mean_reduce(X in ) is fixed. If V 1 , V 2 , V 3 are 0, it means that the input image has no noise degradation and is a high-definition lossless image. At this time, the degradation control factor module removes the lowest degree of various noises, that is, F 1 (V 1 ) + F 2 (V 2 ) + F 3 (V 3 ) is 0. At this time, X DFCM = mean_reduce(X in ), and it has the lowest impact on the output value of the degradation control factor module. If V 1 , V 2 , V 3 are 1, it means that the noise levels of blurring, noise, and compression are the highest, and the input image has severe noise degradation of blurring, noise, and compression. At this time, the degradation control factor module removes the highest degree of various noises, that is, X DFCM = mean_reduce(X in ) + F 1 (1) + F 2 (1) + F 3 (1), and it has the highest impact on the output value of the degradation control factor module. From the above content, it can be seen that by adjusting V1 , V 2 , V 3 With these three values, the module can control the output to achieve a reasonable level of denoising and super-resolution image processing effect.

[0047] S2. The encoder of the backbone network performs 5 downsampling operations on the controllable output features to obtain the highest-level semantic features.

[0048] S3. The hierarchical fusion residual connection network takes the original input features as input and performs downsampling operations synchronized with the backbone network; the semantic features obtained from each downsampling operation of the hierarchical fusion residual connection network are concatenated with the semantic features of the corresponding layer of the backbone network, and the concatenated fusion features are used as the input for the next layer of the backbone network.

[0049] S4. The decoder of the backbone network starts from the highest-level semantic features and restores the semantic feature information through corresponding 5 upsampling operations.

[0050] S5. Based on the decoder stage, the hierarchical fusion residual connection network starts from the highest-level semantic features and performs upsampling operations synchronized with the backbone network; in each upsampling operation, the semantic features of the backbone network are first added to the downsampling features of the corresponding layer of the hierarchical fusion residual connection network in the encoder stage, then after the upsampling operation, they are secondarily added to the upsampling features of the corresponding layer, and after completing the upsampling operation layer by layer, the semantic features of the original image size are obtained; S6. Use convolution operations to restore the number of feature channels of the semantic features of the original size to three channels, and through upsampling operations, increase the resolution by 2 times to obtain the restored 2-fold super-resolution processed image.

[0051] As Figure 1 shown, the original input image array X in passes through the Degradation Control Factor Module (DFCM) to obtain the controllable output feature X DFCM , which serves as the input for the hierarchical fusion residual connection network and the backbone network; X DFCM is fed into the U-Net backbone network and undergoes 5 average pooling (AvgPool) downsampling operations to obtain the feature map X Layer6 of the last layer, Layer6; X DFCM is synchronously fed into the hierarchical fusion residual connection network for 5 average pooling (AvgPool) downsampling operations to obtain X avgpool5 ; X Layer6 is added to X avgpool5 to obtain the fused feature map X Layer6_fusion ; the feature map X Layer6_fusion is obtained through bilinear interpolation upsampling operation to get X up6_fusion ; X avgpool5 is X HFRM5 before pooling, XHFRM5 Sum with the fifth-layer feature map X of the backbone network Layer5 to obtain X Layer5_mix ; Take X up6_fusion and X Layer5_mix sum them to get; After performing such feature information fusion operations layer by layer, X HFRM_out is obtained; Then use convolution operation to restore the number of feature channels of the semantic features of the original size to three channels, and through upsampling operation, the resolution is doubled, and the restored 2x super-resolution processed map X HFRM_out_2X is obtained.

[0052] S7. Perform upsampling operation on the original input features to double the resolution of the size, and obtain a 2x input processed map; Add the 2x super-resolution processed map and the 2x input processed map, and then add the mean value calculated by the mean value calculation to finally obtain an optimized 2x super-resolution processed map.

[0053] As Figure 2 shown, in this embodiment, the original image array X in of the original input is directly subjected to upsampling operation to double the resolution of the size (2x upsampling) to obtain a 2x input processed map X in_2X ; Take X HFRM_out_2X and X in_2X add them, and add the channel mean value subtracted in the degradation control factor module to finally obtain an optimized 2x super-resolution processed map.

[0054] The image super-resolution method based on the hierarchical fusion residual connection network of the present invention degrades and extracts features from the original picture information, uses the U-Net network as the backbone network and introduces the hierarchical fusion residual connection network, effectively performs information fusion of the original features for both feature extraction in the encoding stage and feature restoration in the decoding stage of the backbone network, supplements enhanced details of the original texture for high-quality image generation of the super-resolution network, and better fits the original image for super-resolution improvement, thereby better improving the naturalness, texture details, authenticity and picture quality effect of the super-resolution image.

[0055] Example 4

[0056] This embodiment provides a super-resolution system based on a hierarchical fusion residual connection network, including the following modules: A degradation control factor module for calculating the mean value of the original input features of the image, performing mean subtraction processing, and weighted control factor values to obtain controllable output features.

[0057] A backbone network module, including an encoder and a decoder. The encoder performs multiple downsampling operations on the controllable output features to obtain the highest-layer semantic features; the decoder starts from the highest-layer semantic features and restores the semantic feature information through corresponding multiple upsampling operations.

[0058] The hierarchical fusion residual connection network module is used to take the original input features as input and perform downsampling operations synchronized with the backbone network; based on the decoder stage, the hierarchical fusion residual connection network starts from the highest-level semantic features and performs upsampling operations synchronized with the backbone network; in each upsampling operation, the semantic features of the backbone network are first added to the downsampling features of the corresponding layer of the hierarchical fusion residual connection network in the encoder stage, and then after the upsampling operation, they are added to the upsampling features of the corresponding layer for the second time. After completing the upsampling operation layer by layer, semantic features of the original image size are obtained.

[0059] The multiple upsampling module is used to restore the number of feature channels of the semantic features of the original size to three channels using convolution operations, and improve the resolution by a multiple of the size through upsampling operations to obtain the restored multiple super-resolution processed image.

[0060] Furthermore, the system can also include a result optimization module, which is used to improve the resolution by a multiple of the size of the original input features through upsampling operations to obtain a multiple input processed image; add the multiple super-resolution processed image and the multiple input processed image, and then add the result to the mean value calculated by the mean value, and finally obtain an optimized multiple super-resolution processed image.

[0061] In this embodiment, the backbone network used is specifically the U-Net network, multiple times is specifically 5 times, the downsampling operation is specifically the average pooling downsampling operation, the upsampling operation is specifically the bilinear interpolation upsampling operation, and the multiple is 2 times or 4 times.

[0062] Example 5

[0063] This embodiment provides the performance test results of the super-resolution method based on the hierarchical fusion residual connection network in Embodiment 3 of the present invention.

[0064] Images are obtained from the open-source dataset, and the same images are super-resolved using the super-resolution method of the present invention and the Real-ESRGAN model respectively. Figure 3 and Figure 4 Shows the effect comparison between the method of the present invention and the Real-ESRGAN model at the same magnification factor, where A is the original image, B is the super-resolution processing result of the method of the present invention, and C is the super-resolution processing result of the Real-ESRGAN model. It can be seen that the super-resolution method of the present invention is more natural than the prior art, and the texture details, authenticity, and image quality effect of the image are better.

[0065] In summary, the image super-resolution method based on the hierarchical fusion residual connection network of the present invention degrades and extracts features from the original picture information. The U-Net network is used as the backbone network and the hierarchical fusion residual connection network is introduced, effectively performing information fusion of the original features for both feature extraction in the encoding stage and feature restoration in the decoding stage of the backbone network, supplementing enhanced details of the original texture for high-quality image generation of the super-resolution network, and improving the super-resolution more closely to the original image, thereby better improving the naturalness, texture details, authenticity, and image quality effect of the super-resolution image. Through comparative experiments using images obtained from an open-source dataset, it is proven that the super-resolution images obtained by this method are clearer and more natural, and there are obvious improvements in the texture details, authenticity, and image quality effect of the images.

[0066] Other embodiments of the present disclosure will be readily apparent to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of the present disclosure that follow the general principles of the present disclosure and include common general knowledge or conventional technical means in the technical field not disclosed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of the present disclosure are pointed out by the following claims.

[0067] It should be understood that the present disclosure is not limited to the exact structures described above and shown in the drawings, and various modifications and changes can be made without departing from its scope. The scope of the present disclosure is only limited by the appended claims.

Claims

1. A super-resolution method based on a hierarchical fusion residual connection network, characterized in that: The method comprises: The original input features of the image are averaged, and the mean is subtracted and the control factor is numerically weighted to obtain controllable output features; The encoder of the backbone network performs multiple downsampling operations on the controllable output features to obtain the highest level semantic features; The hierarchical fusion residual connection network takes the original input features as input and performs downsampling operations synchronized with the backbone network; The decoder of the backbone network starts from the highest level semantic features and recovers the semantic feature information through corresponding multiple upsampling operations; Based on the decoder stage, the hierarchical fusion residual connection network starts from the highest level semantic features and performs upsampling operations synchronously with the backbone network; in each upsampling operation, the semantic features of the backbone network are first added with the down-sampled features of the corresponding layer of the hierarchical fusion residual connection network in the encoder stage, and then the up-sampled operation is performed and then the up-sampled features of the corresponding layer are added for the second time. After completing the upsampling operation layer by layer, the semantic features of the original size of the image are obtained; The convolution operation is used to restore the number of feature channels of the semantic features of the original size to three channels, and the resolution is increased by a multiple of the size through an upsampling operation to obtain a restored multiple super-resolution processing image.

2. According to claim 1, the super-resolution method based on the hierarchical fusion residual connection network is characterized in that: The method also includes: connecting the semantic features obtained by each downsampling operation of the hierarchical fusion residual connection network with the semantic features of the corresponding layer of the backbone network, and using the connected fusion features as input to the next layer of the backbone network.

3. The super-resolution method based on hierarchical fusion residual connection network according to claim 1, characterized in that: After obtaining the restored multiple super-resolution processing graph, the method further includes: The original input features are upsampled to a multiple of the size to increase the resolution, thus obtaining a multiple input processing graph. The multiple super-resolution processing map and the multiple input processing map are added together, and then added to the mean obtained by the mean calculation, and finally the optimized multiple super-resolution processing map is obtained.

4. The super-resolution method based on a hierarchical fusion residual connection network according to claim 1 or claim 3, characterized in that: The upsampling operation is specifically a bilinear interpolation upsampling operation.

5. The super-resolution method based on hierarchical fusion residual connection network according to claim 1 or claim 3, characterized in that: The multiple is 2 times or 4 times.

6. The super-resolution method based on hierarchical fusion residual connection network according to claim 1, characterized in that: The multiple times is specifically 5 times.

7. The super-resolution method based on hierarchical fusion residual connection network according to claim 1, characterized in that: The downsampling operation is specifically an average pooling downsampling operation.

8. The super-resolution method based on hierarchical fusion residual connection network according to claim 1, characterized in that: The backbone network is a U-Net network.

9. The super-resolution method based on hierarchical fusion residual connection network according to claim 1, characterized in that: The original input features of the image are averaged, and mean subtraction is performed, and the control factor numerical weighting is performed to obtain the controllable output features, which is performed by the degradation control factor module, and specifically includes: For the input image array X in , whose feature map size is H x W x C in , calculate the channel mean of the three channels of RGB and subtract the channel mean to obtain the standardized result of the input data; Use degradation factor Create a one-dimensional matrix of the same size as the input array, and the values ​​of the entire matrix are all preset degradation factors; The degradation control factor module output X for different types of noise is affected by different degradation factors V1, V2, and V3. DFCM ; The degradation control factor module outputs X DFCM The calculation formula is as follows: ； Where, X in is the input image array, mean_reduce() is the mean reduction process, F1(), F2(), F3() represent the noise degree control for deblurring, de-noising, and decompression respectively, and the degradation factors V1, V2, and V3 are the actual degradation degree scalars corresponding to blurring, noise, and compression respectively; Where: , where a is a heuristic numerical scalar, fixed through training, and has the same size as X in Consistent.

10. A super-resolution system based on a hierarchical fusion residual connection network, applicable to the super-resolution method based on a hierarchical fusion residual connection network according to any one of claims 1 to 9, characterized in that: Includes the following modules: The degradation control factor module is used to calculate the mean of the original input features of the image, and perform mean subtraction processing and control factor numerical weighting to obtain controllable output features; The backbone network module includes an encoder and a decoder. The encoder performs multiple downsampling operations on the controllable output features to obtain the highest-level semantic features. The decoder starts from the highest level semantic features and recovers the semantic feature information through corresponding multiple upsampling operations; Hierarchical fusion residual connection network module, which takes the original input features as input and performs downsampling operations synchronized with the backbone network; Based on the decoder stage, the hierarchical fusion residual connection network starts from the highest level semantic features and performs upsampling operations synchronously with the backbone network; in each upsampling operation, the semantic features of the backbone network are first added with the down-sampled features of the corresponding layer of the hierarchical fusion residual connection network in the encoder stage, and then the up-sampled operation is performed and then the up-sampled features of the corresponding layer are added for the second time. After completing the upsampling operation layer by layer, the semantic features of the original size of the image are obtained; The multiple upsampling module is used to restore the number of feature channels of the semantic features of the original size to three channels by using the convolution operation, and to improve the resolution by a multiple of the size by the upsampling operation to obtain the restored multiple super-resolution processing image.

Images