Image reconstruction method, model training method, device, equipment, medium and product

Abstract

The present invention provides an image reconstruction method, a model training method, an apparatus, a device, a medium and a product, which can be applied to the field of image super-resolution reconstruction. The image reconstruction method includes: extracting features of a degraded image with quality loss to obtain underlying image features that characterize underlying details of the image and implicit degradation features that characterize image degradation information; using an implicit degradation feature regulator to expand the implicit degradation features at the channel level and the spatial level respectively; integrating the expanded implicit degradation features into the underlying image features to obtain channel image features and spatial image features that integrate image degradation information respectively; performing feature fusion of the channel image features and the spatial image features and introducing the underlying image features to restore high-frequency details of the degraded image; inputting the image features obtained after detail restoration into an image super-resolution module, and outputting a reconstructed target image, wherein the resolution of the target image is higher than that of the degraded image.

Description

Technical Field

[0001] The present invention relates to the field of image super-resolution reconstruction, and more specifically, to an image reconstruction method, a model training method, a device, equipment, a medium and a product. Background Art

[0002] Blind image super-resolution refers to super-resolving a low-resolution image to obtain a high-resolution image without knowing the specific cause of image degradation. Related technologies extract the implicit feature representation of degradation information in low-resolution images and integrate the implicit degradation features into the low-resolution image features, thereby generating a corresponding high-resolution image based on the low-resolution image features that integrate the degradation information.

[0003] In the process of realizing the concept of the present invention, the inventors found that there are at least the following problems in the related art: in the process of integrating implicit degradation features into low-resolution image features, the related art achieves feature integration by stacking adaptation units, but the above-mentioned integration method has the problems of insufficient feature adaptation and large amount of calculation, resulting in unsatisfactory super-resolution effect of the image and low image reconstruction efficiency. Summary of the invention

[0004] In view of this, the present invention provides an image reconstruction method, a model training method, an apparatus, a device, a medium and a product.

[0005] One aspect of the present invention provides an image reconstruction method, comprising: extracting features from a degraded image with quality loss to obtain underlying image features representing underlying details of the image and implicit degradation features representing image degradation information; using an implicit degradation feature regulator to expand the implicit degradation features at the channel level and the spatial level respectively; integrating the expanded implicit degradation features into the underlying image features to obtain channel image features and spatial image features that incorporate the image degradation information respectively; fusing the channel image features and the spatial image features and introducing the underlying image features to restore high-frequency details of the degraded image; inputting the image features obtained after detail restoration into an image super-resolution module to output a reconstructed target image, wherein the resolution of the target image is higher than that of the degraded image.

[0006] According to an embodiment of the present invention, the implicit degradation feature regulator is used to expand the implicit degradation feature at the channel level and the spatial level respectively, including: performing a global average pooling operation on the implicit degradation feature in the channel conversion branch of the implicit degradation feature regulator; sending the pooling result to the fully connected layer of the channel conversion branch for nonlinear transformation to obtain the channel degradation feature after the channel level expansion of the implicit degradation feature; using a sub-pixel convolution layer in the spatial conversion branch of the implicit degradation feature regulator to upsample the implicit degradation feature to the same size as the degraded image; inputting the upsampled feature map into the convolution layer of the spatial conversion branch to obtain the spatial degradation feature after the spatial level expansion of the implicit degradation feature.

[0007] According to an embodiment of the present invention, the above-mentioned integrated extended implicit degradation feature into the above-mentioned underlying image feature to obtain a channel image feature and a spatial image feature respectively integrating the above-mentioned image degradation information includes: inputting the above-mentioned channel degradation feature and the above-mentioned underlying image feature into the channel adaptation branch of the implicit degradation feature adaptation unit to integrate the above-mentioned image degradation information into the above-mentioned underlying image feature to obtain the above-mentioned channel image feature; inputting the above-mentioned spatial degradation feature and the above-mentioned underlying image feature into the spatial adaptation branch of the implicit degradation feature adaptation unit to integrate the above-mentioned image degradation information into the above-mentioned underlying image feature to obtain the above-mentioned spatial image feature.

[0008] According to an embodiment of the present invention, the channel adaptation branch includes a global pooling layer, a fully connected layer and a depthwise separable convolutional layer, and the channel degradation features and the underlying image features are input into the channel adaptation branch of the implicit degradation feature adaptation unit to integrate the image degradation information into the underlying image features to obtain the channel image features, including: using the global pooling layer to adjust the shape of the underlying image features; inputting the channel degradation features and the pooled underlying image features into the two fully connected layers; adding the output results of the two fully connected layers element-wise; normalizing the added features using a normalization function; element-wise multiplying the normalized features with the underlying image features to integrate the normalized features into the underlying image features; and using the depthwise separable convolutional layer to perform a nonlinear transformation on the fused features to obtain the channel image features.

[0009] According to an embodiment of the present invention, the above-mentioned spatial degradation features and the above-mentioned underlying image features are input into the spatial adaptation branch of the implicit degradation feature adaptation unit to integrate the above-mentioned image degradation information into the above-mentioned underlying image features to obtain the above-mentioned spatial image features, and the above-mentioned spatial image features are obtained, including: using the two convolutional layers in the above-mentioned spatial adaptation branch to modulate the above-mentioned spatial degradation features; using a normalization function to normalize the modulated features; and performing element-wise multiplication of the normalized features and the above-mentioned underlying image features to integrate the normalized features into the above-mentioned underlying image features to obtain the above-mentioned spatial image features.

[0010] According to an embodiment of the present invention, the above-mentioned channel image features and the above-mentioned spatial image features are feature-fused and the above-mentioned underlying image features are introduced to restore the high-frequency details of the above-mentioned degraded image, including: element-wise addition of the above-mentioned channel image features and the above-mentioned spatial image features; inputting the added features into a convolution block for feature fusion; and element-wise addition of the above-mentioned underlying image features and the fused features output by the above-mentioned convolution block to obtain image features after detail restoration.

[0011] Another aspect of the present invention provides a model training method, comprising: inputting a sample degraded image into a blind image super-resolution model, and outputting a predicted target image, wherein the predicted target image is reconstructed based on underlying image features and implicit degradation features of the sample degraded image; calculating the loss between the predicted target image and the sample degraded image to obtain a loss value; based on the loss value, updating the model parameters of the blind image super-resolution model using a stochastic gradient descent algorithm; and when the number of updates to the model parameters reaches a preset number, outputting a blind image super-resolution model guided by implicit degradation features.

[0012] Another aspect of the present invention provides an image reconstruction device, including: a feature extraction module, which is used to extract features from a degraded image with quality loss, and obtain underlying image features that characterize underlying details of the image and implicit degradation features that characterize image degradation information; a feature expansion module, which is used to use an implicit degradation feature regulator to expand the above implicit degradation features at the channel level and the spatial level respectively; a feature fusion module, which is used to integrate the expanded implicit degradation features into the above underlying image features, and obtain channel image features and spatial image features that integrate the above image degradation information respectively; a feature recovery module, which is used to fuse the above channel image features and the above spatial image features and introduce the above underlying image features to restore high-frequency details of the above degraded image; an image reconstruction module, which is used to input the image features obtained after detail recovery into an image super-resolution module, and output a reconstructed target image, wherein the resolution of the above target image is higher than that of the above degraded image.

[0013] Another aspect of the present invention provides a model training device, including: an image prediction module, used to input a sample degraded image into a blind image super-resolution model, and output a predicted target image, wherein the predicted target image is reconstructed based on the underlying image features and implicit degradation features of the sample degraded image; a loss calculation module, used to calculate the loss between the predicted target image and the sample degraded image to obtain a loss value; a parameter updating module, used to update the model parameters of the blind image super-resolution model using a stochastic gradient descent algorithm based on the loss value; and a model output module, used to output a blind image super-resolution model guided by implicit degradation features when the number of updates to the model parameters reaches a preset number.

[0014] Another aspect of the present invention provides an electronic device, comprising: one or more processors; and a memory for storing one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors implement the method described above.

[0015] Another aspect of the present invention provides a computer-readable storage medium storing computer-executable instructions, which are used to implement the above method when executed.

[0016] Another aspect of the present invention provides a computer program product, the computer program product comprising computer executable instructions, and the instructions are used to implement the above method when executed.

[0017] According to an embodiment of the present invention, by expanding the implicit degradation features at the channel level and the spatial level, the image degradation information is fully mined and extracted, which facilitates the reconstruction of the degraded image. The expanded implicit degradation features are integrated into the underlying image features, and the fused features are re-introduced into the underlying image features, so that the image degradation information is gradually adapted to the degraded image from the channel domain and the spatial domain. By gradually fusing the implicit degradation features with the underlying image features, the two are more fully integrated, which effectively improves the adaptation efficiency and effect of the image degradation information and the degraded image, thereby improving the reconstruction quality and reconstruction speed of the degraded image to the target image. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] The above and other objects, features and advantages of the present invention will become more apparent through the following description of the embodiments of the present invention with reference to the accompanying drawings, in which:

[0019] Figure 1 An exemplary system architecture to which the image reconstruction method, model training method, and apparatus of the present invention can be applied is shown.

[0020] Figure 2 A flow chart of an image reconstruction method according to an embodiment of the present invention is shown.

[0021] Figure 3 A schematic diagram of an implicit degradation feature adjuster in an image reconstruction method according to an embodiment of the present invention is shown.

[0022] Figure 4 A schematic diagram of an implicit degradation feature adaptation unit in an image reconstruction method according to an embodiment of the present invention is shown.

[0023] Figure 5 A schematic diagram of an implicit degradation feature adaptation module in an image reconstruction method according to an embodiment of the present invention is shown.

[0024] Fig. 6A A flow chart of a model training method according to an embodiment of the present invention is shown.

[0025] Figure 6B A flowchart of a model training method according to another embodiment of the present invention is shown.

[0026] Figure 7 A block diagram of an image reconstruction device according to an embodiment of the present invention is shown.

[0027] Figure 8 A block diagram of a model training device according to an embodiment of the present invention is shown.

[0028] Fig. 9 A block diagram of an electronic device suitable for implementing an image reconstruction method and a model training method according to an embodiment of the present invention is shown. DETAILED DESCRIPTION

[0029] Below, embodiments of the present invention will be described with reference to the accompanying drawings. However, it should be understood that these descriptions are exemplary only and are not intended to limit the scope of the present invention. In the following detailed description, for ease of explanation, many specific details are set forth to provide a comprehensive understanding of embodiments of the present invention. However, it is apparent that one or more embodiments may also be implemented without these specific details. In addition, in the following description, descriptions of known structures and technologies are omitted to avoid unnecessary confusion of concepts of the present invention.

[0030] The terms used herein are only for describing specific embodiments and are not intended to limit the present invention. The terms "comprise", "include", etc. used herein indicate the existence of the features, steps, operations and / or components, but do not exclude the existence or addition of one or more other features, steps, operations or components.

[0031] All terms (including technical and scientific terms) used herein have the meanings commonly understood by those skilled in the art unless otherwise defined. It should be noted that the terms used herein should be interpreted as having a meaning consistent with the context of this specification and should not be interpreted in an idealized or overly rigid manner.

[0032] When using expressions such as "at least one of A, B, and C, etc.", they should generally be interpreted according to the meaning of the expression commonly understood by those skilled in the art (for example, "a system having at least one of A, B, and C" should include but is not limited to a system having A alone, B alone, C alone, A and B, A and C, B and C, and / or A, B, C, etc.).

[0033] In the embodiments of the present invention, the collection, updating, analysis, processing, use, transmission, provision, disclosure, storage and other aspects of the data involved (for example, including but not limited to user personal information) are in compliance with the provisions of relevant laws and regulations, are used for legitimate purposes, and do not violate public order and good morals. In particular, necessary measures are taken for user personal information to prevent illegal access to user personal information data and maintain the security of user personal information and network security.

[0034] In the embodiment of the present invention, before obtaining or collecting the user's personal information, the user's authorization or consent is obtained.

[0035] Blind Super-Resolution (BSR) is an important image processing task. It mainly refers to the process of restoring a low-resolution image (LR) to a high-resolution image (HR) without knowing or determining the specific process of image degradation (such as blur kernel, noise level, downsampling operation, etc.). It has been widely used in remote sensing, video surveillance, and medical imaging. At the same time, blind image super-resolution can also be used as a pre-operation for many advanced visual tasks, such as image retrieval, object detection, scene analysis, etc. The challenge faced by the BSR task is the unknown nature of the degradation process, because the image degradation in the real world is often complex and diverse. Therefore, how to accurately estimate the degradation information and how to use the degradation information to guide the reconstruction process are the key issues in the BSR field. Early BSR methods adopted an explicit degradation parameter estimation strategy, using a complex explicit degradation estimator to accurately estimate the degradation information of the LR image and use it as the guiding information of the BSR network. However, this explicit degradation parameter estimation strategy still cannot handle the complex degradation in real scenes well, including noise interference and multiple degradation interactions, etc. Therefore, the BSR field has gradually begun to study implicit degradation parameter estimation strategies and how to use implicit degradation features to guide the reconstruction of LR images to HR images.

[0036] As an emerging research field, the BSR task guided by implicit degradation features is currently composed of four parts: image feature extractor, implicit degradation feature extractor, implicit degradation adaptation module and reconstruction module. Among them, the image feature extractor is used to extract the feature representation of the underlying details in the LR image, the implicit degradation feature extractor is used to extract the implicit feature representation of the degradation information in the LR image, the implicit degradation adaptation module aims to integrate the implicit degradation features into the LR image features, and the reconstruction module aims to generate the corresponding HR image based on the LR image features that integrate the degradation information.

[0037] There are several main problems with related technologies. First, the model is complex and uses a large number of fully connected layers to modulate the degradation features, resulting in a large number of parameters and low computational efficiency. Second, the degradation modulation adaptation is only performed from the channel domain or the spatial domain. This single-domain adaptation cannot fully integrate the implicit degradation features into the LR image features, which will ultimately lead to poor super-resolution effects of the LR images.

[0038] The embodiments of the present invention provide an image reconstruction method, a model training method, an apparatus, a device, a medium and a product. The image reconstruction method includes: extracting features from a degraded image with quality loss to obtain underlying image features representing underlying details of the image and implicit degradation features representing image degradation information; using an implicit degradation feature regulator to expand the implicit degradation features at the channel level and the spatial level respectively; integrating the expanded implicit degradation features into the underlying image features to obtain channel image features and spatial image features that incorporate image degradation information respectively; integrating the channel image features and the spatial image features and introducing the underlying image features to restore the high-frequency details of the degraded image; inputting the image features obtained after detail restoration into an image super-resolution module, and outputting a reconstructed target image, wherein the resolution of the target image is higher than that of the degraded image.

[0039] Figure 1 An exemplary system architecture to which the image reconstruction method, model training method and device of the present invention can be applied is shown. It should be noted that: Figure 1 The examples shown are merely examples of system architectures to which the embodiments of the present invention may be applied, to help those skilled in the art understand the technical content of the present invention, but do not mean that the embodiments of the present invention may not be used in other devices, systems, environments or scenarios.

[0040] like Figure 1 As shown, the system architecture 100 according to this embodiment may include a first terminal device 101, a second terminal device 102, a third terminal device 103, a network 104, and a server 105. The network 104 is used to provide a medium for a communication link between the first terminal device 101, the second terminal device 102, the third terminal device 103, and the server 105. The network 104 may include various connection types, such as wired and / or wireless communication links, etc.

[0041] The user may use the first terminal device 101, the second terminal device 102, and the third terminal device 103 to interact with the server 105 through the network 104 to receive or send messages, etc. Various communication client applications may be installed on the first terminal device 101, the second terminal device 102, and the third terminal device 103, such as shopping applications, web browser applications, search applications, instant messaging tools, email clients, and / or social platform software, etc. (only for example).

[0042] The first terminal device 101, the second terminal device 102, and the third terminal device 103 may be various electronic devices having display screens and supporting web browsing, including but not limited to smart phones, tablet computers, laptop computers, desktop computers, and the like.

[0043] The server 105 may be a server that provides various services, such as a background management server (only as an example) that provides support for websites browsed by users using the first terminal device 101, the second terminal device 102, and the third terminal device 103. The background management server may analyze and process the received data such as user requests, and feed back the processing results (such as web pages, information, or data obtained or generated according to user requests) to the terminal device.

[0044] It should be noted that the image reconstruction method and model training method provided in the embodiment of the present invention can generally be executed by the server 105. Accordingly, the image reconstruction device and model training device provided in the embodiment of the present invention can generally be set in the server 105. The image reconstruction method and model training method provided in the embodiment of the present invention can also be executed by a server or server cluster that is different from the server 105 and can communicate with the first terminal device 101, the second terminal device 102, the third terminal device 103 and / or the server 105. Correspondingly, the image reconstruction device and model training device provided in the embodiment of the present invention can also be set in a server or server cluster that is different from the server 105 and can communicate with the first terminal device 101, the second terminal device 102, the third terminal device 103 and / or the server 105. Alternatively, the image reconstruction method and model training method provided in the embodiment of the present invention can also be performed by the first terminal device 101, the second terminal device 102 or the third terminal device 103, or can also be performed by other terminal devices different from the first terminal device 101, the second terminal device 102 or the third terminal device 103. Correspondingly, the image reconstruction device and model training device provided in the embodiments of the present invention may also be arranged in the first terminal device 101, the second terminal device 102 or the third terminal device 103, or in other terminal devices different from the first terminal device 101, the second terminal device 102 or the third terminal device 103.

[0045] It should be understood that Figure 1 The number of terminal devices, networks and servers in the embodiment is only for illustration. Any number of terminal devices, networks and servers may be provided according to implementation requirements.

[0046] Figure 2 A flow chart of an image reconstruction method according to an embodiment of the present invention is shown.

[0047] like Figure 2 As shown, the method includes operations S210 to S250.

[0048] In operation S210, feature extraction is performed on a degraded image with quality loss to obtain underlying image features representing underlying details of the image and implicit degradation features representing image degradation information.

[0049] In operation S220, the implicit degradation feature adjuster is used to expand the implicit degradation feature at the channel level and the space level respectively.

[0050] In operation S230, the expanded implicit degradation features are integrated into the underlying image features to obtain channel image features and spatial image features that incorporate image degradation information.

[0051] In operation S240, the channel image features and the spatial image features are fused and introduced into the underlying image features to restore high-frequency details of the degraded image.

[0052] In operation S250, the image features obtained after detail restoration are input into an image super-resolution module, and a reconstructed target image is output, wherein the resolution of the target image is higher than that of the degraded image.

[0053] According to an embodiment of the present invention, a blind image super-resolution network guided by implicit degradation features is constructed, including an image feature extractor, an implicit degradation feature extractor, an implicit degradation feature regulator, an implicit degradation feature adaptation module and an image super-resolution module. After obtaining a degraded image with quality loss and low resolution, the image feature extractor and the implicit degradation feature extractor are used to extract the underlying detail features and implicit degradation features in the degraded image.

[0054] According to an embodiment of the present invention, the image feature extractor uses a 3x3 convolutional layer, and the implicit degradation estimator can select a deep convolutional neural network and use the implicit degradation learning task for pre-training. The underlying detail features of the degraded image extracted by the image feature extractor can be expressed as The implicit degradation features of the degraded image extracted by the implicit degradation feature extractor can be expressed as , where H stands for height and W stands for width.

[0055] According to an embodiment of the present invention, an implicit degradation feature regulator is used to further generate a channel-level degradation feature vector and a spatial-level degradation feature map based on the extracted implicit degradation features, that is, a channel degradation feature and a spatial degradation feature. An implicit degradation feature adaptation module is used to adapt the image degradation information to specific underlying image features. Specifically, for the implicit degradation feature adaptation unit in the implicit degradation feature adaptation module, in the spatial adaptation branch, the modulated spatial degradation feature is integrated into the underlying detail feature to obtain the spatial image feature. In the channel adaptation branch, the modulated channel degradation feature is integrated into the underlying detail feature to obtain the channel image feature.

[0056] According to an embodiment of the present invention, the channel image features and the spatial image features are fused, and after fusion, the underlying image features are introduced to restore high-frequency details. The image super-resolution module is used to restore the target image corresponding to the degraded image using the image features obtained after detail restoration as input, and the resolution of the target image is higher than that of the degraded image.

[0057] According to an embodiment of the present invention, by expanding the implicit degradation features at the channel level and the spatial level, the image degradation information is fully mined and extracted, which facilitates the reconstruction of the degraded image. The expanded implicit degradation features are integrated into the underlying image features, and the fused features are re-introduced into the underlying image features, so that the image degradation information is gradually adapted to the degraded image from the channel domain and the spatial domain. By gradually fusing the implicit degradation features with the underlying image features, the two are more fully integrated, which effectively improves the adaptation efficiency and effect of the image degradation information and the degraded image, thereby improving the reconstruction quality and reconstruction speed of the degraded image to the target image.

[0058] Figure 3 A schematic diagram of an implicit degradation feature adjuster in an image reconstruction method according to an embodiment of the present invention is shown.

[0059] According to an embodiment of the present invention, an implicit degradation feature regulator is used to expand the implicit degradation feature at the channel level and the spatial level respectively, including: performing a global average pooling operation on the implicit degradation feature in the channel conversion branch of the implicit degradation feature regulator; sending the pooling result to the fully connected layer of the channel conversion branch for nonlinear transformation to obtain the channel degradation feature after the implicit degradation feature is expanded at the channel level; using a sub-pixel convolution layer in the spatial conversion branch of the implicit degradation feature regulator to upsample the implicit degradation feature to the same size as the degraded image; and inputting the upsampled feature map into the convolution layer of the spatial conversion branch to obtain the spatial degradation feature after the implicit degradation feature is expanded at the spatial level.

[0060] like Figure 3 As shown, the implicit degradation features are respectively input into the channel conversion branch 310 and the spatial conversion branch 320 of the implicit degradation feature regulator. Specifically, the channel conversion branch performs a global average pooling operation on the implicit degradation features, and then sends the pooling result to the fully connected layer for nonlinear transformation to obtain the channel degradation features. The spatial transformation branch uses a sub-pixel convolution layer (PixelShuffle) to upsample the implicit degradation features to the same size as the degraded image, and then feeds the upsampled feature map into a 3×3 convolution layer to obtain the spatial degradation features. By expanding the implicit degradation features at the channel level and the spatial level, the image degradation information can be fully mined and extracted, which facilitates the reconstruction of the degraded image.

[0061] According to an embodiment of the present invention, the extended implicit degradation feature is integrated into the underlying image feature to obtain a channel image feature and a spatial image feature respectively incorporating image degradation information, including: inputting the channel degradation feature and the underlying image feature into a channel adaptation branch of an implicit degradation feature adaptation unit to integrate the image degradation information into the underlying image feature to obtain the channel image feature; inputting the spatial degradation feature and the underlying image feature into a spatial adaptation branch of the implicit degradation feature adaptation unit to integrate the image degradation information into the underlying image feature to obtain the spatial image feature.

[0062] According to an embodiment of the present invention, the implicit degradation feature adaptation unit includes a spatial adaptation branch and a channel adaptation branch. In the spatial adaptation branch, the spatial degradation feature is modulated, and specifically two simple convolutional layers can be used for modulation to reduce the number of parameters and the amount of calculation. The modulated spatial degradation feature is integrated into the underlying detail feature to obtain the spatial image feature.

[0063] According to an embodiment of the present invention, in the channel adaptation branch, the underlying detail features are used to adjust the strength of the degradation features of each channel. Specifically, the channel degradation features are modulated, the modulated channel degradation features are integrated into the underlying detail features, and then the depth-separable convolution layer is used for nonlinear transformation to obtain the channel image features. The joint learning of the underlying detail features and the channel image features helps to improve the expression ability of the degradation details in the channel dimension and improve the adaptation effect between the two.

[0064] Figure 4 A schematic diagram of an implicit degradation feature adaptation unit in an image reconstruction method according to an embodiment of the present invention is shown.

[0065] According to an embodiment of the present invention, channel degradation features and underlying image features are input into a channel adaptation branch of an implicit degradation feature adaptation unit to integrate image degradation information into underlying image features to obtain channel image features, including: using a global pooling layer to adjust the shape of the underlying image features; inputting the channel degradation features and the pooled underlying image features into two fully connected layers; adding the outputs of the two fully connected layers element-wise; normalizing the added features using a normalization function; element-wise multiplying the normalized features with the underlying image features to integrate the normalized features into the underlying image features; and using a depthwise separable convolutional layer to perform a nonlinear transformation on the fused features to obtain channel image features.

[0066] like Figure 4As shown, the channel adaptation branch 410 includes a global pooling layer 411, a fully connected layer 412 and a depth-separable convolution layer 413. In the channel adaptation branch 410, in order to introduce the underlying image features in the channel degradation feature modulation process, the global pooling layer 411 is first used to adjust the shape of the underlying image features. Then, the channel degradation features and the pooled underlying image features are simultaneously sent to two fully connected layers 412, and their transformation results are element-wise added. Then, the modulated channel degradation features are normalized using a normalization function (Sigmoid function). Finally, the modulated channel degradation features are integrated into the underlying image features, and then a 3x3 depth-separable convolution layer is used to perform a nonlinear transformation to finally obtain the channel image features. This process can be expressed as:

[0067]

[0068] in, represents a fully connected layer, represents a 3x3 depth-separable convolutional layer, represents the global average pooling operation, represents the channel image features, represents an element-wise multiplication operation, represents the channel degradation characteristics, represents the GELU (Gaussian Error Linear Unit) activation function, represents the normalization function, Represents the underlying image features.

[0069] According to an embodiment of the present invention, the spatial degradation features and the underlying image features are input into the spatial adaptation branch of the implicit degradation feature adaptation unit to integrate the image degradation information into the underlying image features to obtain the spatial image features, and the spatial image features are obtained, including: modulating the spatial degradation features using two convolutional layers in the spatial adaptation branch; normalizing the modulated features using a normalization function; and element-wise multiplying the normalized features with the underlying image features to integrate the normalized features into the underlying image features to obtain the spatial image features.

[0070] like Figure 4 As shown, the spatial adaptation branch 420 includes two convolution layers 421. In the spatial adaptation branch 420, the spatial degradation features are first modulated using two 3x3 convolution layers 421, and then normalized using a normalization function (Sigmoid function). Finally, the modulated spatial degradation features are integrated into the underlying image features to obtain spatial image features. This process can be expressed as:

[0071]

[0072] in, represents a 3x3 convolutional layer, Represents the spatial image features, Represents spatial degradation characteristics.

[0073] According to an embodiment of the present invention, image degradation information is gradually adapted to a specific degraded image from the channel domain and the spatial domain. At the spatial level, two simple convolutional layers are used to modulate the spatial domain degradation features, which significantly reduces the number of parameters and the amount of calculation. At the channel level, the current underlying image features are used to adjust the strength of the degradation features of each channel. This joint learning of the underlying image features and the channel degradation features helps to improve the expression ability of the degradation details in the channel dimension and improve the adaptation effect between the two.

[0074] According to an embodiment of the present invention, channel image features and spatial image features are fused and underlying image features are introduced to restore high-frequency details of the degraded image, including: element-wise addition of channel image features and spatial image features; inputting the added features into a convolution block for feature fusion; and element-wise addition of the underlying image features and the fused features output by the convolution block to obtain image features after detail restoration.

[0075] like Figure 4 As shown, the implicit degradation feature adaptation unit also includes a convolution block 430, wherein the convolution block 430 includes a 7x7 depth-separable convolution layer and two 1x1 convolution layers. The channel image features and the spatial image features are element-wise added and sent to a convolution block 430 for further fusion, and finally the underlying image features are introduced again using the skip link to restore the high-frequency details. This process can be expressed as:

[0076]

[0077] in, represents the convolutional block, Represents the image features processed by the current implicit degradation feature adaptation unit.

[0078] According to an embodiment of the present invention, an implicit degradation feature adaptation group (including a series of degradation adaptation units and convolution blocks) and an implicit degradation feature adaptation module (including a series of degradation adaptation groups and convolution blocks) are gradually constructed from the bottom to the top based on the implicit degradation feature adaptation unit to achieve continuous interaction and fusion of implicit degradation features and underlying image features.

[0079] Figure 5 A schematic diagram of an implicit degradation feature adaptation module in an image reconstruction method according to an embodiment of the present invention is shown.

[0080] like Figure 5 As shown in the figure, for the implicit degradation feature adaptation group, it consists of 6 implicit degradation feature adaptation units, 1 convolution block and 1 3x3 convolution layer. In the adaptation group, the underlying image features and implicit degradation features are sequentially sent to each component for processing, and a jump link is added between the input and output of the adaptation group.

[0081] The implicit degradation feature adaptation module is composed of 6 implicit degradation feature adaptation groups and 1 3x3 convolution layer in series. In the adaptation module, the underlying image features and implicit degradation features are sequentially sent to each component for processing, and a jump link is added between the input and output of the adaptation module to restore the detailed features of the image. After processing by the implicit degradation feature adaptation module, the image features that fully integrate the degradation information and restore the details can be obtained. .

[0082] According to an embodiment of the present invention, with the help of the proposed implicit degradation feature regulator and the implicit degradation feature adaptation module, the constructed BSR network can effectively improve the adaptation efficiency and effect of implicit degradation information and degraded images, thereby ultimately improving the reconstruction quality and reconstruction speed of degraded images to target images.

[0083] Fig. 6A A flow chart of a model training method according to an embodiment of the present invention is shown.

[0084] like Fig. 6A As shown, the method includes operations S610 to S640.

[0085] In operation S610, the sample degraded image is input into a blind image super-resolution model, and a predicted target image is output, wherein the predicted target image is reconstructed according to underlying image features and implicit degradation features of the sample degraded image.

[0086] In operation S620, a loss between the predicted target image and the sample degraded image is calculated to obtain a loss value.

[0087] In operation S630, model parameters of the blind image super-resolution model are updated using a stochastic gradient descent algorithm according to the loss value.

[0088] In operation S640, when the number of updates of the model parameters reaches a preset number, a blind image super-resolution model guided by implicit degradation features is output.

[0089] According to an embodiment of the present invention, the sample degraded image is input into the blind image super-resolution model, and the image is reconstructed to obtain a predicted target image. The reconstruction method is the same as the above method and is not described here. The loss between the predicted target image and the sample degraded image is calculated to obtain a loss value, and the calculation method is as follows:

[0090]

[0091] in, represents the predicted target image, represents the sample degraded image, Represents the loss value.

[0092] According to an embodiment of the present invention, a stochastic gradient descent algorithm is used as an optimizer to optimize all model parameters, wherein the learning rate is gradually adjusted from 5e-4 to 5e-6 using a cosine annealing adjustment strategy, and the preset number of iterations is set to 800. When the number of updates to the model parameters reaches the preset number, the model parameters at the end of the training are saved, and the expected blind image super-resolution model guided by implicit degradation features can be obtained, which can be directly used for image super-resolution tasks in complex degradation and unknown scenes.

[0093] Figure 6B A flowchart of a model training method according to another embodiment of the present invention is shown.

[0094] like Figure 6B As shown in the figure, during the model training process, low-definition image samples are input into the model to extract the underlying detail features of the low-definition image and the implicit degradation features of the low-definition image. After extraction, the implicit degradation features are expanded into channel implicit domain degradation features and spatial domain implicit degradation features, and the implicit degradation features and the underlying detail features of the image are fused to reconstruct the high-definition version of the low-definition image. The L1 loss between the real high-definition image and the reconstructed high-definition image is calculated, and the model parameters are updated using the gradient descent algorithm. The termination condition is judged, specifically, the number of iterations or the loss signal reaches a preset value. After reaching the termination condition, the blind image super-resolution model guided by the implicit degradation features is output to complete the model training.

[0095] Figure 7 A block diagram of an image reconstruction device according to an embodiment of the present invention is shown.

[0096] like Figure 7 As shown, the image reconstruction device 700 includes a feature extraction module 710 , a feature expansion module 720 , a feature fusion module 730 , a feature recovery module 740 and an image reconstruction module 750 .

[0097] The feature extraction module 710 is used to extract features from the degraded image with quality loss, and obtain underlying image features representing underlying details of the image and implicit degradation features representing image degradation information.

[0098] The feature expansion module 720 is used to expand the implicit degradation feature at the channel level and the space level respectively using the implicit degradation feature regulator.

[0099] The feature fusion module 730 is used to integrate the expanded implicit degradation features into the underlying image features to obtain channel image features and spatial image features that are integrated with image degradation information.

[0100] The feature recovery module 740 is used to perform feature fusion on the channel image features and the spatial image features and introduce the underlying image features to recover the high-frequency details of the degraded image.

[0101] The image reconstruction module 750 is used to input the image features obtained after detail restoration into the image super-resolution module, and output a reconstructed target image, wherein the resolution of the target image is higher than that of the degraded image.

[0102] According to an embodiment of the present invention, the feature expansion module 720 includes an average pooling submodule, a channel expansion submodule, a feature sampling submodule and a space expansion submodule.

[0103] The average pooling submodule is used to perform global average pooling operations on implicitly degraded features in the channel conversion branch of the implicitly degraded feature regulator.

[0104] The channel extension submodule is used to send the pooling result to the fully connected layer of the channel conversion branch for nonlinear transformation, so as to obtain the channel degradation feature after the implicit degradation feature is expanded at the channel level.

[0105] A feature sampling submodule is used to upsample the implicit degradation features to the same size as the degraded image using a sub-pixel convolutional layer in the spatial transformation branch of the implicit degradation feature conditioner.

[0106] The spatial expansion submodule is used to input the upsampled feature map into the convolution layer of the spatial transformation branch to obtain the spatially degraded features after spatial-level expansion of the implicitly degraded features.

[0107] According to an embodiment of the present invention, the feature fusion module 730 includes a channel fusion submodule and a space fusion submodule.

[0108] The channel fusion submodule is used to input the channel degradation features and the underlying image features into the channel adaptation branch of the implicit degradation feature adaptation unit, so as to integrate the image degradation information into the underlying image features to obtain the channel image features.

[0109] The spatial fusion submodule is used to input the spatial degradation features and the underlying image features into the spatial adaptation branch of the implicit degradation feature adaptation unit to integrate the image degradation information into the underlying image features to obtain the spatial image features.

[0110] According to an embodiment of the present invention, the channel fusion submodule includes a shape adjustment unit, a fully connected processing unit, a result addition unit, a normalization processing unit, a feature multiplication unit and a feature transformation unit.

[0111] The shape adjustment unit is used to adjust the shape of the underlying image features using the global pooling layer.

[0112] The fully connected processing unit is used to input the channel degradation features and the pooled underlying image features into two fully connected layers.

[0113] The result addition unit is used to perform element-wise addition of the output results of the two fully connected layers.

[0114] The normalization processing unit is used to perform normalization processing on the added features using a normalization function.

[0115] The feature multiplication unit is used to perform element-wise multiplication of the normalized features with the underlying image features to integrate the normalized features into the underlying image features.

[0116] The feature transformation unit uses a depth-wise separable convolutional layer to perform nonlinear transformation on the fused features to obtain channel image features.

[0117] According to an embodiment of the present invention, the spatial fusion submodule includes a feature modulation unit, a feature normalization unit and a feature integration unit.

[0118] The feature modulation unit is used to modulate the spatially degraded features using two convolutional layers in the spatial adaptation branch.

[0119] The feature normalization unit is used to normalize the modulated features using a normalization function.

[0120] The feature integration unit is used to perform element-wise multiplication of the normalized features with the underlying image features to integrate the normalized features into the underlying image features to obtain spatial image features.

[0121] According to an embodiment of the present invention, the feature recovery module 740 includes a feature addition submodule, a convolution fusion submodule and a detail recovery submodule.

[0122] The feature addition submodule is used to perform element-wise addition of channel image features and spatial image features.

[0123] The convolution fusion submodule is used to input the added features into the convolution block for feature fusion.

[0124] The detail recovery submodule is used to perform element-wise addition of the underlying image features and the fused features output by the convolution block to obtain the image features after detail recovery.

[0125] Figure 8 A block diagram of a model training device according to an embodiment of the present invention is shown.

[0126] like Figure 8 As shown, the model training device 800 includes an image prediction module 810, a loss calculation module 820, a parameter updating module 830 and a model output module 840.

[0127] The image prediction module 810 is used to input the sample degraded image into the blind image super-resolution model and output a predicted target image, wherein the predicted target image is reconstructed based on the underlying image features and implicit degradation features of the sample degraded image.

[0128] The loss calculation module 820 is used to calculate the loss between the predicted target image and the sample degraded image to obtain a loss value.

[0129] The parameter updating module 830 is used to update the model parameters of the blind image super-resolution model using a stochastic gradient descent algorithm according to the loss value.

[0130] The model output module 840 is used to output a blind image super-resolution model guided by implicit degradation features when the number of updates to the model parameters reaches a preset number.

[0131] According to the embodiments of the present invention, any one or more of the modules, submodules, units, and subunits, or at least part of the functions of any one of them can be implemented in one module. According to the embodiments of the present invention, any one or more of the modules, submodules, units, and subunits can be split into multiple modules for implementation. According to the embodiments of the present invention, any one or more of the modules, submodules, units, and subunits can be at least partially implemented as hardware circuits, such as field programmable gate arrays (FPGAs), programmable logic arrays (PLAs), systems on chips, systems on substrates, systems on packages, application specific integrated circuits (ASICs), or can be implemented by hardware or firmware in any other reasonable way of integrating or packaging the circuit, or by any one of the three implementation methods of software, hardware, and firmware, or by a proper combination of any of them. Alternatively, according to the embodiments of the present invention, one or more of the modules, submodules, units, and subunits can be at least partially implemented as computer program modules, and when the computer program modules are run, the corresponding functions can be executed.

[0132] For example, any of the feature extraction module 710, feature expansion module 720, feature fusion module 730, feature recovery module 740 and image reconstruction module 750, image prediction module 810, loss calculation module 820, parameter update module 830 and model output module 840 can be combined in one module / unit / sub-unit for implementation, or any one of the modules / units / sub-units can be split into multiple modules / units / sub-units. Alternatively, at least part of the functions of one or more of these modules / units / sub-units can be combined with at least part of the functions of other modules / units / sub-units and implemented in one module / unit / sub-unit. According to an embodiment of the present invention, at least one of the feature extraction module 710, the feature expansion module 720, the feature fusion module 730, the feature recovery module 740 and the image reconstruction module 750, the image prediction module 810, the loss calculation module 820, the parameter update module 830 and the model output module 840 can be at least partially implemented as a hardware circuit, such as a field programmable gate array (FPGA), a programmable logic array (PLA), a system on a chip, a system on a substrate, a system on a package, an application-specific integrated circuit (ASIC), or can be implemented by hardware or firmware such as any other reasonable way of integrating or packaging the circuit, or by any one of the three implementation methods of software, hardware and firmware or by an appropriate combination of any of them. Alternatively, at least one of the feature extraction module 710, the feature expansion module 720, the feature fusion module 730, the feature recovery module 740 and the image reconstruction module 750, the image prediction module 810, the loss calculation module 820, the parameter update module 830 and the model output module 840 can be at least partially implemented as a computer program module, and when the computer program module is run, the corresponding function can be performed.

[0133] It should be noted that the image reconstruction device and model training device parts in the embodiments of the present invention correspond to the image reconstruction method and model training method parts in the embodiments of the present invention. The description of the image reconstruction device and model training device parts specifically refers to the image reconstruction method and model training method parts, which will not be repeated here.

[0134] Fig. 9 A block diagram of an electronic device suitable for implementing an image reconstruction method and a model training method according to an embodiment of the present invention is shown. Fig. 9 The electronic device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present invention.

[0135] like Fig. 9As shown, the electronic device 900 according to an embodiment of the present invention includes a processor 901, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 902 or a program loaded from a storage part 908 into a random access memory (RAM) 903. The processor 901 may include, for example, a general-purpose microprocessor (e.g., a CPU), an instruction set processor and / or a related chipset and / or a special-purpose microprocessor (e.g., an application-specific integrated circuit (ASIC)), etc. The processor 901 may also include an onboard memory for caching purposes. The processor 901 may include a single processing unit or multiple processing units for performing different actions of the method flow according to an embodiment of the present invention.

[0136] In RAM 903, various programs and data required for the operation of electronic device 900 are stored. Processor 901, ROM 902 and RAM 903 are connected to each other via bus 904. Processor 901 performs various operations of the method flow according to the embodiment of the present invention by executing the program in ROM 902 and / or RAM 903. It should be noted that the program can also be stored in one or more memories other than ROM 902 and RAM 903. Processor 901 can also perform various operations of the method flow according to the embodiment of the present invention by executing the program stored in one or more memories.

[0137] According to an embodiment of the present invention, the electronic device 900 may further include an input / output (I / O) interface 905, which is also connected to the bus 904. The electronic device 900 may further include one or more of the following components connected to the input / output (I / O) interface 905: an input portion 906 including a keyboard, a mouse, etc.; an output portion 907 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage portion 908 including a hard disk, etc.; and a communication portion 909 including a network interface card such as a LAN card, a modem, etc. The communication portion 909 performs communication processing via a network such as the Internet. A drive 910 is also connected to the input / output (I / O) interface 905 as needed. A removable medium 911, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 910 as needed, so that a computer program read therefrom is installed into the storage portion 908 as needed.

[0138] According to an embodiment of the present invention, the method flow according to an embodiment of the present invention can be implemented as a computer software program. For example, an embodiment of the present invention includes a computer program product, which includes a computer program carried on a computer-readable storage medium, and the computer program includes a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from the network through the communication part 909, and / or installed from the removable medium 911. When the computer program is executed by the processor 901, the above-mentioned functions defined in the system of the embodiment of the present invention are executed. According to an embodiment of the present invention, the system, equipment, device, module, unit, etc. described above can be implemented by a computer program module.

[0139] The present invention also provides a computer-readable storage medium, which may be included in the device / apparatus / system described in the above embodiment; or may exist independently without being assembled into the device / apparatus / system. The above computer-readable storage medium carries one or more programs, and when the above one or more programs are executed, the method according to the embodiment of the present invention is implemented.

[0140] According to an embodiment of the present invention, the computer-readable storage medium may be a non-volatile computer-readable storage medium. For example, it may include but is not limited to: a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In the present invention, the computer-readable storage medium may be any tangible medium containing or storing a program, which may be used by or in combination with an instruction execution system, an apparatus or a device.

[0141] For example, according to an embodiment of the present invention, the computer-readable storage medium may include the ROM 902 and / or the RAM 903 described above and / or one or more memories other than the ROM 902 and the RAM 903 .

[0142] An embodiment of the present invention also includes a computer program product, which includes a computer program, which contains program code for executing the method provided by the embodiment of the present invention. When the computer program product runs on an electronic device, the program code is used to enable the electronic device to implement the image reconstruction method and model training method provided by the embodiment of the present invention.

[0143] When the computer program is executed by the processor 901, the above functions defined in the system / device of the embodiment of the present invention are executed. According to the embodiment of the present invention, the system, device, module, unit, etc. described above can be implemented by a computer program module.

[0144] In one embodiment, the computer program may be based on a tangible storage medium such as an optical storage device, a magnetic storage device, etc. In another embodiment, the computer program may also be transmitted and distributed in the form of a signal on a network medium, and downloaded and installed through the communication part 909, and / or installed from a removable medium 911. The program code contained in the computer program may be transmitted using any appropriate network medium, including but not limited to: wireless, wired, etc., or any suitable combination of the above.

[0145] According to an embodiment of the present invention, the program code for executing the computer program provided by the embodiment of the present invention can be written in any combination of one or more programming languages. Specifically, these computing programs can be implemented using high-level process and / or object-oriented programming languages, and / or assembly / machine languages. Programming languages ​​include, but are not limited to, Java, C++, python, "C" language or similar programming languages. The program code can be executed entirely on the user computing device, partially on the user device, partially on the remote computing device, or entirely on the remote computing device or server. In the case of a remote computing device, the remote computing device can be connected to the user computing device through any type of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computing device (e.g., using an Internet service provider to connect through the Internet).

[0146] The flowcharts and block diagrams in the accompanying drawings illustrate the possible architecture, functions and operations of the systems, methods and computer program products according to various embodiments of the present invention. In this regard, each box in the flowchart or block diagram may represent a module, a program segment, or a part of a code, and the above-mentioned module, program segment, or a part of the code contains one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box may also occur in an order different from the order marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flowchart, and the combination of boxes in the block diagram or flowchart, can be implemented with a dedicated hardware-based system that performs the specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions. It can be understood by those skilled in the art that the features recorded in the various embodiments of the present invention can be combined and / or combined in various ways, even if such a combination or combination is not explicitly recorded in the present invention. In particular, without departing from the spirit and teachings of the present invention, the features recorded in the various embodiments of the present invention can be combined and / or combined in various ways. All such combinations and / or combinations fall within the scope of the present invention.

[0147] The embodiments of the present invention are described above. However, these embodiments are only for the purpose of illustration, and are not intended to limit the scope of the present invention. Although each embodiment is described above, it does not mean that the measures in each embodiment cannot be used in combination advantageously. Without departing from the scope of the present invention, those skilled in the art may make various substitutions and modifications, which should all fall within the scope of the present invention.

Claims

1. An image reconstruction method, characterized in that: include: Extract features from degraded images with quality loss to obtain underlying image features that represent underlying details of the image and implicit degradation features that represent image degradation information; Using an implicit degradation feature adjuster to expand the implicit degradation feature at the channel level and the space level respectively; Integrating the expanded implicit degradation features into the underlying image features to obtain channel image features and spatial image features that incorporate the image degradation information; Performing feature fusion on the channel image features and the spatial image features and introducing the underlying image features to restore high-frequency details of the degraded image; The image features obtained after detail restoration are input into an image super-resolution module, and a reconstructed target image is output, wherein the resolution of the target image is higher than that of the degraded image.

2. The method according to claim 1, characterized in that The step of using an implicit degradation feature adjuster to expand the implicit degradation feature at the channel level and the space level respectively includes: Performing a global average pooling operation on the implicit degradation feature in a channel conversion branch of the implicit degradation feature regulator; The pooling result is sent to the fully connected layer of the channel conversion branch for nonlinear transformation to obtain the channel degradation feature after the implicit degradation feature is expanded at the channel level; Upsampling the implicit degradation feature to the same size as the degraded image using a sub-pixel convolutional layer in a spatial transformation branch of the implicit degradation feature regulator; The upsampled feature map is input into the convolution layer of the spatial transformation branch to obtain a spatially degraded feature after spatial-level expansion of the implicitly degraded feature.

3. The method according to claim 2, characterized in that The step of integrating the extended implicit degradation features into the underlying image features to obtain channel image features and spatial image features respectively integrating the image degradation information includes: Inputting the channel degradation feature and the underlying image feature into a channel adaptation branch of an implicit degradation feature adaptation unit, so as to integrate the image degradation information into the underlying image feature to obtain the channel image feature; The spatial degradation feature and the underlying image feature are input into a spatial adaptation branch of an implicit degradation feature adaptation unit to integrate the image degradation information into the underlying image feature to obtain the spatial image feature.

4. The method according to claim 3, characterized in that The channel adaptation branch includes a global pooling layer, a fully connected layer, and a depth-separable convolutional layer. The channel degradation feature and the underlying image feature are input into the channel adaptation branch of the implicit degradation feature adaptation unit to integrate the image degradation information into the underlying image feature to obtain the channel image feature, including: Using the global pooling layer to adjust the shape of the underlying image features; Inputting the channel degradation features and the pooled bottom layer image features into the two fully connected layers; Performing element-wise addition on the outputs of the two fully connected layers; Use the normalization function to normalize the added features; Performing element-wise multiplication of the normalized features and the underlying image features to integrate the normalized features into the underlying image features; The fused features are nonlinearly transformed using the depthwise separable convolutional layer to obtain the channel image features.

5. The method according to claim 3, characterized in that: The step of inputting the spatial degradation feature and the underlying image feature into a spatial adaptation branch of an implicit degradation feature adaptation unit to integrate the image degradation information into the underlying image feature to obtain the spatial image feature comprises: Modulating the spatially degraded features using two convolutional layers in the spatial adaptation branch; The modulated features are normalized using a normalization function; The normalized features are element-wise multiplied with the underlying image features to integrate the normalized features into the underlying image features, thereby obtaining the spatial image features.

6. The method according to claim 1, characterized in that The step of fusing the channel image features and the spatial image features and introducing the underlying image features to restore high-frequency details of the degraded image includes: performing element-wise addition of the channel image features and the spatial image features; The added features are input into the convolution block for feature fusion; The underlying image features and the fused features output by the convolution block are added element-wise to obtain image features after detail restoration.

7. A model training method, characterized in that: include: The sample degraded image is input into the blind image super-resolution model, and a predicted target image is output, wherein the predicted target image is obtained according to the following method: Extracting features of the sample degraded image to obtain underlying image features representing underlying details of the image and implicit degradation features representing image degradation information; Using an implicit degradation feature adjuster to expand the implicit degradation feature at the channel level and the space level respectively; Integrating the expanded implicit degradation features into the underlying image features to obtain channel image features and spatial image features that incorporate the image degradation information; Performing feature fusion on the channel image features and the spatial image features and introducing the underlying image features to restore high-frequency details of the degraded image; Inputting the image features obtained after detail restoration into an image super-resolution module, and outputting a reconstructed predicted target image, wherein the resolution of the predicted target image is higher than that of the sample degraded image; Calculating the difference between the predicted target image and the real high-definition image corresponding to the sample degraded image to obtain a loss value; According to the loss value, the model parameters of the blind image super-resolution model are updated using a stochastic gradient descent algorithm; When the number of updates to the model parameters reaches a preset number, a blind image super-resolution model guided by implicit degradation features is output.

8. An image reconstruction device, characterized in that: include: A feature extraction module is used to extract features from degraded images with quality loss, and obtain underlying image features representing underlying details of the image and implicit degradation features representing image degradation information; A feature expansion module, used for expanding the implicit degradation feature at the channel level and the space level respectively by using an implicit degradation feature regulator; A feature fusion module, used to integrate the expanded implicit degradation features into the underlying image features, and obtain channel image features and spatial image features fused with the image degradation information; A feature recovery module, used for performing feature fusion of the channel image features and the spatial image features and introducing the underlying image features to recover the high-frequency details of the degraded image; The image reconstruction module is used to input the image features obtained after detail restoration into the image super-resolution module, and output a reconstructed target image, wherein the resolution of the target image is higher than that of the degraded image.

9. A model training device, characterized in that: include: The image prediction module is used to input the sample degraded image into the blind image super-resolution model and output a predicted target image, wherein the predicted target image is obtained according to the following method: Extracting features of the sample degraded image to obtain underlying image features representing underlying details of the image and implicit degradation features representing image degradation information; Using an implicit degradation feature adjuster to expand the implicit degradation feature at the channel level and the space level respectively; Integrating the expanded implicit degradation features into the underlying image features to obtain channel image features and spatial image features that incorporate the image degradation information; Performing feature fusion on the channel image features and the spatial image features and introducing the underlying image features to restore high-frequency details of the degraded image; Inputting the image features obtained after detail restoration into an image super-resolution module, and outputting a reconstructed predicted target image, wherein the resolution of the predicted target image is higher than that of the sample degraded image; A loss calculation module, used to calculate the difference between the predicted target image and the real high-definition image corresponding to the sample degraded image to obtain a loss value; A parameter updating module, used to update the model parameters of the blind image super-resolution model using a stochastic gradient descent algorithm according to the loss value; The model output module is used to output a blind image super-resolution model guided by implicit degradation features when the number of updates to the model parameters reaches a preset number.

10. An electronic device, characterized in that: include: one or more processors; a memory for storing one or more programs, Wherein, when the one or more programs are executed by the one or more processors, the one or more processors implement the method of any one of claims 1 to 6 or claim 7.

11. A computer-readable storage medium, characterized in that: Executable instructions are stored thereon, and when the instructions are executed by a processor, the processor implements the method of any one of claims 1 to 6 or claim 7.

12. A computer program product, characterized in that The method comprises a computer program which, when executed by a processor, implements the method according to any one of claims 1 to 6 or claim 7.

Images