Infrared image super-resolution reconstruction method and device based on feature separation

Abstract

The present invention discloses a method for super-resolution reconstruction of infrared images based on feature separation, comprising: obtaining an image to be processed; up-sampling the image to be processed to obtain a first image; inputting the first image into a pre-trained feature separation network to obtain low-resolution low-frequency features and low-resolution high-frequency features of the first image; determining a structural image of the first image based on the low-resolution low-frequency features, and obtaining texture information and a noise image of the first image based on the low-resolution high-frequency features; and super-resolution reconstruction to obtain a second image based on the structural image and texture information. The feature separation network in the present invention can achieve high- and low-frequency separation of low-resolution images and high-resolution images, thereby enabling high-frequency features to be better learned, and using different networks for processing according to the characteristics of the separated structural image, making full use of the multi-scale features of the input, further guiding the restoration of image details, and increasing the amount of information in the image.

Description

Technical Field

[0001] The invention belongs to the technical field of image processing, and in particular relates to a method and a device for super-resolution reconstruction of infrared images based on feature separation. Background Art

[0002] Infrared imaging is not affected by factors such as light and can work all day long, so it is widely used in security monitoring and other fields. However, limitations in materials and manufacturing processes lead to problems such as loss of details and low image quality in infrared images.

[0003] At present, deep learning algorithms have become a research hotspot in the field of super-resolution due to their powerful information mining capabilities. In related technologies, super-resolution reconstruction methods based on deep learning can be divided into single-branch networks and multi-branch networks according to their task branches. Single-branch networks do not distinguish between feature scales, but directly perform deep superposition of original image features to improve feature performance; however, deep learning networks such as the classic super-resolution reconstruction network VDSR tend to learn the characteristics of simple features, resulting in limited learning ability for details; multi-branch networks improve the detail performance of reconstructed images by fusing information extracted from different feature map branches, but often branch networks still use global features, which limits the further improvement of feature extraction capabilities. Summary of the invention

[0004] In order to solve the above problems existing in the prior art, the present invention provides a method and device for super-resolution reconstruction of infrared images based on feature separation. The technical problem to be solved by the present invention is achieved through the following technical solutions:

[0005] In a first aspect, the present invention provides an infrared image super-resolution reconstruction method based on feature separation, comprising:

[0006] Get the image to be processed;

[0007] Upsampling the image to be processed to obtain a first image;

[0008] Inputting the first image into a pre-trained feature separation network to obtain low-resolution low-frequency features and low-resolution high-frequency features of the first image;

[0009] Determine the structural image of the first image according to the low-resolution and low-frequency features, and obtain the texture information and the noise image of the first image according to the low-resolution and high-frequency features;

[0010] A second image is obtained by super-resolution reconstruction according to the structural image and the texture information.

[0011] In one embodiment of the present invention, the step of determining the structural image of the first image according to the low-resolution and low-frequency features, and obtaining the texture information and the noise image of the first image according to the low-resolution and high-frequency features includes:

[0012] Inputting the low-score and low-frequency features into a pre-trained multi-scale attention module to obtain a structural image of the first image;

[0013] The low-resolution and high-frequency features are input into a pre-trained multi-scale skip connection module to obtain texture information and a noise image of the first image.

[0014] In one embodiment of the present invention, the step of inputting the low-score and low-frequency features into a pre-trained multi-scale attention module to obtain a structural image of the first image includes:

[0015] The low-resolution and low-frequency features are input into a pre-trained multi-scale attention module so that the multi-scale attention module obtains multi-scale features after multiple downsampling of the low-resolution and low-frequency features, and generates a structural image of the first image based on the multi-scale features, the first preset DSSA module, the second preset DSSA module and the third preset DSSA module.

[0016] In one embodiment of the present invention, the feature separation network is trained using the following steps:

[0017] Obtaining training samples; the training samples include a plurality of first sample images and second sample images corresponding to the first sample images, wherein the resolution of the second sample images is greater than the resolution of the first sample images;

[0018] Upsampling the first sample image, where a resolution of the upsampled first sample image is the same as a resolution of the second sample image;

[0019] Inputting the upsampled first sample image into a first feature separation network to be trained to obtain a first low-frequency feature and a first high-frequency feature of the upsampled first sample image, and inputting the second sample image into a second feature separation network to be trained to obtain a second low-frequency feature and a second high-frequency feature of the second sample image; wherein the first feature separation network to be trained and the second feature separation network to be trained have the same structure and share network parameters;

[0020] According to the first low-frequency feature, the first high-frequency feature, the second low-frequency feature, the second high-frequency feature and the preset loss function, the loss values ​​of the current first feature separation network to be trained and the current second feature separation network to be trained are determined, and the loss values ​​are back-propagated to the first feature separation network to be trained and the second feature separation network to be trained until the two converge.

[0021] In one embodiment of the present invention, the preset loss function includes: a high- and low-frequency separation and reconstruction loss function and a low-frequency similarity loss function.

[0022] In one embodiment of the present invention, the preset loss function is:

[0023] L=L recon +λ low L low

[0024] Where, L recon represents the high- and low-frequency separation and reconstruction loss function, L low represents the low-frequency similarity loss function, L low =||LR S -HR S ||1,λ low represents the preset coefficient used to adjust the consistency of low-frequency separation, λ ij Represents the preset reconstruction loss weight coefficient, i D represents the first high frequency feature or the second high frequency feature, j S represents the first low-frequency feature or the second low-frequency feature, i represents the first sample image or the second sample image, j represents the first sample image or the second sample image, LR represents the first sample image, HR represents the second sample image, LR S Represents the first low-frequency feature, HR S Represents the second lowest frequency feature.

[0025] In one embodiment of the present invention, the step of obtaining a second image by super-resolution reconstruction according to the structural image and the detail information comprises:

[0026] The structural image and the detail information are concat-connected, and the concat-connected result is input into a pre-trained convolutional network to obtain a second image after super-resolution reconstruction.

[0027] In a second aspect, the present invention provides an infrared image super-resolution reconstruction device based on feature separation, comprising:

[0028] An acquisition module, used for acquiring an image to be processed;

[0029] An upsampling module, used for upsampling the image to be processed to obtain a first image;

[0030] A feature separation module, used for inputting the first image into a pre-trained feature separation network to obtain low-resolution low-frequency features and low-resolution high-frequency features of the first image;

[0031] A determination module, configured to determine a structural image of the first image according to the low-resolution and low-frequency features, and obtain texture information and a noise image of the first image according to the low-resolution and high-frequency features;

[0032] The reconstruction module is used to obtain a second image by super-resolution reconstruction based on the structural image and the texture information. Compared with the prior art, the beneficial effects of the present invention are:

[0033] The present invention provides a method and device for super-resolution reconstruction of infrared images based on feature separation. Since the feature separation network learns the same part of a low-resolution first sample image and a high-resolution second sample image through a first feature separation network to be trained and a second feature separation network to be trained that share parameters during the training process, the trained feature separation network can achieve high- and low-frequency separation of the low-resolution image and the high-resolution image, thereby enabling the high-frequency features to be better learned, and different networks are used for processing according to the characteristics of the separated feature map image, so as to make full use of the input multi-scale features, further guide the restoration of image details, and improve the amount of information in the image.

[0034] The present invention will be further described in detail below with reference to the accompanying drawings and embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] Figure 1 This is a flow chart of a method for super-resolution reconstruction of infrared images based on feature separation provided by an embodiment of the present invention;

[0036] Figure 2 is a structural schematic diagram of a multi-scale attention module provided by an embodiment of the present invention;

[0037] Figure 3 is a structural schematic diagram of a DSSA module provided in an embodiment of the present invention;

[0038] Figure 4 is a structural schematic diagram of a multi-scale skip connection module provided by an embodiment of the present invention;

[0039] Figure 5 is a structural schematic diagram of a multi-scale connection module provided by an embodiment of the present invention;

[0040] Figure 6 It is a structural schematic diagram of an infrared image super-resolution reconstruction device based on feature separation provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0041] The present invention is further described in detail below with reference to specific embodiments, but the embodiments of the present invention are not limited thereto.

[0042] Figure 1FIG. 1 is a flow chart of a method for super-resolution reconstruction of infrared images based on feature separation provided by an embodiment of the present invention. Figure 1 As shown, an embodiment of the present invention provides an infrared image super-resolution reconstruction method based on feature separation, comprising:

[0043] S1, obtaining the image to be processed;

[0044] S2, upsampling the image to be processed to obtain a first image;

[0045] S3, inputting the first image into a pre-trained feature separation network to obtain low-resolution low-frequency features and low-resolution high-frequency features of the first image;

[0046] S4, determining the structural image of the first image according to the low-resolution and low-frequency features, and obtaining the texture information and the noise image of the first image according to the low-resolution and high-frequency features;

[0047] S5. Super-resolution reconstruction is performed according to the structural image and texture information to obtain a second image.

[0048] In this embodiment, the image to be processed is a low-resolution infrared image, and a sub-pixel method can be used to perform an upsampling operation on it. Sub-pixel upsampling can make full use of the original information of the image to be processed and avoid introducing redundant information during the upsampling process. Then, the first image obtained by upsampling is input into a pre-trained feature separation network. Optionally, the feature separation network consists of five convolutional layers. The convolution kernel size of the first convolutional layer is 3*3, which is used to extract the features of the first image. The second and third convolutional layers use the ReLU activation function based on the 3*3 convolution. The convolution kernel size of the fourth convolutional layer is also 3*3, which is used to distinguish the low-resolution and low-frequency features and low-resolution and high-frequency features of the first image. The fifth layer is a sigmoid activation layer, which is used to limit the low-resolution and low-frequency features and low-resolution and high-frequency features to between [0,1].

[0049] In the above step S4, after the structural image of the first image is obtained by using the low-resolution and low-frequency features, since the low-resolution and low-frequency features contain most of the information representation of the first image, such as all the structural contour information and most of the implicit texture information of the first image, the low-resolution and low-frequency features are further processed for detail recovery to retain the detail information. Finally, super-resolution reconstruction is performed based on the feature information and detail information to obtain the second image.

[0050] Optionally, in the above step S4, the step of determining the structural image of the first image according to the low-resolution and low-frequency features, and obtaining the texture information and the noise image of the first image according to the low-resolution and high-frequency features includes:

[0051] S401, inputting low-score and low-frequency features into a pre-trained multi-scale attention module to obtain a structural image of the first image;

[0052] S402: Input the low-resolution and high-frequency features into a pre-trained multi-scale skip connection module to obtain texture information and a noise image of the first image.

[0053] Specifically, in this embodiment, a pre-trained multi-scale attention module is used to obtain a structural image from low-score and low-frequency features, and a multi-scale jump connection module is used to obtain texture information and a noise image of the first image from low-score and high-frequency features.

[0054] Optionally, in step S401, the step of inputting the low-score and low-frequency features into a pre-trained multi-scale attention module to obtain a structural image of the first image includes:

[0055] The low-score and low-frequency features are input into a pre-trained multi-scale attention module so that the multi-scale attention module obtains multi-scale features after multiple downsampling of the low-score and low-frequency features, and a structural image of the first image is generated according to the multi-scale features, the first preset DSSA module, the second preset DSSA module and the third preset DSSA module.

[0056] Figure 2 Schematic diagram of a multi-scale attention module provided by an embodiment of the present invention. Figure 2 As shown in Figure 2, after the low-score and low-frequency features are input into the pre-trained multi-scale attention module, the multi-scale attention module downsamples the low-score and low-frequency features three times by a factor of 2 to obtain multi-scale features, namely Figure 2 The first-level features, second-level features and third-level features shown in the figure are further input into different scale self-attention DSSA modules with low-score and low-frequency features for processing, and are fused with the higher-level features in sequence starting from the output of the lowest level to form the final structural image.

[0057] Figure 3 is a schematic diagram of the structure of a DSSA module provided by an embodiment of the present invention. Figure 3 , because the scales of input features are different, and the non-local attention mechanism can only process input features of the same scale, the DSSA module no longer uses the global pairing method based on pixel matching, but adopts the block matching method to compress low-scale features into lower-scale descriptors, and use this descriptor as the weight to calculate with low-score and low-frequency features. The specific process is as follows:

[0058] First, three image features are extracted for low-resolution and low-frequency features, which are used for reconstruction and transformation, corresponding to w1 and w2 respectively; then, the block feature w2 extracted at the low scale is linearly spliced ​​as the kernel function weight for convolution matching with the input high-scale feature, and the Softmax function is applied to obtain the self-similarity feature map tmp; another block feature w1 extracted at the low scale is linearly spliced ​​as the transposed convolution kernel weight, and deconvolved with the self-similarity feature map tmp to obtain the output feature.

[0059] It should be understood that the multi-scale attention module overcomes the shortcoming of the non-local attention mechanism that only performs local similarity calculations at a single scale, and extends the local similarity calculation of the image to multi-scale dimensions, thereby better mining information at different scales. The DSSA module overcomes the limitations of pixel-by-pixel matching of non-local attention in related technologies, and uses block-by-block matching, allowing the network to pay more attention to relevant areas while suppressing irrelevant areas. In this process, regional features are compressed into low-dimensional descriptors to reduce the expression of noise.

[0060] Figure 4 : is a structural diagram of a multi-scale jump connection module provided by an embodiment of the present invention. In the above step S402, the low-resolution and high-frequency features are input into a pre-trained multi-scale jump connection module, which uses U-Net as the basic network skeleton and replaces the jump connection in the existing U-Net with a multi-scale connection module. Figure 5 Schematic diagram of a multi-scale connection module provided by an embodiment of the present invention. Figure 5 As shown in the figure, the multi-scale connection module extracts and combines the multi-scale features of the input, where the multi-scale feature branches include four: 1×1 convolution, 3×3 dilated convolution with a dilation rate of 6, 3×3 dilated convolution with a dilation rate of 12, and pooling with a multiple of 2. The multi-scale connection module extracts different scale features in the U-Net encoding process and connects them to the U-Net decoding process at the same scale to guide the formation of output features. Compared with the traditional direct jump 5-connection U-Net, this structure not only removes redundant information (noise, etc.) in the U-Net encoding-decoding process, but also can better retain the details and structural information of the input features.

[0061] Optionally, the feature separation network is trained using the following steps:

[0062] Obtaining training samples; the training samples include a plurality of first sample images and second sample images corresponding to the first sample images, wherein the resolution of the second sample images is greater than the resolution of the first sample images;

[0063] Upsampling the first sample image, where a resolution of the upsampled first sample image is the same as a resolution of the second sample image;

[0064] Inputting the upsampled first sample image into a first feature separation network to be trained to obtain a first low-frequency feature and a first high-frequency feature of the upsampled first sample image;

[0065] Inputting the second sample image into the second feature separation network to be trained to obtain a second low-frequency feature and a second high-frequency feature of the second sample image; the first feature separation network to be trained and the second feature separation network to be trained have the same structure and share network parameters;

[0066] According to the first low-frequency feature, the first high-frequency feature, the second low-frequency feature, the second high-frequency feature and the preset loss function, the loss values ​​of the current first feature separation network to be trained and the current second feature separation network to be trained are determined, and the loss values ​​are back-propagated to the first feature separation network to be trained and the second feature separation network to be trained until the two converge.

[0067] In this embodiment, the training samples include a large number of first sample images and second sample images corresponding to the first sample images, wherein the first sample images and the second sample images are both infrared images, the resolution of the second sample images is greater than the resolution of the first sample images, and the resolution ratio is an integer. The upsampled first sample images and the second sample images are simultaneously input into the first feature separation network to be trained and the second feature separation network to be trained, respectively. The first feature separation network to be trained and the second feature separation network to be trained have the same structure and share network parameters. Based on the theory that there is information overlap between high-resolution images and low-resolution images, the first feature separation network to be trained and the second feature separation network to be trained learn to separate the same low-frequency parts and different high-frequency parts in the two images, and the separation result is the first low-frequency feature LR. S , the first high frequency feature LR D , the second low frequency feature HR S , the second high frequency feature HR D , where the first low-frequency feature LR S With the second low frequency feature HR S is the overlapping part of the two images, and the first high-frequency feature LR D It mainly contains a small amount of high-frequency detail information and various types of noise information.

[0068] In order to better realize feature separation representation, the preset loss function in this embodiment includes two parts: high- and low-frequency separation and reconstruction loss function L recon And the low-frequency similarity loss function L low .

[0069] Specifically, the preset loss function L = L recon +λ low L low , where λ lowA factor used to adjust the consistency of low frequency separation.

[0070] In the formula, the high and low frequency separation reconstruction loss function L recon It can be expressed as:

[0071]

[0072] Low frequency similarity loss function L low It can be expressed as:

[0073] L low =||LR D -HR D ||1

[0074] Where, L recon represents the high- and low-frequency separation and reconstruction loss function, L low represents the low-frequency similarity loss function, L low =||LR S -HR S ||1,λ low represents the preset coefficient used to adjust the consistency of low-frequency separation, λ ij Represents the preset reconstruction loss weight coefficient, i D represents the first high frequency feature or the second high frequency feature, j S represents the first low-frequency feature or the second low-frequency feature, i represents the first sample image or the second sample image, j represents the first sample image or the second sample image, LR represents the first sample image, HR represents the second sample image, LR S Represents the first low-frequency feature, HR S Represents the second lowest frequency feature.

[0075] Optionally, in step S5, the step of obtaining the second image by super-resolution reconstruction according to the structural image and the detail information includes:

[0076] The structural image and detail information are concat-connected, and the concat connection result is input into a pre-trained convolutional network to obtain a second image after super-resolution reconstruction.

[0077] Specifically, in this step, the structural image and the low-resolution and high-frequency features obtained in step S4 are first concat-connected, and then a simple convolutional network is used to restore the reconstructed high-resolution second image.

[0078] Figure 6 FIG. 1 is a schematic diagram of a structure of an infrared image super-resolution reconstruction device based on feature separation provided by an embodiment of the present invention. Figure 6 As shown, an embodiment of the present invention provides an infrared image super-resolution reconstruction device based on feature separation, comprising:

[0079] An acquisition module 610 is used to acquire an image to be processed;

[0080] An upsampling module 620, configured to upsample the image to be processed to obtain a first image;

[0081] A feature separation module 630 is used to input the first image into a pre-trained feature separation network to obtain low-resolution low-frequency features and low-resolution high-frequency features of the image to be processed;

[0082] A determination module 640, configured to determine a structural image of the first image according to the low-resolution and low-frequency features, and obtain texture information and a noise image of the first image according to the low-resolution and high-frequency features;

[0083] The reconstruction module 650 is used to obtain a second image by super-resolution reconstruction according to the structural image and the texture information.

[0084] It can be seen from the above embodiments that the beneficial effects of the present invention are:

[0085] The present invention provides a method and device for super-resolution reconstruction of infrared images based on feature separation. Since the feature separation network learns the same part of a low-resolution first sample image and a high-resolution second sample image through a first feature separation network to be trained and a second feature separation network to be trained that share parameters during the training process, the trained feature separation network can achieve high- and low-frequency separation of the low-resolution image and the high-resolution image, thereby enabling the high-frequency features to be better learned, and different networks are used for processing according to the characteristics of the separated feature map image, so as to make full use of the input multi-scale features, further guide the restoration of image details, and improve the amount of information in the image.

[0086] In the description of the present invention, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of the features. In the description of the present invention, the meaning of "plurality" is two or more, unless otherwise clearly and specifically defined.

[0087] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine different embodiments or examples described in this specification.

[0088] Although the present application is described herein in conjunction with various embodiments, in the process of implementing the claimed application, those skilled in the art may understand and implement other variations of the disclosed embodiments by viewing the drawings, the disclosure, and the appended claims. In the claims, the word "comprising" does not exclude other components or steps, and "one" or "an" does not exclude multiple situations. A single processor or other unit may implement several functions listed in a claim. Certain measures are recorded in different dependent claims, but this does not mean that these measures cannot be combined to produce good results.

[0089] The above contents are further detailed descriptions of the present invention in combination with specific preferred embodiments, and it cannot be determined that the specific implementation of the present invention is limited to these descriptions. For ordinary technicians in the technical field to which the present invention belongs, several simple deductions or substitutions can be made without departing from the concept of the present invention, which should be regarded as falling within the protection scope of the present invention.

Claims

1. A method for super-resolution reconstruction of infrared images based on feature separation, characterized in that: include: Get the image to be processed; Upsampling the image to be processed to obtain a first image; Inputting the first image into a pre-trained feature separation network to obtain low-resolution low-frequency features and low-resolution high-frequency features of the first image; Determine the structural image of the first image according to the low-resolution and low-frequency features, and obtain the texture information and the noise image of the first image according to the low-resolution and high-frequency features; According to the structural image and the texture information, super-resolution reconstruction is performed to obtain a second image; The feature separation network is trained using the following steps: Obtaining training samples; the training samples include a plurality of first sample images and second sample images corresponding to the first sample images, wherein the resolution of the second sample images is greater than the resolution of the first sample images; Upsampling the first sample image, where a resolution of the upsampled first sample image is the same as a resolution of the second sample image; Inputting the upsampled first sample image into a first feature separation network to be trained to obtain a first low-frequency feature and a first high-frequency feature of the upsampled first sample image, and inputting the second sample image into a second feature separation network to be trained to obtain a second low-frequency feature and a second high-frequency feature of the second sample image; wherein the first feature separation network to be trained and the second feature separation network to be trained have the same structure and share network parameters; Determine the loss values ​​of the current first feature separation network to be trained and the current second feature separation network to be trained according to the first low-frequency feature, the first high-frequency feature, the second low-frequency feature, the second high-frequency feature and a preset loss function, and back-propagate the loss values ​​to the first feature separation network to be trained and the second feature separation network to be trained until the two converge; The step of determining the structural image of the first image according to the low-resolution and low-frequency features, and obtaining the texture information and the noise image of the first image according to the low-resolution and high-frequency features comprises: Inputting the low-score and low-frequency features into a pre-trained multi-scale attention module to obtain a structural image of the first image; The low-resolution and high-frequency features are input into a pre-trained multi-scale skip connection module to obtain texture information and a noise image of the first image.

2. The infrared image super-resolution reconstruction method based on feature separation according to claim 1 is characterized in that: The step of inputting the low-score and low-frequency features into a pre-trained multi-scale attention module to obtain a structural image of the first image comprises: The low-resolution and low-frequency features are input into a pre-trained multi-scale attention module so that the multi-scale attention module obtains multi-scale features after multiple downsampling of the low-resolution and low-frequency features, and a structural image of the first image is generated according to the multi-scale features, a first preset different-scale self-attention DSSA module, a second preset DSSA module and a third preset DSSA module.

3. The infrared image super-resolution reconstruction method based on feature separation according to claim 1 is characterized in that: The preset loss function includes: a high- and low-frequency separation and reconstruction loss function and a low-frequency similarity loss function.

4. The infrared image super-resolution reconstruction method based on feature separation according to claim 3 is characterized in that: The preset loss function is: L=L recon +λ low L low Where, L recon represents the high- and low-frequency separation and reconstruction loss function, L low represents the low-frequency similarity loss function, L low =‖LR S -HR S ‖1,λ low represents the preset coefficient used to adjust the consistency of low-frequency separation, λ ij Represents the preset reconstruction loss weight coefficient, i D represents the first high frequency feature or the second high frequency feature, j S represents the first low-frequency feature or the second low-frequency feature, i represents the first sample image or the second sample image, j represents the first sample image or the second sample image, LR represents the first sample image, HR represents the second sample image, LR S Represents the first low-frequency feature, HR S represents the second lowest frequency feature, and ° represents the Hadamard product.

5. An infrared image super-resolution reconstruction device based on feature separation, characterized in that: include: An acquisition module, used for acquiring an image to be processed; An upsampling module, used for upsampling the image to be processed to obtain a first image; A feature separation module, used for inputting the first image into a pre-trained feature separation network to obtain low-resolution low-frequency features and low-resolution high-frequency features of the first image; A determination module, configured to determine a structural image of the first image according to the low-resolution and low-frequency features, and obtain texture information and a noise image of the first image according to the low-resolution and high-frequency features; A reconstruction module, used for super-resolution reconstruction to obtain a second image according to the structural image and the texture information; The feature separation network is trained using the following steps: Obtaining training samples; the training samples include a plurality of first sample images and second sample images corresponding to the first sample images, wherein the resolution of the second sample images is greater than the resolution of the first sample images; Upsampling the first sample image, where a resolution of the upsampled first sample image is the same as a resolution of the second sample image; Inputting the upsampled first sample image into a first feature separation network to be trained to obtain a first low-frequency feature and a first high-frequency feature of the upsampled first sample image, and inputting the second sample image into a second feature separation network to be trained to obtain a second low-frequency feature and a second high-frequency feature of the second sample image; wherein the first feature separation network to be trained and the second feature separation network to be trained have the same structure and share network parameters; Determine the loss values ​​of the current first feature separation network to be trained and the current second feature separation network to be trained according to the first low-frequency feature, the first high-frequency feature, the second low-frequency feature, the second high-frequency feature and a preset loss function, and back-propagate the loss values ​​to the first feature separation network to be trained and the second feature separation network to be trained until the two converge; The determination module is specifically used to input the low-score and low-frequency features into a pre-trained multi-scale attention module to obtain the structural image of the first image; and input the low-score and high-frequency features into a pre-trained multi-scale jump connection module to obtain the texture information and noise image of the first image.

Images