Unsupervised magnetic resonance image super-resolution method and device based on multi-channel down-sampling

Abstract

The application discloses a method and device for unsupervised magnetic resonance image super-resolution based on multi-channel down-sampling, which comprises the following steps: (1) obtaining a high-resolution auxiliary image and corresponding training labels, wherein the auxiliary image is obtained by scanning a living body by using a high-resolution sequence of a magnetic resonance scanner, and the training labels are n original resolution images obtained by sampling and scanning the same living body by using different scanning settings; (2) constructing a convolutional neural network as a generation network, inputting the auxiliary image, and obtaining a high-resolution image; (3) performing down-sampling on the generated high-resolution image by using multi-channel down-sampling; (4) calculating the loss of the n original resolution images obtained in the step (3) after down-sampling and the training labels respectively, and updating the generation network; (5) iteratively training the generation network and prematurely stopping the output image: repeating the steps (2)-(4), reducing the loss size, and stopping the cycle to output the image after a predetermined stop condition is reached.

Description

Technical Field

[0001] This invention belongs to the field of deep learning image super-resolution, and particularly relates to an unsupervised magnetic resonance image super-resolution method and apparatus based on multi-channel downsampling. Background Technology

[0002] Magnetic resonance imaging (MRI) is a powerful and harmless imaging technique with many advantages. However, some MRI methods have limitations in obtaining high-quality, high-resolution images due to the limitations of the scanning method. These include arterial spin labeling imaging, diffusion MRI, and functional magnetic resonance imaging (fMRI).

[0003] For example, arterial spin labeling imaging, which measures cerebral blood flow using endogenous tracers (i.e., arterial water protons labeled by radiofrequency pulses) without the need for exogenous injection, has wide applications in vascular diseases. However, the labeling effect is weak, resulting in low signal-to-noise ratio and low spatial resolution. Efficiently and cost-effectively acquiring high-resolution images of this type of magnetic resonance imaging is a pressing problem in this field.

[0004] In the field of magnetic resonance imaging, supervised deep learning methods are commonly used for image super-resolution. However, these methods require low-resolution to high-resolution image pairs for supervised training of the network. For arterial spin labeling imaging, diffusion magnetic resonance imaging, and functional magnetic resonance imaging, high-resolution images are often difficult to obtain. Summary of the Invention

[0005] The purpose of this application is to provide an unsupervised super-resolution method and apparatus for magnetic resonance imaging based on multi-channel downsampling, which addresses the challenges of acquiring high-resolution images in magnetic resonance imaging methods such as arterial spin labeling imaging, diffusion magnetic resonance imaging, and functional magnetic resonance imaging.

[0006] According to a first aspect of the embodiments of this application, an unsupervised magnetic resonance image super-resolution method based on multi-channel downsampling is provided, comprising:

[0007] (1) Obtain high-resolution auxiliary images and corresponding training labels, wherein the high-resolution auxiliary images are obtained by scanning the organism using a high-resolution sequence of a magnetic resonance scanner, and the training labels are n original resolution images obtained by performing n different sampling scans on the same organism using different scanning settings.

[0008] (2) Generate high-resolution images through a generative network: Construct a convolutional neural network as a generative network, limit the scale of the output of the generative network to the target resolution, input the high-resolution auxiliary image to provide prior information, and obtain a high-resolution image;

[0009] (3) Downsample the generated high-resolution image by multi-channel downsampling: Use n sampling settings corresponding to the training labels acquired in step (1) to downsample the high-resolution image to obtain n original resolution images corresponding to the training labels;

[0010] (4) Calculate loss and update network: Calculate the loss function for each of the n downsampled original resolution images obtained in step (3) and the training labels, and pass the loss function to the generator network in step (2) to update the generator network;

[0011] (5) Iteratively train the network and stop the output image early: Repeat steps (2), (3), and (4) to reduce the value of the loss function in step (4) and improve the quality of the high-resolution image generated in step (2). Stop the loop output image after the predetermined stopping condition is met.

[0012] Furthermore, in magnetic resonance imaging (MRI) scans, different original resolution images using different sampling methods can be obtained through different scanning settings, or different original resolution images can be introduced through the subject's own minute movements such as breathing during the scan.

[0013] Furthermore, a 3D U-net network is used as the generating network.

[0014] Furthermore, the different downsampling modules correspond to the scanning settings used when obtaining the original resolution image in step (1). Specifically, the generated high-resolution image is transformed according to different translation, rotation and scaling parameters. These parameters can be calculated by affine transformation of the high-resolution auxiliary image and the corresponding training label image. Alternatively, the corresponding 7 parameters of each original resolution image can be saved during scanning for use in the downsampling step of the network.

[0015] Furthermore, for the n downsampled original resolution images, the mean square error is calculated and the mean value is obtained by comparing them with the n original resolution images obtained in step (1).

[0016] Furthermore, the generator network is updated using the gradient descent algorithm.

[0017] According to a second aspect of the embodiments of this application, an unsupervised magnetic resonance image super-resolution device based on multi-channel downsampling is provided, comprising:

[0018] The acquisition module is used to acquire high-resolution auxiliary images and corresponding training labels, wherein the high-resolution auxiliary images are obtained by scanning an organism using a high-resolution sequence of a magnetic resonance scanner, and the training labels are n original resolution images obtained by performing n different sampling scans on the same organism using different scanning settings.

[0019] A generation module is used to generate high-resolution images through a generation network: a convolutional neural network is constructed as the generation network, the scale of the output of the generation network is limited to the target resolution, the high-resolution auxiliary image is input to provide prior information, and a high-resolution image is obtained.

[0020] The downsampling module is used to downsample the generated high-resolution image through multi-channel downsampling: the high-resolution image is downsampled using n sampling settings corresponding to the training labels acquired by the acquisition module to obtain n original resolution images corresponding to the training labels;

[0021] The calculation module is used to calculate the loss and update the network: calculate the loss for each of the n downsampled low-resolution images obtained by the downsampling module and the training labels, and pass the loss to the generation network in the generation module to update the generation network;

[0022] The training module is used to iteratively train the generator network and stop the output image early: it loops through the generation module, downsampling module, and calculation module to reduce the value of the loss function, improve the quality of the high-resolution image generated in the generation module, and stop the loop output image when an image that meets the predetermined requirements is obtained.

[0023] According to a third aspect of the embodiments of this application, an electronic device is provided, comprising:

[0024] One or more processors;

[0025] Memory, used to store one or more programs;

[0026] When the one or more programs are executed by the one or more processors, the one or more processors perform the method as described in the first aspect.

[0027] According to a fourth aspect of the embodiments of this application, a computer-readable storage medium is provided, having stored thereon computer instructions, characterized in that, when executed by a processor, the instructions implement the steps of the method as described in the first aspect.

[0028] The technical solutions provided by the embodiments of this application may include the following beneficial effects:

[0029] As can be seen from the above embodiments, the unsupervised magnetic resonance super-resolution imaging network based on conditional depth image priors in this application can achieve good super-resolution results without high-resolution images as training labels. This solves the problem that high-resolution images are difficult to obtain for certain special magnetic resonance imaging methods, making it impossible to use supervised deep learning methods to improve resolution. Simultaneously, this invention uses auxiliary images with the target resolution as network input, which can provide anatomical structural information, further helping to improve the quality of the output high-resolution image. This invention employs a multi-channel downsampling method, fully utilizing the information differences in multiple input low-resolution images to improve the image quality of the final output high-resolution image.

[0030] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and do not limit this application. Attached Figure Description

[0031] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0032] Figure 1 This is a flowchart illustrating an unsupervised magnetic resonance image super-resolution method based on multi-channel downsampling, according to an exemplary embodiment.

[0033] Figure 2 The images shown are the output results of this method. (a) is the original low-resolution image (size 16*88*72), (b) is a magnified view of the marked area in (a), (c) is the high-resolution image obtained using this method (size 32*176*144), and (d) is a magnified view of the marked area in (c).

[0034] Figure 3 The output images are obtained using existing methods. (a) is a high-resolution image (size 16*88*72) obtained by directly performing linear interpolation on the original low-resolution image. (b) is an enlarged view of the marked area in (a).

[0035] Figure 4 This is a block diagram illustrating an unsupervised magnetic resonance imaging super-resolution device based on multi-channel downsampling, according to an exemplary embodiment.

[0036] Figure 5 This is a schematic diagram of an electronic device according to an exemplary embodiment. Detailed Implementation

[0037] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application.

[0038] The terminology used in this application is for the purpose of describing particular embodiments only and is not intended to be limiting of the application. The singular forms “a,” “the,” and “the” used in this application and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used herein refers to and includes any or all possible combinations of one or more of the associated listed items.

[0039] It should be understood that although the terms first, second, third, etc., may be used in this application to describe various information, such information should not be limited to these terms. These terms are only used to distinguish information of the same type from one another. For example, without departing from the scope of this application, first information may also be referred to as second information, and similarly, second information may also be referred to as first information. Depending on the context, the word "if" as used herein may be interpreted as "when," "when," or "in response to determination."

[0040] Figure 1 This is a flowchart illustrating an unsupervised magnetic resonance image super-resolution method based on multi-channel downsampling, according to an exemplary embodiment, as follows: Figure 1 As shown, the method may include the following steps:

[0041] (1) Acquire high-resolution auxiliary images and corresponding training labels, wherein the high-resolution auxiliary images are obtained by scanning the organism with a high-resolution sequence of a magnetic resonance scanner, and the training labels are n original resolution images obtained by performing n different sampling scans on the same organism with different scanning settings.

[0042] (2) Generate high-resolution images through a generative network: Construct a convolutional neural network as a generative network, limit the scale of the output of the generative network to the target resolution, input the high-resolution auxiliary image to provide prior information, and obtain the target resolution image;

[0043] (3) Downsample the generated high-resolution image by multi-channel downsampling: Use n sampling settings corresponding to the training labels acquired in step (1) to downsample the high-resolution image to obtain n original resolution images corresponding to the training labels;

[0044] (4) Calculate loss and update network: Calculate the loss function for each of the n downsampled original resolution images obtained in step (3) and the training labels, and pass the loss function to the generator network in step (2) to update the generator network using the gradient descent algorithm;

[0045] (5) Iteratively train the network and stop the output image early: Repeat steps (2), (3), and (4) to reduce the value of the loss function in step (4) and improve the quality of the high-resolution image generated in step (2). Stop the output image after the stopping condition is met. The stopping condition is usually the optimal number of training times required to validate the data.

[0046] As can be seen from the above embodiments, the unsupervised magnetic resonance super-resolution imaging network based on conditional depth image priors in this application can achieve good super-resolution results without high-resolution images as training labels. This solves the problem that some special magnetic resonance imaging methods are difficult to obtain high-resolution images, making it impossible to use supervised deep learning methods to improve resolution. Simultaneously, this invention uses high-resolution auxiliary images as network input, which can provide anatomical structural information, further helping to improve the quality of the output high-resolution image. This invention employs a multi-channel downsampling method, fully utilizing the information differences in multiple original resolution images obtained through different sampling methods to improve the image quality of the final output high-resolution image.

[0047] In the specific implementation of step (1), a high-resolution auxiliary image and a corresponding training label are obtained. The high-resolution auxiliary image is obtained by scanning the organism with a high-resolution sequence (such as T1 weighted) of a magnetic resonance scanner. The training label is n original resolution images obtained by performing n different sampling scans on the same organism with different scanning settings, where 10≥n≥1.

[0048] Specifically, the high-resolution auxiliary image can be a T1-weighted image, etc. In magnetic resonance imaging (MRI), different original resolution images using different sampling methods can usually be obtained through different scanning settings (such as translation, rotation, scaling, etc. of the scanning slice position), such as arterial spin-labeled images, diffusion MRI images, functional MRI images, etc. These images are limited by the imaging principle and have a low original resolution. Alternatively, they can be introduced through the subject's own subtle movements such as breathing during the scan. It can be described by seven parameters, including three translational degrees of freedom, three rotational degrees of freedom, and one scaling factor, similar to the description of rigid body motion parameters. These parameters can be obtained through affine transformation of training labels and high-resolution auxiliary images, or the corresponding seven parameters of each original resolution image can be saved during the scan for use in the downsampling step of the network.

[0049] In the specific implementation of step (2), a high-resolution image is generated by a generative network: a convolutional neural network is constructed as the generative network, the scale of the output of the generative network is limited to the target resolution, the high-resolution auxiliary image is input to provide prior information, and a high-resolution image is obtained.

[0050] Specifically, step (2) includes the following sub-steps:

[0051] (2.1) Construct a generator network G with input size (S T1 H T1 W T1 The output size is (S) HR H HR W HR );

[0052] Specifically, a 3D U-net network structure can be used. The input size is the size of the auxiliary high-resolution image (such as a T1-weighted image), and the output size is the size of the target high-resolution image (usually the target high-resolution image and the auxiliary high-resolution image have the same resolution unless there is a specific requirement). This method does not have specific requirements for the network structure; other network structures can be used. 3D U-net is chosen here because it is a commonly used network structure in medical image processing. Using the auxiliary high-resolution image (such as a T1-weighted image) as the network input effectively utilizes the structural information of the auxiliary image to help improve the quality of the high-resolution image output by the network.

[0053] (2.2) Using the generator network G obtained in step (2.1), the input size is (S T1 H T1 W T1 The auxiliary image T1 is obtained with size (S) HR H HR W HR High-resolution images (HR) of )

[0054] Specifically, this step is a conventional technical method in this field and will not be described in detail here.

[0055] In the specific implementation of step (3), the generated high-resolution image is downsampled by multi-channel downsampling: the high-resolution image is downsampled using n sampling settings corresponding to those used when collecting training labels in step (1) to obtain n original resolution images corresponding to the training labels;

[0056] Furthermore, step (3) includes the following sub-steps:

[0057] (3.1) Apply n downsampling methods to the high-resolution image obtained in step (2.2), corresponding to the downsampling methods used when acquiring training labels in step (1). Each downsampling method undergoes an affine transformation based on the corresponding seven parameters, and the scale after sampling is (S). LR H LR W LR The LR image is scaled the same as the original resolution image input. HR1 LR HR2 …LR HRn There are n images in total.

[0058] Specifically, the implementation of this step is the same as step (1), and will not be repeated here.

[0059] In the specific implementation of step (4), the loss is calculated to update the network: the loss function is calculated for each of the n downsampled original resolution images obtained in step (3) and the training labels, and the loss function is passed to the generator network in step (2), and the generator network is updated using the gradient descent algorithm;

[0060] Step (4) includes the following sub-steps:

[0061] (4.1) For the n downsampled images obtained in step (3.1), calculate the mean square error and the mean value of the n input original resolution images to obtain the loss L;

[0062] (4.2) Based on the loss L obtained in step (4.1), the generator network G in step (1) is updated using the gradient descent algorithm.

[0063] Specifically, in this embodiment, mean squared error is used as the loss. Those skilled in the art can also choose other methods to calculate the loss as needed. The same applies to the network update method.

[0064] In the specific implementation of step (5), the network is trained iteratively and the output image is stopped in advance: steps (2), (3), and (4) are repeated, the value of the loss function in step (4) is reduced, the quality of the high-resolution image generated in step (2) is improved, and the output image is stopped after the stopping condition is met.

[0065] Step (5) includes the following sub-steps:

[0066] (5.1) Repeat steps (2.2) to (4.2) to reduce the value of the loss function L in step (4.1) and improve the image quality of the high-resolution image HR obtained in step (2.2);

[0067] (5.2) After the stopping condition is met, the loop is terminated and the image is output.

[0068] In practice, the stopping condition is usually the optimal number of training iterations required using validation data.

[0069] This method constructs a multi-channel downsampling module to fully utilize information from the original resolution image set obtained through various sampling methods, thereby improving the quality of the generated high-resolution image. This module is included in an unsupervised magnetic resonance super-resolution imaging network. The downsampling operation is determined by the sampling information introduced during the original image acquisition. Based on the parameters of the original image acquisition, the high-resolution image output by the network is subjected to the same downsampling method as the original acquisition. A loss function is calculated using the corresponding original resolution image labels to fully utilize the information from different downsampled images, thus improving the image quality of the final output high-resolution image. The downsampling rate of this module can be arbitrarily adjusted, allowing for flexible control of the super-resolution ratio according to the task objectives.

[0070] An unsupervised magnetic resonance super-resolution imaging network was constructed based on the principle of conditional depth image prior. It utilizes a target resolution auxiliary image as input to provide structural information and multiple original resolution images as training labels for unsupervised learning. This eliminates the need for a target high-resolution image as a training label, and high-resolution images can be output by early termination of training. This network effectively addresses the problem that high-resolution images are unavailable as training labels for some magnetic resonance images.

[0071] Implementation Examples

[0072] An embodiment of the invention was implemented on a machine equipped with an Intel Core i7-9750H central processing unit, an NVIDIA GTX1650 graphics processor, and 32GB of memory. Low-resolution ASL images (such as...) were used. Figure 2 Images (a) and (b) (16*88*72 pixels) were used as training labels. A corresponding high-resolution auxiliary image, a T1-weighted image (32*176*144 pixels), was used as network input. The network was trained for 800 epochs using the corresponding downsampling method, yielding the experimental results shown in the attached figure. The figure is a mid-section of a 3D image from an ASL (Automatic Brain Resonance Imaging) MRI of the brain.

[0073] like Figure 2 as well as Figure 3 As shown, compared with existing methods (such as...) Figure 3 Compared to (a) and (b) in the previous method, this method (e.g.) Figure 2 (c) and (d) in the figure can achieve super-resolution of ASL images and obtain high-resolution images with high quality (size 32*176*144).

[0074] Corresponding to the aforementioned embodiments of the unsupervised magnetic resonance image super-resolution method based on multi-channel downsampling, this application also provides embodiments of an unsupervised magnetic resonance image super-resolution device based on multi-channel downsampling.

[0075] Figure 4 This is a block diagram of an unsupervised magnetic resonance imaging super-resolution device based on multi-channel downsampling, according to an exemplary embodiment. (Refer to...) Figure 4 The device may include:

[0076] The acquisition module 21 is used to acquire high-resolution auxiliary images and corresponding training labels. The high-resolution auxiliary images are obtained by scanning the organism using a high-resolution sequence (such as a T1-weighted image) of a magnetic resonance scanner. The training labels are n original low-resolution arterial spin label images (or diffuse magnetic resonance images, functional magnetic resonance images, etc., which are limited by the imaging principle and have low original resolution) obtained by performing n different sampling scans on the same organism with different scanning settings.

[0077] The generation module 22 is used to generate a high-resolution image through a generation network: a convolutional neural network is constructed as the generation network, the scale of the output of the generation network is limited to the target resolution, the high-resolution auxiliary image is input to provide prior information, and a high-resolution image is obtained.

[0078] Downsampling module 23 is used to downsample the generated high-resolution image through multi-channel downsampling: the high-resolution image is downsampled using n sampling settings corresponding to the training labels to obtain n original resolution images corresponding to the training labels;

[0079] The calculation module 24 is used to calculate the loss and update the network: calculate the loss for each of the n downsampled original resolution images obtained by the downsampling module and the training labels, and pass the loss to the generation network in the generation module, and update the generation network using the gradient descent algorithm;

[0080] Training module 25 is used to iteratively train the generation network and stop the output image early: it loops through the generation module, downsampling module, and calculation module to reduce the value of the loss function, improve the quality of the high-resolution image generated in the generation module, and stop the loop output image when an image that meets the predetermined requirements is obtained.

[0081] Regarding the apparatus in the above embodiments, the specific manner in which each module performs its operation has been described in detail in the embodiments related to the method, and will not be elaborated upon here.

[0082] For the device embodiments, since they basically correspond to the method embodiments, the relevant parts can be referred to in the description of the method embodiments. The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this application according to actual needs. Those skilled in the art can understand and implement this without creative effort.

[0083] Accordingly, this application also provides an electronic device, comprising: one or more processors; a memory for storing one or more programs; and, when the one or more programs are executed by the one or more processors, causing the one or more processors to implement the unsupervised magnetic resonance image super-resolution method based on multi-channel downsampling as described above. Figure 5 The diagram shown is a hardware structure diagram of any device with data processing capabilities, used in an embodiment of the present invention to provide an unsupervised magnetic resonance image super-resolution method based on multi-channel downsampling. Except for... Figure 5 In addition to the processor, memory, and network interface shown, any data processing device in the embodiment may also include other hardware depending on the actual function of the data processing device, which will not be described in detail here.

[0084] Accordingly, this application also provides a computer-readable storage medium storing computer instructions that, when executed by a processor, implement the unsupervised magnetic resonance image super-resolution method based on multi-channel downsampling as described above. The computer-readable storage medium can be an internal storage unit of any data-processing device as described in any of the foregoing embodiments, such as a hard disk or memory. The computer-readable storage medium can also be an external storage device, such as a plug-in hard disk, smart media card (SMC), SD card, flash card, etc., equipped on the device. Furthermore, the computer-readable storage medium can include both internal storage units of any data-processing device and external storage devices. The computer-readable storage medium is used to store the computer program and other programs and data required by the data-processing device, and can also be used to temporarily store data that has been output or will be output.

[0085] Other embodiments of this application will readily occur to those skilled in the art upon consideration of the specification and practice of the disclosure herein. This application is intended to cover any variations, uses, or adaptations of this application that follow the general principles of this application and include common knowledge or customary techniques in the art not disclosed herein.

[0086] It should be understood that this application is not limited to the precise structure described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope.

Claims

1. An unsupervised super-resolution method for magnetic resonance images based on multi-channel downsampling, characterized in that, include: (1) Obtain high-resolution auxiliary images and corresponding training labels, wherein the high-resolution auxiliary images are obtained by scanning the organism using a high-resolution sequence of a magnetic resonance scanner, and the training labels are n original resolution images obtained by performing n different sampling scans on the same organism using different scanning settings. (2) Generate high-resolution images through a generative network: Construct a convolutional neural network as a generative network, limit the scale of the output of the generative network to the target resolution, input the high-resolution auxiliary image to provide prior information, and obtain a high-resolution image; (3) Downsample the generated high-resolution image by multi-channel downsampling: Use n sampling settings corresponding to the training labels acquired in step (1) to downsample the high-resolution image to obtain n original resolution images corresponding to the training labels; (4) Calculate loss and update network: Calculate the loss function for each of the n downsampled original resolution images obtained in step (3) and the training labels, and pass the loss function to the generator network in step (2) to update the generator network; (5) Iteratively train the network and stop the output image early: Repeat steps (2), (3), and (4) to reduce the value of the loss function in step (4) and improve the quality of the high-resolution image generated in step (2). Stop the loop output image after the predetermined stopping condition is met.

2. The method according to claim 1, characterized in that, In magnetic resonance imaging (MRI) scans, different original resolution images can be obtained by using different scanning settings and sampling methods, or by introducing different original resolution images through the subject's own minute movements such as breathing during the scan.

3. The method according to claim 1, characterized in that, A 3D U-net network is used as the generating network.

4. The method according to claim 1, characterized in that, Different downsampling modules correspond to the scanning settings used when obtaining the original resolution image in step (1). Specifically, the generated high-resolution image is transformed according to different translation, rotation and scaling parameters. These parameters can be calculated by affine transformation of the high-resolution auxiliary image and the corresponding training label image. Alternatively, the corresponding 7 parameters of each original resolution image can be saved during scanning for use in the downsampling step of the network.

5. The method according to claim 1, characterized in that, For each of the n downsampled original resolution images, the mean square error is calculated and the mean value is obtained to obtain the loss.

6. The method according to claim 1, characterized in that, The generator network is updated using the gradient descent algorithm.

7. An unsupervised magnetic resonance imaging super-resolution device based on multi-channel downsampling, characterized in that, include: The acquisition module is used to acquire high-resolution auxiliary images and corresponding training labels, wherein the high-resolution auxiliary images are obtained by scanning an organism using a high-resolution sequence of a magnetic resonance scanner, and the training labels are n original resolution images obtained by performing n different sampling scans on the same organism using different scanning settings. A generation module is used to generate high-resolution images through a generation network: a convolutional neural network is constructed as the generation network, the scale of the output of the generation network is limited to the target resolution, the high-resolution auxiliary image is input to provide prior information, and a high-resolution image is obtained. The downsampling module is used to downsample the generated high-resolution image through multi-channel downsampling: the high-resolution image is downsampled using n sampling settings corresponding to the training labels acquired by the acquisition module to obtain n original resolution images corresponding to the training labels; The calculation module is used to calculate the loss and update the network: calculate the loss for each of the n downsampled low-resolution images obtained by the downsampling module and the training labels, and pass the loss to the generation network in the generation module to update the generation network; The training module is used to iteratively train the generator network and stop the output image early: it loops through the generation module, downsampling module, and calculation module to reduce the value of the loss function, improve the quality of the high-resolution images generated in the generation module, and stop the loop output image when an image that meets the predetermined requirements is obtained.

8. An electronic device, characterized in that, include: One or more processors; Memory, used to store one or more programs; When the one or more programs are executed by the one or more processors, the one or more processors implement the method as described in any one of claims 1-6.

9. A computer-readable storage medium storing computer instructions thereon, characterized in that, When executed by the processor, this instruction implements the steps of the method as described in any one of claims 1-6.

Images