image scaling

Abstract

According to implementations of the present disclosure, a scheme for image scaling is proposed. According to the scheme, an input image having a first resolution is obtained. An output image having a second resolution and high-frequency information subject to a predetermined distribution are generated based on the input image by using a trained invertible neural network, where the first resolution is higher than the second resolution, and the input image and the output image have the same semantics. Furthermore, another input image having the second resolution is obtained. Another output image having the first resolution is generated based on the other input image and the high-frequency information subject to the predetermined distribution by using an inverse network of the invertible neural network, where the other input image and the other output image have the same semantics. The scheme is capable of scaling down original images into low-resolution images with the same semantics and visually pleasing, and is capable of reconstructing high-quality high-resolution images from the low-resolution images.

Description

Background Technology

[0001] Image scaling is one of the most common operations when processing digital images. On the one hand, with the widespread use of high-resolution images / videos on the internet, image downscaling is essential for storing, transmitting, and sharing such large amounts of data, as downscaled images can significantly save storage space and efficiently improve bandwidth utilization while maintaining the same semantic information. On the other hand, many such image downscaling scenarios inevitably place high demands on the inverse task (i.e., enlarging the downscaled image to its original size).

[0002] Traditional image downscaling (i.e., reducing a high-resolution image to a low-resolution image) often results in the loss of high-frequency information in the high-resolution image. Due to the loss of high-frequency information, traditional image upscaling (i.e., enlarging a low-resolution image to a high-resolution image) often fails to reconstruct a high-quality high-resolution image from a low-resolution image. Summary of the Invention

[0003] According to an implementation of this disclosure, a scheme for image scaling is proposed. According to this scheme, an input image with a first resolution is acquired. Using a trained invertible neural network, an output image with a second resolution and high-frequency information following a predetermined distribution are generated based on the input image, wherein the first resolution is higher than the second resolution, and the input and output images have the same semantics. Furthermore, another input image with the second resolution is acquired. Using the inverse network of the invertible neural network, another output image with a first resolution is generated based on the other input image and high-frequency information following a predetermined distribution, wherein the other input image and the other output image have the same semantics. This scheme can reduce the original image to a low-resolution image with the same semantics and visual appeal, and can reconstruct a high-quality high-resolution image from the low-resolution image.

[0004] The summary section is provided to present the chosen concepts in a simplified form, which will be further described in the detailed description below. The summary section is not intended to identify key or principal features of the claimed subject matter, nor is it intended to limit the scope of the claimed subject matter. Attached Figure Description

[0005] Figure 1A A schematic block diagram of a computing device capable of implementing multiple implementations of the present disclosure is shown;

[0006] Figure 1B A schematic diagram illustrating the working principle of an image scaling module implemented according to this disclosure is shown;

[0007] Figure 2A A schematic block diagram of a reversible neural network implemented according to this disclosure is shown;

[0008] Figure 2B A schematic diagram of an example reversible neural network unit according to an implementation of this disclosure is shown;

[0009] Figure 3A A schematic block diagram of the inverse network of a reversible neural network implemented according to the present disclosure is shown;

[0010] Figure 3B A schematic diagram of an example reversible neural network unit according to an implementation of this disclosure is shown;

[0011] Figure 4 A flowchart of an example method for image scaling according to an implementation of this disclosure is shown;

[0012] Figure 5 A flowchart illustrating an example method for image scaling according to an implementation of this disclosure is shown; and

[0013] Figure 6 A block diagram of an example system capable of implementing the present disclosure is shown.

[0014] In these accompanying figures, the same or similar reference symbols are used to indicate the same or similar elements. Detailed Implementation

[0015] This disclosure will now be discussed with reference to several example implementations. It should be understood that these implementations are discussed only to enable those skilled in the art to better understand and thus implement this disclosure, and not to imply any limitation on the scope of this disclosure.

[0016] As used herein, the term "comprising" and its variations are to be interpreted as open-ended terms meaning "including but not limited to". The term "based on" is to be interpreted as "at least partially based on". The terms "an implementation" and "an implementation" are to be interpreted as "at least one implementation". The term "another implementation" is to be interpreted as "at least one other implementation". The terms "first", "second", etc., may refer to different or the same objects. Other explicit and implicit definitions may also be included below.

[0017] As used herein, a "neural network" is capable of processing input and providing corresponding output. It typically comprises an input layer, an output layer, and one or more hidden layers between the input and output layers. The layers in a neural network are connected sequentially, so that the output of the previous layer is provided as the input to the next layer. The input layer receives the inputs to the neural network model, while the output layer's output serves as the final output of the neural network model. Each layer of a neural network model includes one or more nodes (also called processing nodes or neurons), each of which processes the input from the layer above. In this paper, the terms "neural network," "model," "network," and "neural network model" are used interchangeably.

[0018] As mentioned above, image scaling is one of the most common operations when processing digital images. However, traditional image downscaling (i.e., reducing a high-resolution image to a low-resolution image) often results in the loss of high-frequency information in the high-resolution image. This loss of high-frequency information makes image upscaling (i.e., enlarging a low-resolution image to a high-resolution image) very challenging, meaning that for the same low-resolution (LR) image, there may be multiple corresponding high-resolution (HR) images (also known as the ill-posedness of the image upscaling process). Therefore, traditional methods often fail to reconstruct high-quality HR images from LR images.

[0019] Traditional approaches typically employ super-resolution (SR) methods to upscale LR images. Current SR methods primarily focus on learning prior information through example-based policies or deep learning models. Clearly, if the target LR image is obtained by pre-scaling a corresponding HR image, considering the image downscaling method during the upscaling process would help improve the quality of the HR image reconstruction. However, current SR methods do not take this into account.

[0020] Traditional image downscaling methods employ frequency-based kernels (such as bilinear interpolation, bicubic interpolation, etc.) as low-pass filters to resample the input high-resolution image to the target resolution. Because high-frequency information is suppressed, these methods often result in over-smoothing of the image. Several image downscaling methods that preserve detail or structural similarity have recently been proposed. However, these perception-oriented image downscaling methods fail to consider the potential mutual reinforcement between image downscaling and its inverse task (i.e., image upscaling).

[0021] Some traditional approaches consider the potential mutual reinforcement between image downscaling and its inverse task (i.e., image upscaling), attempting to model image downscaling and image upscaling as a joint task. For example, some approaches propose image downscaling models based on autoencoder frameworks, where the encoder and decoder serve as the image downscaling and SR models, respectively, allowing the image downscaling and upscaling processes to be jointly trained as a unified task. Other approaches propose using convolutional neural networks to estimate the downsized low-resolution image and leveraging the learned or specified SR model for HR image reconstruction. Still others propose image downscaling methods based on content-adaptive resamplers, which can be trained alongside any existing SR model. While these approaches can improve the quality of HR image recovery from downsized LR images to some extent, they cannot fundamentally address the ill-posedness of the image upscaling process, and therefore cannot reconstruct high-quality HR images from LR images.

[0022] According to an implementation of this disclosure, a scheme for image scaling is proposed. In this scheme, an input image with a first resolution is scaled into an output image with a second resolution using a reversible neural network. Furthermore, the inverse network of this neural network can scale an input image with the second resolution back into an output image with the first resolution. Specifically, during image downscaling, the neural network can convert the high-resolution (HR) image into a low-resolution (LR) image and high-frequency information following a specific distribution. During image upscaling, the inverse network of this neural network can convert the LR image and the high-frequency information following the specific distribution back into an HR image. By using a reversible neural network to model the image downscaling and upscaling processes, this scheme can reduce the original image to a visually pleasing low-resolution image and greatly alleviate the ill-posedness problem of the image upscaling process, thereby enabling the reconstruction of a high-quality high-resolution image from a low-resolution image.

[0023] The following section provides a detailed description of various example implementations of this scheme, with reference to the accompanying drawings.

[0024] Figure 1A A block diagram of a computing device 100 capable of implementing multiple implementations of the present disclosure is shown. It should be understood that... Figure 1A The computing device 100 shown is merely exemplary and should not constitute any limitation on the functionality and scope of the implementation described in this disclosure. Figure 1A As shown, computing device 100 includes computing device 100 in the form of general computing device. Components of computing device 100 may include, but are not limited to, one or more processors or processing units 110, memory 120, storage device 130, one or more communication units 140, one or more input devices 150, and one or more output devices 160.

[0025] In some implementations, computing device 100 can be implemented as various user terminals or service terminals. Service terminals can be servers, large computing devices, etc., provided by various service providers. User terminals can be any type of mobile terminal, fixed terminal, or portable terminal, including mobile phones, sites, units, devices, multimedia computers, multimedia tablets, internet nodes, communicators, desktop computers, laptop computers, notebook computers, netbook computers, tablet computers, personal communication system (PCS) devices, personal navigation devices, personal digital assistants (PDAs), audio / video players, digital cameras / camcorders, positioning devices, television receivers, radio receivers, e-book devices, gaming devices, or any combination thereof, including accessories and peripherals of these devices, or any combination thereof. It is also foreseeable that computing device 100 can support any type of user-facing interface (such as "wearable" circuitry).

[0026] Processing unit 110 can be a physical or virtual processor and is capable of performing various processes according to programs stored in memory 120. In a multiprocessor system, multiple processing units execute computer-executable instructions in parallel to improve the parallel processing capability of computing device 100. Processing unit 110 may also be referred to as a central processing unit (CPU), microprocessor, controller, or microcontroller.

[0027] Computing device 100 typically includes multiple computer storage media. Such media can be any available media accessible to computing device 100, including but not limited to volatile and non-volatile media, removable and non-removable media. Memory 120 can be volatile memory (e.g., registers, cache, random access memory (RAM)), non-volatile memory (e.g., read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory), or some combination thereof.

[0028] Storage device 130 may be a removable or non-removable medium and may include machine-readable media, such as memory, flash drives, disks, or any other media capable of storing information and / or data and accessible within computing device 100. Computing device 100 may further include additional removable / non-removable, volatile / non-volatile storage media. Although not explicitly stated... Figure 1A As shown, disk drives for reading from or writing to removable, non-volatile disks and optical disc drives for reading from or writing to removable, non-volatile optical discs can be provided. In these cases, each drive can be connected to a bus (not shown) via one or more data media interfaces.

[0029] The communication unit 140 enables communication with other computing devices via a communication medium. Additionally, the functionality of the components of the computing device 100 can be implemented as a single computing cluster or multiple computing machines capable of communicating via communication connections. Therefore, the computing device 100 can operate in a networked environment using logical connections to one or more other servers, personal computers (PCs), or another general network node.

[0030] Input device 150 can be one or more various input devices, such as a mouse, keyboard, trackball, voice input device, etc. Output device 160 can be one or more output devices, such as a monitor, speaker, printer, etc. Computing device 100 can also communicate as needed with one or more external devices (not shown) via communication unit 140. These external devices include storage devices, display devices, etc., and can communicate with one or more devices that enable user interaction with computing device 100, or with any device that enables computing device 100 to communicate with one or more other computing devices (e.g., network card, modem, etc.). Such communication can be performed via input / output (I / O) interface (not shown).

[0031] In some implementations, in addition to being integrated into a single device, some or all of the components of computing device 100 may be configured in the form of a cloud computing architecture. In a cloud computing architecture, these components can be remotely deployed and can work together to achieve the functionality described herein. In some implementations, cloud computing provides computing, software, data access, and storage services without requiring end users to know the physical location or configuration of the systems or hardware providing these services. In various implementations, cloud computing provides services over a wide area network (such as the Internet) using appropriate protocols. For example, cloud computing providers offer applications over a wide area network, and these applications can be accessed through a web browser or any other computing component. The software or components of the cloud computing architecture, along with the corresponding data, may be stored on servers at remote locations. Computing resources in a cloud computing environment may be consolidated at remote data center locations or they may be distributed. Cloud computing infrastructure can provide services through shared data centers, even if they appear as a single access point for users. Therefore, the components and functionality described herein can be provided from service providers at remote locations using a cloud computing architecture. Alternatively, they may be provided from conventional servers, or they may be installed directly or otherwise on client devices.

[0032] The computing device 100 can be used to implement image scaling according to various implementations of the present disclosure. The memory 120 may include an image scaling module 122 having one or more program instructions that can be accessed and executed by the processing unit 110 to implement the functions of the various implementations described herein.

[0033] During image scaling, computing device 100 can receive input image 170 via input device 150. In some implementations, input image 170 may be, for example, an image with a first resolution. Input image 170 can be input to image scaling module 122 in memory 120. Image scaling module 122 can utilize a trained reversible neural network to generate output image 180 with a second resolution and high-frequency information following a specific distribution based on input image 170, wherein the first resolution is higher than the second resolution, and the input image 170 and output image 180 have the same semantics. In other implementations, input image 170 may be, for example, an image with a second resolution. Input image 170 can be input to image scaling module 122 in memory 120. Image scaling module 122 can utilize the inverse network of the reversible neural network to generate output image 180 with a first resolution based on input image 170 and high-frequency information following a specific distribution, wherein the first resolution is higher than the second resolution, and the input image 170 and output image 180 have the same semantics. Output image 180 can be output via output device 160.

[0034] In some implementations, the image scaling module 122 can use a trained reversible neural network to downscale the image (i.e., convert the HR image to an LR image), and the image scaling module 122 can use the inverse network of the reversible neural network to upscale the image (i.e., reconstruct the LR image into an HR image). Figure 1B A schematic diagram illustrating the operation of the image scaling module 122 implemented according to this disclosure is shown. Figure 1B As shown, the image scaling module 122 can utilize a reversible neural network 191 (represented as "f"). θ The image scaling module 122 generates a low-resolution output image 180 and high-frequency information 185 following a predetermined distribution based on a high-resolution input image 170. For example, the high-frequency information 185 may be manifested as high-frequency noise that is semantically independent of the input image 170. The image scaling module 122 may utilize the inverse network 192 of the invertible neural network 191 (represented as...). The system generates a high-resolution output image 180 based on a low-resolution input image 170 and high-frequency information 175 following a predetermined distribution. The "predetermined distribution" may include, but is not limited to, a Gaussian distribution, a uniform distribution, etc., and can be specified during the training of the invertible neural network.

[0035] Invertible neural networks (INNs) are a popular network architecture in generative models that allows specifying the mapping relationship m = f. θ (n) and its inverse mapping relationship An INN typically consists of at least one reversible block. For the l-th block, the input h l Divided along the channel axis into and And it undergoes an affine transformation:

[0036]

[0037]

[0038] The corresponding output is Given an output, its inverse transform can be calculated as follows:

[0039]

[0040]

[0041] in ρ and η can be any functions, and ⊙ represents the XOR operation.

[0042] When applying an INN to an image scaling task, based on a high-resolution input image x, the INN can output not only a scaled-down low-resolution image y, but also high-frequency information z following a specific distribution, which may manifest as high-frequency noise independent of the image's semantics. This allows the inverse network of the INN to reconstruct a high-quality high-resolution image x from the low-resolution image y and the noise z. That is, the high-frequency information z lost during image scaling needs to be maintained to make the image scaling process reversible, and the entire image scaling process can be achieved using the mapping relationship (y, z) = f θ (x) and To express.

[0043] However, in image upscaling, it is often necessary to upscale any LR image. Therefore, the high-frequency information z corresponding to the input LR image is usually absent. The inventors noted that, according to the Nyquist-Shannon sampling theorem, the information lost during image downscaling corresponds to high-frequency details. Assuming a set of HR images corresponding to the same LR image contains different high-frequency details, these details can typically exhibit a certain degree of variability and randomness. Therefore, z can be represented as a random variable, whose distribution is denoted by INN (i.e., f). θ The output z is determined by the distribution p(z). Specifically, an INN can be trained to satisfy a specified distribution p(z). In this way, the high-frequency noise z output by the invertible neural network does not need to be preserved during image downscaling. Furthermore, during image upscaling, a high-resolution image can be reconstructed based on the low-resolution image and any sample under this specified distribution.

[0044] Figure 2A A schematic block diagram of a reversible neural network 191 implemented according to this disclosure is shown. It should be understood that, as Figure 2A The structure of the reversible neural network 191 shown is merely exemplary and is not intended to limit the scope of this disclosure. Implementations of this disclosure are also applicable to reversible neural networks with different structures.

[0045] like Figure 2A As shown, the reversible neural network 191 can be composed of one or more downsampling modules 210 connected in series. For simplification purposes, in Figure 2A The diagram shows a downsampling module 210. The image downsampling ratio supported by the reversible neural network 191 can be determined by the image downsampling ratio supported by each downsampling module 210 and the number of downsampling modules 210 included. For example, assuming each downsampling module 210 supports downsampling the image by a factor of 2 and the reversible neural network 191 includes 2 downsampling modules 210, then the reversible neural network 191 supports downsampling the image by a factor of 4.

[0046] like Figure 2A As shown, for example, the downsampling module 210 may include a transformation module 230 and one or more INN units 220-1, 220-2...220-M (collectively or individually referred to as "INN unit 220", where M≥1).

[0047] Transform module 230 can decompose a high-resolution input image 170 into low-frequency components 242 and high-frequency components 241, where the low-frequency component 242 represents the semantics of the input image 170, and the high-frequency component 241 is related to that semantics. In some implementations, transform module 230 can be implemented as a wavelet transform module, such as a Haar transform module. For example, when transform module 230 is implemented as a Haar transform module, downsampling module 210 can support downsampling the image by a factor of 2. Specifically, the Haar transform module can convert an input image or a set of feature maps with length H, width W, and number of channels C into an output tensor. The first C-slice in the output tensor can be approximated as a low-pass representation equivalent to bilinear interpolation downsampling. The remaining three sets of C-slices contain residual components in the vertical, horizontal, and diagonal directions, respectively. These residual components are based on high-frequency information in the original HR image. Alternatively, the transform module 230 can also be implemented using a 1×1 reversible convolution block, or implemented as any known or future-developed transform module capable of decomposing the input image 170 into low-frequency and high-frequency components. It should be understood that the implementation of the transform module 230 can differ depending on the image downsampling ratio supported by the downsampling module 210. In this way, low-frequency information 242 and high-frequency information 241 can be fed into the subsequent INN unit 220-1.

[0048] As described above, the structure of each INN unit 220 should be reversible, thereby ensuring the reversibility of the network structure of the neural network 191. The INN unit 220 is used to extract corresponding features from the low-frequency and high-frequency components of the input, and to convert the high-frequency components related to image semantics into high-frequency information that follows a predetermined distribution and is independent of image semantics.

[0049] Figure 2B A schematic diagram of an example INN unit 220 according to an implementation of this disclosure is shown. Here, it is assumed that the low-frequency and high-frequency components input to the INN unit 220 are represented as follows: and like Figure 2B As shown, it can be directed to low-frequency components. Apply the affine transformation shown in formula (1) above, and to the high-frequency components Apply the affine transformation as shown in formula (2) above. Figure 2B The transformation function shown η and ρ can be arbitrary functions. It should be understood that, for example... Figure 2B The INN unit 220 shown is for illustrative purposes only and is not intended to limit the scope of this disclosure. Implementations of this disclosure are also applicable to INN units with other different structures. Examples of INN units include, but are not limited to, reversible convolutional blocks, reversible residual network units, reversible generative network units, deep reversible network units, and so on.

[0050] Figure 3A It shows Figure 2A A schematic block diagram of the inverse network 192 of the reversible neural network 191 shown. (See diagram below.) Figure 3A As shown, network 192 can be composed of one or more upsampling modules 310 connected in series. For simplicity, in Figure 3A An upsampling module 310 is shown. The image magnification ratio supported by the inverse network 192 can be determined by the image magnification ratio supported by each upsampling module 310 and the number of upsampling modules 310 included. For example, assuming that each upsampling module 310 supports magnifying the image by 2 times and the inverse network 192 includes 2 upsampling modules 310, then the inverse network 192 supports magnifying the image by 4 times.

[0051] like Figure 3A As shown, for example, the upsampling module 310 may include a transformation module 330 and one or more INN units 320-1, 320-2...320-M (collectively or individually referred to as "INN unit 320", where M≥1). The structure of the INN unit 320 is similar to that shown below. Figure 2B The structure of the INN unit 220 shown is inverse, for example, as Figure 3BAs shown. Taking INN unit 320-M as an example, here we assume that the low-resolution input image 170 input to INN unit 320-M is represented as... And the high-frequency information 175, which follows a predetermined distribution, is represented as like Figure 3B As shown, it can be directed to Applying the inverse transformation of the affine transformation as shown in formula (3) above, and towards Apply the inverse transformation of the affine transformation as shown in formula (4) above. Figure 3B The transformation function shown η and ρ can be arbitrary functions. It should be understood that, for example... Figure 3B The INN unit 320 shown is for illustrative purposes only and is not intended to limit the scope of this disclosure. Implementations of this disclosure are also applicable to INN units with other different structures. Examples of INN units include, but are not limited to, reversible convolutional blocks, reversible residual network units, reversible generative network units, deep reversible network units, and so on.

[0052] like Figure 3A As shown, one or more INN units 320 can convert a low-resolution input image 170 and high-frequency information 175 following a predetermined distribution into high-frequency components 341 and low-frequency components 342 to be merged. (This is similar to...) Figure 2A Conversely to the transform module 230 shown, transform module 330 can combine high-frequency components 341 and low-frequency components 342 into an output image 180 with high resolution. In some implementations, when transform module 230 is implemented as a wavelet transform module, transform module 330 can be implemented as an inverse wavelet transform module. For example, when transform module 230 is implemented as a Haar transform module, transform module 330 can be implemented as an inverse Haar transform module. Alternatively, transform module 330 can also be implemented using a 1×1 reversible convolution block, or implemented as any known or future-developed transform module capable of combining low-frequency and high-frequency components into an image.

[0053] The training process of the invertible neural network will be described in further detail below. For simplicity, the neural network to be trained and its inverse network will be collectively referred to as the "model". As can be seen from the above description, the training objective of the model is to determine the mapping relationship f between the high-resolution image x, the low-resolution image y, and the specified distribution p(z). θ .

[0054] To achieve this training objective, some implementations can acquire a set of high-resolution images. A set of low-resolution images (also referred to as the "first set of training images," where N represents the number of images) and a set of semantically corresponding low-resolution images (also referred to as the "second set of training images") are used as training data to train the model. In some implementations, the second set of training images with low resolution can be generated based on the first set of training images with high resolution. For example, interpolation methods or any known or to be developed methods can be used to generate semantically corresponding low-resolution training images from high-resolution training images. The scope of this disclosure is not limited herein. In some implementations, an objective function for training the model can be generated based on the first set of training images and the second set of training images. The parameters of the model are then determined by minimizing the objective function.

[0055] In some implementations, the objective function used to train the model can be determined based on the difference between low-resolution training images and low-resolution images generated by the model based on high-resolution training images. For example, for a high-resolution training image x in the first set of training images... (n) Assuming the model is trained on this high-resolution image x (n) The generated low-resolution image is represented as Furthermore, in the second set of training images, the high-resolution training image x... (n) The corresponding low-resolution training image is represented as Training images can be based on low resolution. Low-resolution images generated by the model The difference between the values ​​is used to generate the objective function (also known as the "first objective function" or "low-resolution guided loss function") for training the invertible neural network. For example, the first objective function can be expressed as:

[0056]

[0057] in This represents a difference measurement function, such as the L1 loss function or the L2 loss function.

[0058] Additionally or alternatively, in some implementations, the objective function used to train the model can be determined based on the difference between high-resolution training images and high-resolution images reconstructed by the model from low-resolution images. For example, for a high-resolution training image x in the first set of training images... (n) Assuming the model is trained on this high-resolution image x (n) The generated low-resolution image is represented as The model is based on low-resolution images The resulting high-resolution reconstructed image is represented as Where z follows a predetermined distribution p(z) (i.e., z ~ p(z)), it can be based on a high-resolution training image x(n) With high-resolution reconstructed images The difference between z) is used to generate the objective function (also known as the "second objective function" or "high-resolution reconstruction loss function") for training the invertible neural network. For example, the second objective function can be expressed as:

[0059]

[0060] in Measuring the difference between the high-resolution original image and the reconstructed image. This indicates that when z follows a predetermined distribution p(z) Data expectations.

[0061] Additionally or alternatively, another goal of model training is to encourage the model to capture the data distribution of high-resolution training images. Here, it is assumed that the first set of training data... The data distribution on the dataset is represented as q(x). For example, for the high-resolution training image x in the first set of training images... (n) The high-resolution image reconstructed by the model is represented as in The model represents the high-resolution training image x. (n) The low-resolution image obtained after scaling down, and Let represent a random variable that follows a predetermined distribution p(z). This is achieved by iterating through the first set of training data. A set of scaled-down, low-resolution images can be obtained. The data distribution on can be represented as It represents the transformed random variable. The data distribution is given by the original random variable x, which follows a data distribution q(x), i.e., x ~ q(x). Similarly, the high-resolution image reconstructed by the model can be represented as... The data distribution on it can be represented as because In some implementations, the data distribution can be based on the original data distribution q(x) and the data distribution reconstructed from the model. The difference between the values ​​is used to generate the objective function (also known as the "third objective function" or "distribution matching loss function") for training the invertible neural network. For example, the third objective function can be expressed as:

[0062]

[0063] in Used to measure the difference between two data distributions.

[0064] In some cases, directly minimizing the third objective function as shown in Equation (6) can be difficult because both distributions are high-dimensional and may have unknown density functions. In some implementations, JS divergence can be used to measure the difference between the two data distributions. That is, the third objective function can also be expressed as:

[0065]

[0066] In some implementations, the overall objective function for training the model can be generated based on a combination of the first, second, and third objective functions. For example, the overall objective function can be expressed as:

[0067] L total :=λ1L recon +λ2L guide +λ3L distr (8)

[0068] Among them, λ1, λ2 and λ3 are coefficients used to balance different loss terms.

[0069] In some implementations, to improve the stability of model training, a pre-training process can be performed before training the model using the overall objective function as shown in Equation (8). During pre-training, a weakened but more stable distribution matching loss function can be used. For example, this distribution matching loss function can be constructed based on the cross-entropy loss function to improve the stability of model training. For instance, the distribution matching loss function constructed based on the cross-entropy (CE) loss function (also known as the "fourth objective function") can be expressed as:

[0070]

[0071] Where CE represents the cross-entropy loss function. Correspondingly, the overall objective function used during pre-training can be expressed as:

[0072] L IRN :=λ1L recon +λ2L guide +λ3L′ distr (10)

[0073] Among them, λ1, λ2 and λ3 are coefficients used to balance different loss terms.

[0074] In some implementations, after pre-training, the overall objective function shown in Equation (8) can be reused to perform a second round of training on the model. Alternatively, in some implementations, after pre-training, the overall objective function shown in Equation (11) can be used to perform a second round of training on the model:

[0075] L IRN+:=λ1L recon +λ2L guide +λ3L distr +λ4L percp (11)

[0076] Where the perceptual loss function L percp This is used to measure the difference in semantic features between the original high-resolution image and the reconstructed high-resolution image. For example, the semantic features of both the original and reconstructed high-resolution images can be extracted by other known benchmark models, which will not be elaborated here. λ1, λ2, λ3, and λ4 are coefficients used to balance different loss terms.

[0077] Figure 4 A flowchart of a method 400 for image scaling according to some implementations of this disclosure is shown. Method 400 can be implemented by a computing device 100, for example, it can be implemented at an image scaling module 122 in the memory 120 of the computing device 100. At block 410, the computing device 100 acquires an input image having a first resolution. At block 420, the computing device 100 uses a trained reversible neural network to generate an output image having a second resolution and high-frequency information following a predetermined distribution based on the input image, wherein the first resolution is higher than the second resolution and the input image and the output image have the same semantics.

[0078] In some implementations, the reversible neural network includes a transformation module and at least one reversible network unit, and generating the output image and the high-frequency information based on the input image includes: using the transformation module to decompose the input image into low-frequency components representing the semantics of the input image and high-frequency components related to the semantics; and using the at least one reversible network unit to generate the output image and the high-frequency information unrelated to the semantics based on the low-frequency components and the high-frequency components.

[0079] In some implementations, the transform module includes any one of the following: a wavelet transform module; and a reversible convolution block.

[0080] In some implementations, method 400 further includes: training the reversible neural network such that the trained reversible neural network can generate a second image with a second resolution and first high-frequency information following the predetermined distribution based on a first image with the first resolution, and the inverse network of the trained reversible neural network can generate a fourth image with the first resolution based on a third image with the second resolution and second high-frequency information following the predetermined distribution.

[0081] In some implementations, training the reversible neural network includes: acquiring a first set of training images having the first resolution; acquiring a second set of training images that semantically correspond to the first set of training images and have the second resolution; and training the reversible neural network based on the first set of training images and the second set of training images.

[0082] In some implementations, obtaining the second set of training images includes generating the second set of training images based on the first set of training images using an interpolation method.

[0083] In some implementations, training the reversible neural network includes: determining a plurality of objective functions based on the first set of training images and the second set of training images; determining a total objective function for training the reversible neural network by combining at least a portion of the plurality of objective functions; and determining the network parameters of the reversible neural network by minimizing the total objective function.

[0084] In some implementations, determining the plurality of objective functions includes: using the reversible neural network to generate a third set of training images with the second resolution and a set of random variables based on the first set of training images; and determining a first objective function based on the difference between the second set of training images and the third set of training images.

[0085] In some implementations, determining the plurality of objective functions includes: using the reversible neural network to generate a third set of training images with the second resolution and a set of random variables based on the first set of training images; using the inverse network to generate a fourth set of training images with the first resolution based on the third set of training images and high-frequency information following the predetermined distribution; and determining a second objective function based on the difference between the first set of training images and the fourth set of training images.

[0086] In some implementations, determining the plurality of objective functions includes: determining a first data distribution of the first set of training images; determining a second data distribution of the fourth set of training images; and determining a third objective function based on the difference between the first data distribution and the second data distribution.

[0087] In some implementations, determining the plurality of objective functions includes: determining a third data distribution of the set of random variables; and determining a fourth objective function based on the difference between the third data distribution and the predetermined distribution.

[0088] Figure 5A flowchart of a method 500 for image scaling according to some implementations of this disclosure is shown. Method 500 can be implemented by a computing device 100, for example, it can be implemented at an image scaling module 122 in the memory 120 of the computing device 100. At block 510, the computing device 100 acquires an input image having a second resolution. At block 520, the computing device 100 uses a trained invertible neural network to generate an output image having a first resolution based on the input image and high-frequency information following a predetermined distribution, wherein the first resolution is higher than the second resolution and the input image and the output image have the same semantics.

[0089] In some implementations, the reversible neural network includes a transformation module and at least one reversible network unit, and generating the output image based on the input image and the high-frequency information includes: using the at least one reversible network unit to generate low-frequency components and high-frequency components to be merged based on the input image and the high-frequency information, wherein the low-frequency components represent the semantics of the input image and the high-frequency components are related to the semantics; and using the transformation module to merge the low-frequency components and the high-frequency components into the output image.

[0090] In some implementations, the transform module includes any one of the following: an inverse wavelet transform module; and a reversible convolution block.

[0091] As described above, the present disclosure proposes a scheme for image scaling. During image downscaling, a reversible neural network can transform a high-resolution (HR) image into a low-resolution (LR) image and high-frequency noise following a specific distribution. During image upscaling, the inverse network of this reversible neural network can transform the LR image and random noise following the specific distribution into an HR image. By utilizing a reversible neural network to model the image downscaling and upscaling processes, this scheme can reduce the original image to a visually pleasing low-resolution image and reconstruct a high-quality high-resolution image from the low-resolution image, thereby greatly alleviating the ill-posedness problem in the image upscaling process. Furthermore, experimental data show that, compared with traditional image scaling schemes, the implementation of this disclosure achieves better image reconstruction performance metrics, such as higher peak signal-to-noise ratio (PSNR) and / or structural similarity (SSIM).

[0092] The implementation of this disclosure can be widely applied in the field of image and / or video processing. For example, online video streaming plays a vital role in our lives, such as video websites, live streaming websites, and video streaming mobile applications. High-quality online video streaming is desirable, such as high-resolution video with rich perceptual detail. High-resolution video often consumes a significant amount of network bandwidth for transmission. Therefore, to save network bandwidth, high-resolution video is typically processed and compressed when sent to the user client. This often results in low-quality, low-resolution video being presented to the user client. By applying the image scaling scheme implemented according to this disclosure, the above problems can be solved.

[0093] Figure 6 A block diagram of an example system 600 capable of implementing the present disclosure is shown. Figure 6 As shown, system 600 may include a video streaming service provider 610, a server 620, and a client device 630. For example, the video streaming service provider 610 may provide the server 620 with the video data requested by the client device 630, and the server 620 may send the video data from the video streaming service provider 610 to the client device 630 via the network.

[0094] like Figure 6 As shown, in some implementations, the video streaming service provider 610 can provide a high-resolution video stream 601, also known as a high-resolution image sequence 601, to the server 620. The server 620 can use the reversible neural network 191 described above to convert the high-resolution image sequence 601 into a low-resolution image sequence. In some implementations, the server 620 can directly send the obtained low-resolution image sequence as a low-resolution video stream 602 to the client device 430. In this case, the client device 630 can receive the low-resolution image sequence. Additionally or alternatively, in some implementations, the server 620 can perform video encoding on the low-resolution image sequence to generate an encoded low-resolution video stream 602, and send the encoded low-resolution video 602 to the client device 430 via the network. In this case, the client device 630 can decode the received encoded low-resolution video 602 to obtain a decoded low-resolution image sequence. Then, the client device 630 can use the inverse network 192 of the reversible neural network 191 described above to reconstruct the obtained low-resolution image sequence into a high-resolution video stream 603. In this way, customers can obtain high-quality video streams while saving network bandwidth.

[0095] Beyond the field of image and / or video processing, the implementation of this disclosure can also be applied to the field of image and / or video storage. For example, when storing high-resolution images and / or videos in a storage device, the reversible neural network 191 described above can be used to convert the high-resolution images and / or videos into low-resolution images and / or videos along with corresponding high-frequency information following a predetermined distribution. The resulting low-resolution images and / or videos can then be stored in the storage device, discarding the obtained high-frequency information. When accessing images and / or videos stored in the storage device, the low-resolution images and / or videos can first be retrieved from the storage device, and then the high-resolution images and / or videos can be reconstructed using the inverse network 192 of the reversible neural network 191 described above, based on the retrieved images and / or videos and random noise following the predetermined distribution. In this way, storage space for storing images and / or videos can be saved without sacrificing image and / or video quality.

[0096] The following are some example implementations of this disclosure.

[0097] In a first aspect, this disclosure provides a computer-implemented method. The method includes: acquiring an input image having a first resolution; and generating, based on the input image, an output image having a second resolution and high-frequency information following a predetermined distribution using a trained reversible neural network, wherein the first resolution is higher than the second resolution and the input image and the output image have the same semantics.

[0098] In some embodiments, the method further includes storing the output image without storing the high-frequency information.

[0099] In some embodiments, the method further includes: encoding an output image; and providing the encoded output image.

[0100] In some implementations, the reversible neural network includes a transformation module and at least one reversible network unit, and generating the output image and the high-frequency information based on the input image includes: using the transformation module to decompose the input image into low-frequency components representing the semantics of the input image and high-frequency components related to the semantics; and using the at least one reversible network unit to generate the output image and the high-frequency information unrelated to the semantics based on the low-frequency components and the high-frequency components.

[0101] In some implementations, the transform module includes any one of the following: a wavelet transform module; and a reversible convolution block.

[0102] In some implementations, the method further includes: training the reversible neural network such that the trained reversible neural network can generate a second image with a second resolution and first high-frequency information following the predetermined distribution based on a first image with the first resolution, and the inverse network of the trained reversible neural network can generate a fourth image with the first resolution based on a third image with the second resolution and second high-frequency information following the predetermined distribution.

[0103] In some implementations, training the reversible neural network includes: acquiring a first set of training images having the first resolution; acquiring a second set of training images that semantically correspond to the first set of training images and have the second resolution; and training the reversible neural network based on the first set of training images and the second set of training images.

[0104] In some implementations, obtaining the second set of training images includes generating the second set of training images based on the first set of training images using an interpolation method.

[0105] In some implementations, training the reversible neural network includes: determining a plurality of objective functions based on the first set of training images and the second set of training images; determining a total objective function for training the reversible neural network by combining at least a portion of the plurality of objective functions; and determining the network parameters of the reversible neural network by minimizing the total objective function.

[0106] In some implementations, determining the plurality of objective functions includes: using the reversible neural network to generate a third set of training images with the second resolution and a set of random variables based on the first set of training images; and determining a first objective function based on the difference between the second set of training images and the third set of training images.

[0107] In some implementations, determining the plurality of objective functions includes: using the reversible neural network to generate a third set of training images with the second resolution and a set of random variables based on the first set of training images; using the inverse network to generate a fourth set of training images with the first resolution based on the third set of training images and high-frequency information following the predetermined distribution; and determining a second objective function based on the difference between the first set of training images and the fourth set of training images.

[0108] In some implementations, determining the plurality of objective functions includes: determining a first data distribution of the first set of training images; determining a second data distribution of the fourth set of training images; and determining a third objective function based on the difference between the first data distribution and the second data distribution.

[0109] In some implementations, determining the plurality of objective functions includes: determining a third data distribution of the set of random variables; and determining a fourth objective function based on the difference between the third data distribution and the predetermined distribution.

[0110] In a second aspect, this disclosure provides a computer-implemented method. The method includes: acquiring an input image having a second resolution; and generating an output image having a first resolution based on the input image and high-frequency information following a predetermined distribution using a trained invertible neural network, wherein the first resolution is higher than the second resolution and the input image and the output image have the same semantics.

[0111] In some implementations, acquiring the input image includes: acquiring the encoded input image; and decoding the encoded input image.

[0112] In some implementations, the reversible neural network includes a transformation module and at least one reversible network unit, and generating the output image based on the input image and the high-frequency information includes: using the at least one reversible network unit to generate low-frequency components and high-frequency components to be merged based on the input image and the high-frequency information, wherein the low-frequency components represent the semantics of the input image and the high-frequency components are related to the semantics; and using the transformation module to merge the low-frequency components and the high-frequency components into the output image.

[0113] In some implementations, the transform module includes any one of the following: an inverse wavelet transform module; and a reversible convolution block.

[0114] In a third aspect, this disclosure provides an electronic device. The electronic device includes: a processing unit; and a memory coupled to the processing unit and containing instructions stored thereon, the instructions, when executed by the processing unit, causing the device to perform actions, the actions including: acquiring an input image having a first resolution; and generating an output image having a second resolution and high-frequency information following a predetermined distribution based on the input image using a trained reversible neural network, wherein the first resolution is higher than the second resolution and the input image and the output image have the same semantics.

[0115] In some embodiments, the action further includes storing the output image without storing the high-frequency information.

[0116] In some embodiments, the action further includes: encoding the output image; and providing the encoded output image.

[0117] In some implementations, the reversible neural network includes a transformation module and at least one reversible network unit, and generating the output image and the high-frequency information based on the input image includes: using the transformation module to decompose the input image into low-frequency components representing the semantics of the input image and high-frequency components related to the semantics; and using the at least one reversible network unit to generate the output image and the high-frequency information unrelated to the semantics based on the low-frequency components and the high-frequency components.

[0118] In some implementations, the transform module includes any one of the following: a wavelet transform module; and a reversible convolution block.

[0119] In some implementations, the action further includes: training the reversible neural network such that the trained reversible neural network can generate a second image with a second resolution and first high-frequency information following the predetermined distribution based on a first image with the first resolution, and the inverse network of the trained reversible neural network can generate a fourth image with the first resolution based on a third image with the second resolution and second high-frequency information following the predetermined distribution.

[0120] In some implementations, training the reversible neural network includes: acquiring a first set of training images having the first resolution; acquiring a second set of training images that semantically correspond to the first set of training images and have the second resolution; and training the reversible neural network based on the first set of training images and the second set of training images.

[0121] In some implementations, obtaining the second set of training images includes generating the second set of training images based on the first set of training images using an interpolation method.

[0122] In some implementations, training the reversible neural network includes: determining a plurality of objective functions based on the first set of training images and the second set of training images; determining a total objective function for training the reversible neural network by combining at least a portion of the plurality of objective functions; and determining the network parameters of the reversible neural network by minimizing the total objective function.

[0123] In some implementations, determining the plurality of objective functions includes: using the reversible neural network to generate a third set of training images with the second resolution and a set of random variables based on the first set of training images; and determining a first objective function based on the difference between the second set of training images and the third set of training images.

[0124] In some implementations, determining the plurality of objective functions includes: using the reversible neural network to generate a third set of training images with the second resolution and a set of random variables based on the first set of training images; using the inverse network to generate a fourth set of training images with the first resolution based on the third set of training images and high-frequency information following the predetermined distribution; and determining a second objective function based on the difference between the first set of training images and the fourth set of training images.

[0125] In some implementations, determining the plurality of objective functions includes: determining a first data distribution of the first set of training images; determining a second data distribution of the fourth set of training images; and determining a third objective function based on the difference between the first data distribution and the second data distribution.

[0126] In some implementations, determining the plurality of objective functions includes: determining a third data distribution of the set of random variables; and determining a fourth objective function based on the difference between the third data distribution and the predetermined distribution.

[0127] In a fourth aspect, this disclosure provides an electronic device. The electronic device includes: a processing unit; and a memory coupled to the processing unit and containing instructions stored thereon, the instructions, when executed by the processing unit, causing the device to perform actions, the actions including: acquiring an input image having a second resolution; and generating an output image having a first resolution based on the input image and high-frequency information following a predetermined distribution using a trained reversible neural network, wherein the first resolution is higher than the second resolution and the input image and the output image have the same semantics.

[0128] In some implementations, acquiring the input image includes: acquiring the encoded input image; and decoding the encoded input image.

[0129] In some implementations, the reversible neural network includes a transformation module and at least one reversible network unit, and generating the output image based on the input image and the high-frequency information includes: using the at least one reversible network unit to generate low-frequency components and high-frequency components to be merged based on the input image and the high-frequency information, wherein the low-frequency components represent the semantics of the input image and the high-frequency components are related to the semantics; and using the transformation module to merge the low-frequency components and the high-frequency components into the output image.

[0130] In some implementations, the transform module includes any one of the following: an inverse wavelet transform module; and a reversible convolution block.

[0131] In a fifth aspect, this disclosure provides a computer program product tangibly stored in a non-transient computer storage medium and including machine-executable instructions that, when executed by a device, cause the device to perform the method described in accordance with the first or second aspect above.

[0132] In another aspect, this disclosure provides a computer-readable medium having stored thereon machine-executable instructions that, when executed by a device, cause the device to perform the method described in accordance with the first or second aspect above.

[0133] The functions described above in this document can be performed at least in part by one or more hardware logic components. For example, exemplary types of hardware logic components that can be used, without limitation, include: field programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), payload programmable logic devices (CPLDs), and so on.

[0134] The program code used to implement the methods of this disclosure may be written in any combination of one or more programming languages. This program code may be provided to a processor or controller of a general-purpose computer, special-purpose computer, or other programmable data processing apparatus, such that when executed by the processor or controller, the program code causes the functions / operations specified in the flowcharts and / or block diagrams to be implemented. The program code may be executed entirely on a machine, partially on a machine, as a standalone software package partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0135] In the context of this disclosure, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0136] Furthermore, although the operations are described in a specific order, this should be understood as requiring that such operations be performed in the specific order shown or in sequential order, or requiring that all illustrated operations be performed to achieve the desired result. In certain environments, multitasking and parallel processing may be advantageous. Similarly, although several specific implementation details are included in the above discussion, these should not be construed as limiting the scope of this disclosure. Certain features described in the context of a single implementation may also be implemented in combination in a single implementation. Conversely, various features described in the context of a single implementation may also be implemented individually or in any suitable sub-combination in multiple implementations.

[0137] Although the subject matter has been described using language specific to structural features and / or methodological logic, it should be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or actions described above. Rather, the specific features and actions described above are merely illustrative examples of implementing the claims.

Claims

1. A computer-implemented method comprising: obtaining an input image having a first resolution; and generating, with a trained invertible neural network, an output image having a second resolution and high-frequency information subject to a predetermined distribution based on the input image, wherein the first resolution is higher than the second resolution and the input image and the output image have a same semantics, wherein training the invertible neural network comprises: obtaining a first set of training images having the first resolution; obtaining a second set of training images corresponding to the semantics of the first set of training images respectively and having the second resolution; and training the invertible neural network based on the first set of training images and the second set of training images, wherein training the invertible neural network further comprises: determining a plurality of objective functions based on the first set of training images and the second set of training images; determining a total objective function for training the invertible neural network by combining at least a portion of the plurality of objective functions; and determining network parameters of the invertible neural network by minimizing the total objective function.

2. The method of claim 1, wherein the invertible neural network comprises a transform module and at least one invertible network unit, and generating the output image and the high-frequency information based on the input image comprises: decomposing, with the transform module, the input image into a low-frequency component representing the semantics of the input image and a high-frequency component related to the semantics; and generating, with the at least one invertible network unit, the output image and the high-frequency information independent of the semantics based on the low-frequency component and the high-frequency component.

3. The method of claim 2, wherein the transform module comprises either of: a wavelet transform module; and an invertible convolution block.

4. The method of claim 1, wherein: the trained invertible neural network is capable of generating a second image having the second resolution and first high-frequency information subject to the predetermined distribution based on a first image of the first resolution, and an inverse network of the trained invertible neural network is capable of generating a fourth image having the first resolution based on a third image of the second resolution and second high-frequency information subject to the predetermined distribution.

5. The method of claim 1, wherein obtaining the second set of training images comprises: generating the second set of training images based on the first set of training images with an interpolation method.

6. The method of claim 1, wherein determining the plurality of objective functions comprises: generating, with the invertible neural network, a third set of training images having the second resolution and a set of random variables based on the first set of training images; and determining a first objective function based on a difference between the second set of training images and the third set of training images.

7. The method of claim 1, wherein determining the plurality of objective functions comprises: generating, with the invertible neural network, a third set of training images having the second resolution and a set of random variables based on the first set of training images; ​ generating, based on the third set of training images and high frequency information subject to the predetermined distribution, a fourth set of training images having the first resolution; and determining a second objective function based on a difference between the first set of training images and the fourth set of training images.

8. The method of claim 7, wherein determining the plurality of objective functions comprises: determining a first data distribution of the first set of training images; determining a second data distribution of the fourth set of training images; and and determining a third objective function based on a difference between the first data distribution and the second data distribution.

9. The method of claim 8, wherein determining the plurality of objective functions comprises: determining a third data distribution of the set of random variables; and and determining a fourth objective function based on a difference between the third data distribution and the predetermined distribution.

10. A computer-implemented method, comprising: obtaining an input image having a second resolution; and generating, based on the input image and high frequency information subject to a predetermined distribution, an output image having a first resolution using a trained invertible neural network, wherein the first resolution is higher than the second resolution and the input image and the output image have the same semantics, wherein training the invertible neural network comprises: obtaining a first set of training images having the first resolution; obtaining a second set of training images corresponding to the semantics of the first set of training images respectively and having the second resolution; and training the invertible neural network based on the first set of training images and the second set of training images, wherein training the invertible neural network further comprises: determining a plurality of objective functions based on the first set of training images and the second set of training images; determining a total objective function for training the invertible neural network by combining at least part of the plurality of objective functions; and determining network parameters of the invertible neural network by minimizing the total objective function.

11. The method of claim 10, wherein the invertible neural network comprises a transformation module and at least one invertible network unit, and generating the output image based on the input image and the high frequency information comprises: generating, based on the input image and the high frequency information, a low frequency component and a high frequency component to be merged using the at least one invertible network unit, the low frequency component representing semantics of the input image and the high frequency component being related to the semantics; and merging the low frequency component and the high frequency component into the output image using the transformation module.

12. The method of claim 11, wherein the transformation module comprises any of: a wavelet inverse transform module; and an invertible convolution block.

13. An electronic device, comprising: a processing unit; and a memory coupled to the processing unit and containing instructions stored thereon that, when executed by the processing unit, cause the device to perform acts comprising: obtaining an input image having a first resolution; and ​ generating, based on the input image, an output image having a second resolution and high-frequency information subject to a predetermined distribution using the trained invertible neural network, wherein the first resolution is higher than the second resolution and the input image and the output image have a same semantics, wherein training the invertible neural network comprises: obtaining a first set of training images having the first resolution; obtaining a second set of training images corresponding to the semantics of the first set of training images respectively and having the second resolution; and training the invertible neural network based on the first set of training images and the second set of training images, wherein training the invertible neural network further comprises: determining a plurality of objective functions based on the first set of training images and the second set of training images; determining a total objective function for training the invertible neural network by combining at least part of the plurality of objective functions; and determining network parameters of the invertible neural network by minimizing the total objective function.

14. The device of claim 13, wherein the invertible neural network comprises a transform module and at least one invertible network unit, and generating the output image and the high-frequency information based on the input image comprises: decomposing, using the transform module, the input image into a low-frequency component representing semantics of the input image and a high-frequency component related to the semantics; and generating, using the at least one invertible network unit, the output image and the high-frequency information independent of the semantics based on the low-frequency component and the high-frequency component.

15. The device of claim 14, wherein the transform module comprises any of: a wavelet transform module; and an invertible convolution block.

16. The device of claim 13, wherein: the trained invertible neural network is capable of generating a second image having the second resolution and first high-frequency information subject to the predetermined distribution based on a first image having the first resolution, and an inverse network of the trained invertible neural network is capable of generating a fourth image having the first resolution based on a third image having the second resolution and second high-frequency information subject to the predetermined distribution.

Images