Methods, apparatus, devices, storage media, and program products for generating sample images

Abstract

Embodiments of this disclosure provide methods, apparatus, devices, storage media, and program products for generating sample images, relating to the field of artificial intelligence, and more specifically to the field of computer vision. In the method for generating sample images according to this application, the sample generation device first acquires a first image and a second image corresponding to the first image, wherein the first image and the second image have the same resolution, and the second image has a higher image quality than the first image. Subsequently, the sample generation device determines the spectral difference between a first frequency domain representation of the first image and a second frequency domain representation of the second image, and updates the first frequency domain representation based on the spectral difference using at least a portion of the second frequency domain representation. The sample generation device further generates a sample image based on the updated first frequency domain representation. Thus, embodiments of this disclosure can fully consider the differences between original samples, thereby improving the effectiveness of sample images for model training.

Description

Technical Field

[0001] The embodiments of this disclosure primarily relate to the field of artificial intelligence, and particularly to the field of computer vision. More specifically, the embodiments of this disclosure relate to methods, apparatus, devices, computer-readable storage media, and computer program products for generating sample images. Background Technology

[0002] Machine learning has always had extremely high data requirements. Typically, the data used to train machine learning models requires extensive preprocessing and manual labeling, making its acquisition very expensive. To alleviate the problem of insufficient data, data augmentation is an effective technique. Data augmentation refers to expanding the existing training dataset through simple operations, thereby enriching the training dataset and significantly improving the performance of models trained on the augmented dataset.

[0003] In the field of computer vision, traditional data augmentation techniques rely on pixel-level operations, such as stitching different images together to construct new sample images. However, the sample images produced by such augmentation techniques have poor realism and are difficult to use for training image processing models for handling low-level vision tasks (such as image super-resolution or image denoising). Summary of the Invention

[0004] Embodiments of this disclosure provide a scheme for generating sample images.

[0005] In a first aspect of this disclosure, a method for generating a sample image is provided. The method includes: acquiring a first image and a second image corresponding to the first image, the first image and the second image having the same resolution, and the second image having a higher image quality than the first image; determining a spectral difference between a first frequency domain representation of the first image and a second frequency domain representation of the second image; updating the first frequency domain representation based on the spectral difference using at least a portion of the second frequency domain representation; and generating a sample image based on the updated first frequency domain representation.

[0006] In this disclosure, frequency domain representation refers to the information obtained by transforming an image from the spatial domain to the frequency domain using frequency domain methods; it can, for example, represent a spectrogram. Furthermore, in this disclosure, image quality can represent aspects such as image sharpness, noise level, or artifact severity. A second image having higher image quality than a first image can mean that the second image, for example, has higher sharpness, less noise, or fewer artifacts. An image with higher image quality may, for example, have better visual presentation (e.g., be visually clearer) or facilitate image analysis processes (e.g., make the analysis results of the image processing model more accurate).

[0007] In this disclosure, the correspondence between the first image and the second image means that the first image and the second image have corresponding image content. For example, the first image and the second image present at least partially the same visual content. Exemplarily, the first image and the second image can be different resolution versions of the same image, a pre-denoising and post-denoising version of the same image, a pre-dehazing and post-dehazing version of the same image, etc.

[0008] By determining the spectral differences between frequency domain representations and updating the original first spectral representation based on these differences, embodiments of this disclosure can fully consider the differences between the original samples, thereby improving the effectiveness of the generated sample images for model training.

[0009] In some embodiments of the first aspect, the method may further include constructing a training dataset for training an image processing model based on sample images. This approach enables data augmentation.

[0010] In some embodiments of the first aspect, the training dataset includes at least training image pairs constructed based on the second image and the sample images. Such training image pairs can, for example, be used to train an image processing model for low-level computer vision processing, thereby improving the performance of such an image processing model. Such an image processing model may, for example, include an image denoising model, an image dehazing model, or an image artifact removal model, etc.

[0011] In some embodiments of the first aspect, the training dataset includes at least training image pairs constructed based on a second image and a third image, the third image being generated based on downsampling of the sample images. Such training image pairs can, for example, be used to train an image super-resolution model, thereby improving the performance of the image super-resolution model.

[0012] In some embodiments of the first aspect, the first frequency domain representation and the second frequency domain representation are generated based on the Discrete Cosine Transform (DCT) or the Discrete Fourier Transform (DFT).

[0013] In some embodiments of the first aspect, determining the spectral difference includes: determining the elemental differences between spectral elements in a first frequency domain representation and corresponding spectral elements in a second frequency domain representation, thereby determining the spectral difference. Based on this approach, spectral elements with significant differences between different frequency domain representations can be identified.

[0014] In some embodiments of the first aspect, updating the first frequency domain representation using at least a portion of the second frequency domain representation includes: determining a first set of spectral elements from the second frequency domain representation based on spectral differences; and updating a second set of spectral elements in the first frequency domain representation corresponding to the first set of spectral elements using the first set of spectral elements.

[0015] Based on this approach, spectral elements with significant differences can be selectively chosen and updated accordingly, thereby enabling the sample image to focus on a range of frequencies with significant differences. This allows the generated sample image to more effectively help train the image processing model.

[0016] In some embodiments of the first aspect, the spectral differences include multiple region spectral differences corresponding to multiple spectral regions, and determining a first set of spectral elements from a second frequency domain representation based on the spectral differences includes: determining a predetermined number of spectral regions from multiple spectral regions based on the multiple region spectral differences; and determining a first set of spectral elements corresponding to the predetermined number of spectral regions from the second frequency domain representation.

[0017] In examples of frequency domain representations generated based on DCT, multiple spectral regions can, for example, represent multiple fan-shaped annular regions centered at the top-left vertex of the spectrogram, corresponding to different frequency domain ranges. In examples of frequency domain representations generated based on DFT, multiple spectral regions can, for example, represent multiple annular regions centered at the center of the spectrogram, corresponding to different frequency domain ranges.

[0018] Based on this approach, regions with significant differences can be selected specifically according to regional spectral differences, thereby making the generated sample images more realistic and effectively helping to train the corresponding image processing models.

[0019] In some embodiments of the first aspect, determining a predetermined number of spectrum regions from multiple spectrum regions based on multiple spectral differences includes: determining multiple selection probabilities corresponding to the multiple spectrum regions based on the multiple spectral differences; and selecting a predetermined number of spectrum regions from the multiple spectrum regions based on the multiple selection probabilities.

[0020] By using a probability-based selection method, embodiments of this disclosure can improve the randomness of frequency domain representation updates, thereby enriching the diversity of generated sample images and facilitating the training of image processing models.

[0021] In some embodiments of the first aspect, the plurality of spectral regions are determined based on the resolution of the second image. In some embodiments, the number of spectral regions may be determined based on the height and width of the second image. For example, the number may be expressed as: Where H represents the height of the second image 210, and W represents the width of the second image 210.

[0022] In some embodiments of the first aspect, the regional spectral differences are determined based on the average difference between spectral elements represented in the first frequency domain and spectral elements represented in the second frequency domain within the corresponding spectral region. This approach allows for a more objective representation of the differences in spectral elements between different spectral regions.

[0023] In some embodiments of the first aspect, updating the second set of spectral elements corresponding to the first set of spectral elements in the first frequency domain representation using the first set of spectral elements includes replacing the second set of spectral elements in the first frequency domain representation with a weighted sum of the values ​​of the first set of spectral elements and the values ​​of the second set of spectral elements. By utilizing the difference method, the sample images generated by the embodiments of this disclosure can expand the diversity of samples in the difference region.

[0024] In some embodiments of the first aspect, the weighted sum is determined based on random weights. This approach can further enhance the diversity of the generated sample images.

[0025] In some embodiments of the first aspect, the first image is generated based on upsampling of a fourth image corresponding to the second image, the fourth image having a lower resolution than the second image. Exemplarily, such a second and fourth image may be, for example, a pair of samples from an existing training dataset used to train an image super-resolution model.

[0026] In some embodiments of the first aspect, image quality indicates at least one of the following: image sharpness, the amount of noise in the image, or the degree of image artifacts.

[0027] In a second aspect of this disclosure, an apparatus for generating a sample image is provided. The apparatus includes: an acquisition unit configured to acquire a first image and a second image corresponding to the first image, the first image and the second image having the same resolution, and the second image having a higher image quality than the first image; a determination unit configured to determine a spectral difference between a first frequency domain representation of the first image and a second frequency domain representation of the second image; an update unit configured to update the first frequency domain representation based on the spectral difference using at least a portion of the second frequency domain representation; and a generation unit configured to generate a sample image based on the updated first frequency domain representation.

[0028] By determining the spectral differences between frequency domain representations and updating the original first spectral representation based on these differences, embodiments of this disclosure can fully consider the differences between the original samples, thereby improving the effectiveness of the generated sample images for model training.

[0029] In some embodiments of the second aspect, the apparatus further includes a dataset construction unit configured to construct a training dataset for training an image processing model based on sample images.

[0030] In some embodiments of the second aspect, the training dataset includes at least training image pairs constructed based on the second image and the sample images.

[0031] In some embodiments of the second aspect, the training dataset includes at least a training image pair constructed based on a second image and a third image, the third image being generated based on downsampling of the sample image.

[0032] In some embodiments of the second aspect, the first frequency domain representation and the second frequency domain representation are generated based on the Discrete Cosine Transform (DCT) or the Discrete Fourier Transform (DFT).

[0033] In some embodiments of the second aspect, the determining unit is configured to: determine the elemental differences between spectral elements in the first frequency domain representation and corresponding spectral elements in the second frequency domain representation, in order to determine the spectral differences.

[0034] In some embodiments of the second aspect, the updating unit is configured to: determine a first set of spectral elements from a second frequency domain representation based on spectral differences; and update a second set of spectral elements in the first frequency domain representation corresponding to the first set of spectral elements using the first set of spectral elements.

[0035] In some embodiments of the second aspect, the spectral differences include multiple region spectral differences corresponding to multiple spectral regions, and the update unit 630 is configured to: determine a predetermined number of spectral regions from the multiple spectral regions based on the multiple region spectral differences; and determine a first set of spectral elements corresponding to the predetermined number of spectral regions from the second frequency domain representation.

[0036] In some embodiments of the second aspect, the updating unit is configured to: determine a plurality of selection probabilities corresponding to a plurality of spectral regions based on the spectral differences of a plurality of regions; and select a predetermined number of spectral regions from the plurality of spectral regions based on the plurality of selection probabilities.

[0037] In some embodiments of the second aspect, the multiple spectral regions are determined based on the resolution of the second image.

[0038] In some embodiments of the second aspect, the regional spectral difference is determined based on the mean of the differences between spectral elements represented in the first frequency domain in the corresponding spectral region and spectral elements represented in the second frequency domain in the corresponding spectral region.

[0039] In some embodiments of the second aspect, the updating unit is configured to replace the second set of spectrum elements in the first frequency domain representation with a weighted sum of the values ​​of the first set of spectrum elements and the values ​​of the second set of spectrum elements.

[0040] In some embodiments of the second aspect, the weighted sum is determined based on random weights.

[0041] In some embodiments of the second aspect, the first image is generated based on upsampling of a fourth image corresponding to the second image, the fourth image having a lower resolution than the second image.

[0042] In some embodiments of the second aspect, image quality indicates at least one of the following: image sharpness, the amount of noise in the image, or the degree of image artifacts.

[0043] In some embodiments of the second aspect, the first image and the second image present at least partially the same visual content.

[0044] In a third aspect of this disclosure, a method for generating a sample image is provided. The method includes: acquiring a first image and a second image corresponding to the first image, the first image and the second image having the same resolution, and the second image having a higher image quality than the first image; determining a spectral difference between a first frequency domain representation of the first image and a second frequency domain representation of the second image; updating the first frequency domain representation based on the spectral difference using at least a portion of the second frequency domain representation; and generating a sample image based on the updated first frequency domain representation.

[0045] In a third aspect of this disclosure, a method for generating a sample image is provided. The method includes: acquiring a first image and a second image corresponding to the first image, the first image and the second image having the same resolution, and the second image having a higher image quality than the first image; determining a spectral difference between a first frequency domain representation of the first image and a second frequency domain representation of the second image; updating the first frequency domain representation based on the spectral difference using at least a portion of the second frequency domain representation; and generating a sample image based on the updated first frequency domain representation.

[0046] In a third aspect of this disclosure, an electronic device is provided, comprising: at least one computing unit; at least one memory coupled to the at least one computing unit and storing instructions for execution by the at least one computing unit, wherein, when executed by the at least one computing unit, the instructions cause the device to perform a method in the first aspect or any implementation thereof.

[0047] In a fourth aspect of this disclosure, a computer-readable storage medium is provided that stores one or more computer instructions thereon, wherein the one or more computer instructions are executed by a processor to implement the method of the first aspect or any implementation thereof.

[0048] In a fifth aspect of this disclosure, a computer program product is provided that, when run on a computer, causes the computer to execute instructions for some or all of the steps of the method in the first aspect or any implementation thereof.

[0049] It is understood that the electronic device of the third aspect, the computer storage medium of the fourth aspect, or the computer program product of the fifth aspect provided above are all used to perform the method provided in the first aspect. Therefore, the explanations or descriptions regarding the first aspect also apply to the third, fourth, and fifth aspects. Furthermore, the beneficial effects achieved by the third, fourth, and fifth aspects can be referred to the beneficial effects in the corresponding methods, and will not be repeated here.

[0050] Embodiments of this disclosure also provide a scheme for training an image processing model.

[0051] In a sixth aspect of this disclosure, a method for training an image processing model is provided. The method includes: acquiring a training dataset; and training an image processing model using the training dataset, wherein the training dataset is constructed based on sample images generated by: acquiring a first image and a second image corresponding to the first image by an image generation device, the first image and the second image having the same resolution, and the second image having a higher image quality than the first image; determining a spectral difference between a first frequency domain representation of the first image and a second frequency domain representation of the second image by the image generation device; updating the first frequency domain representation by the image generation device based on the spectral difference using at least a portion of the second frequency domain representation; and generating sample images by the image generation device based on the updated first frequency domain representation.

[0052] Based on this approach, embodiments of this disclosure can improve the performance of trained image processing models, and in particular, improve the ability of such image processing models to process natural images.

[0053] The embodiments of this disclosure also provide an image processing scheme.

[0054] In a seventh aspect of this disclosure, a method for image processing is provided. The method includes: receiving an input image; and processing the input image using an image processing model to obtain a processing result for the input image; wherein the image processing model is trained using a training dataset constructed based on sample images, the sample images being generated based on the following process: acquiring a first image and a second image corresponding to the first image by an image generation device, the first image and the second image having the same resolution, and the second image having a higher image quality than the first image; determining a spectral difference between a first frequency domain representation of the first image and a second frequency domain representation of the second image by the image generation device; updating the first frequency domain representation by the image generation device based on the spectral difference using at least a portion of the second frequency domain representation; and generating the sample image by the image generation device based on the updated first frequency domain representation.

[0055] Based on this approach, embodiments of this disclosure can utilize a higher-performance image processing model to process the input image, thereby improving image processing performance. Attached Figure Description

[0056] The above and other features, advantages, and aspects of the embodiments of this disclosure will become more apparent from the accompanying drawings and the following detailed description. In the drawings, the same or similar reference numerals denote the same or similar elements, wherein:

[0057] Figure 1 A schematic diagram of an example environment in which several embodiments of the present disclosure can be implemented is shown;

[0058] Figure 2 A schematic diagram of the architecture of an example sample generation device according to some embodiments of the present disclosure is shown;

[0059] Figure 3A and Figure 3B A schematic diagram illustrating the construction of a training dataset according to some embodiments of the present disclosure is shown;

[0060] Figure 4A , Figure 4B and Figure 4C A schematic diagram illustrating experimental results according to some embodiments of the present disclosure is shown;

[0061] Figure 5 A flowchart illustrating a process for generating sample images according to some embodiments of the present disclosure is shown;

[0062] Figure 6 A schematic block diagram of a sample generation apparatus according to some embodiments of the present disclosure is shown; and

[0063] Figure 7 A block diagram of a computing device capable of implementing several embodiments of the present disclosure is shown. Detailed Implementation

[0064] Embodiments of this disclosure will now be described in more detail with reference to the accompanying drawings. While some embodiments of this disclosure are shown in the drawings, it should be understood that this disclosure can be implemented in various forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided to provide a more thorough and complete understanding of this disclosure. It should be understood that the accompanying drawings and embodiments of this disclosure are for illustrative purposes only and are not intended to limit the scope of protection of this disclosure.

[0065] In the description of embodiments of this disclosure, the term "comprising" and similar terms should be understood as open-ended inclusion, i.e., "including but not limited to". The term "based on" should be understood as "at least partially based on". The term "one embodiment" or "the embodiment" should be understood as "at least one embodiment". The terms "first", "second", etc., may refer to different or the same objects. Other explicit and implicit definitions may also be included below.

[0066] As discussed above, with the widespread adoption of artificial intelligence technology, the demands for training data are increasing. Typically, data used to train machine learning models requires extensive preprocessing and manual annotation, making its acquisition often very expensive. To alleviate the problem of insufficient data, some traditional solutions involve augmenting existing training datasets to enrich them. Augmented datasets can significantly improve the performance of models trained on those datasets.

[0067] In computer vision, traditional data augmentation schemes are designed for high-level vision tasks (such as image classification, object detection, and segmentation), making it difficult to directly transfer these schemes to lower-level vision tasks (such as image super-resolution and image denoising). For example, some traditional schemes use random stitching to construct new sample images, which may be helpful for image classification. However, such sample images are difficult to use effectively for lower-level vision tasks, such as image super-resolution, image denoising, image artifact removal, or image dehazing. This results in ineffective data augmentation and may even degrade the performance of the trained machine learning model. Furthermore, the sample images generated by traditional schemes have poor realism and lack perception of the input samples, leading to low data augmentation efficiency.

[0068] Example Environment

[0069] According to various embodiments of this disclosure, a scheme for generating sample images is provided. In embodiments of this disclosure, a first image and a corresponding second image are first acquired, wherein the first and second images have the same resolution, and the second image has higher image quality than the first image. Subsequently, a spectral difference between a first frequency domain representation of the first image and a second frequency domain representation of the second image is determined, and the first frequency domain representation is updated using at least a portion of the second frequency domain representation based on the spectral difference. Further, a sample image is generated based on the updated first frequency domain representation. In this manner, embodiments of this disclosure can fully consider the differences between original samples, thereby improving the effectiveness of sample images for model training.

[0070] Figure 1A schematic diagram of an example environment 100 in which various embodiments of the present disclosure can be implemented is shown. For example... Figure 1 As shown, environment 100 includes sample generation device 120, which can receive image pairs 115 from existing image dataset 110 and generate sample images 125 based on image pairs 115.

[0071] In some embodiments, image pair 115 comprises two images presenting at least partially the same visual content, and one image having a higher image quality than the other. Depending on the type of image processing model trained using the image dataset 110, image quality can indicate aspects that the image processing model can improve, examples of which include, but are not limited to, sharpness, noise, artifacts, etc. Higher image quality, for example, can indicate higher sharpness, allowing people to view more details in the image. Alternatively, higher image quality can indicate less noise or fewer artifacts.

[0072] In one example, when the image processing model is an image super-resolution model, the image pair 115 can include different resolution versions of the same image; for example, one image is a low-resolution LR image, and the corresponding image is a high-resolution HR image. In another example, when the image processing model is an image denoising model, the image pair 115 can, for example, include a noisy version and a noise-free version of the same image, which may, for example, have the same resolution.

[0073] In some embodiments, the sample generation device 120 may first convert the sample pair 115 to the same resolution and generate a sample image 125 based on the spectral difference between their frequency domain representations. For example, when the sample pair 115 includes an LR image and an HR image, the sample generation device 120 may first upsample the LR image to obtain an image of the same size as the HR image to determine the spectral difference. The specific generation process of the sample image 125 will be referenced below. Figure 2 The details will not be elaborated here.

[0074] like Figure 1 As shown, sample image 125 can be further used to construct training dataset 130. It should be understood that the process of constructing training dataset 130 can be performed by sample generation device 120 or other suitable device. Depending on the type of image processing model to which training dataset 130 is intended, different methods can be used to construct training dataset 130.

[0075] In one example, when the image processing model is used for high-level vision tasks such as classification or object detection, the training dataset 130 can be directly constructed using sample images 125. The corresponding ground truth can be determined, for example, based on sample pair 115 with images of higher image quality, or manually labeled.

[0076] In another example, the image processing model could be trained on pairs of images with the same resolution but different image qualities, such as an image denoising model, an image dehazing model, or an image artifact removal model. Accordingly, training image pairs in the training dataset 130 can be constructed using sample image 115 and images with higher image quality from the image dataset.

[0077] In another example, the image processing model could be trained on image pairs with different resolutions, such as an image super-resolution model. Accordingly, sample image 115 can be downsampled first to obtain a new LR image. This LR image, along with the HR image from sample pair 115, can then be used to construct training image pairs in training dataset 130.

[0078] Furthermore, such as Figure 1 As shown, environment 100 may also include, for example, a model training device 140, which can receive a training dataset 130 and use the training dataset 130 to train an image processing model to obtain a trained image processing model 150. It should be understood that, although in Figure 1 In this illustration, image processing model 150 is shown as a deep neural network model, but this is merely illustrative, and any other suitable type of model is also available, including but not limited to: various deep neural networks (DNNs), convolutional neural networks (CNNs), support vector machines (SVMs), decision trees, random forest models, and so on. This disclosure is not intended to limit the specific structure of image processing model 150.

[0079] Furthermore, such as Figure 1 As shown, the environment 100 may further include, for example, a model deployment device 160, which may deploy, for example, an image processing model 150 trained by the model training device 140. Accordingly, the model deployment device 160 may receive an input image 170 and process the input image 170 using the image processing model 150 to generate a processing result 180. Depending on the type of image processing model 150, the processing result 180 may, for example, be a target recognition result of the input image 170. Alternatively, the processing result 180 may also be a high-resolution image generated based on the input image 170. Alternatively, the processing result 180 may also be a dehazed image of the input image 170, etc.

[0080] It should be understood that Figure 1 The sample generation device 120, model training device 140, and model deployment device 160 shown may be the same or different computing devices, and they should have a certain computing power to meet the computing resource requirements for implementing the method of this application. This disclosure does not limit the specific form of the above devices; for example, they may be network devices or terminal devices.

[0081] Sample image generation

[0082] First refer to Figure 2 , Figure 2 A schematic diagram of the architecture 200 of an example sample generation device 120 according to some embodiments of the present disclosure is shown. Figure 2 As shown, the sample generation device 120 includes multiple modules for generating the sample image 125. Specifically, the sample generation device 120 may include a frequency domain transformation unit 215, which is configured to acquire a first image 205 and a second image 210. The first image 205 and the second image 210 may have the same resolution, and the second image 210 has a higher image quality than the first image 205. In some embodiments, image quality may represent image sharpness, noise level, or artifact severity, etc.

[0083] As discussed above, in some embodiments, when the images in the sample pair 115 have the same resolution, the sample generation device 120 can directly use the sample pair 115 as the first image 205 and the second image 210.

[0084] In some other embodiments, the images in sample pair 115 may have different resolutions, for example, one is an LR image and the other is an HR image. Accordingly, in order to perform the spectral difference determination described below, sample generation device 120 may first upsample the LR image using an appropriate method to obtain a low-resolution image with the same resolution as the HR image. Accordingly, sample generation device 120 may use this low-resolution image as the first image 205 and the HR image as the second image 210. It should be understood that sample generation device 120 may use any suitable method for upsampling, examples of which include, but are not limited to, linear interpolation, bicubic interpolation, or bilinear interpolation.

[0085] like Figure 1 As shown, after obtaining the first image 205 and the second image 210, the frequency domain transformation unit 215 can transform the first image 205 to a first frequency domain representation 220 and transform the second image 210 to a second frequency domain representation 225. In this document, frequency domain representation refers to the frequency domain information obtained by transforming an image from the spatial domain to the frequency domain, such as a spectrogram.

[0086] In some embodiments, the frequency domain transformation unit 215 may use Discrete Cosine Transform (DCT) or Discrete Fourier Transform (DFT) to obtain a first frequency domain representation 220 and a second frequency domain representation 225.

[0087] like Figure 2 As shown, the sample generation device 220 may further include a frequency domain difference determination unit 230. In some embodiments, the frequency domain difference determination unit 230 may determine a frequency domain difference 235 between the first frequency domain representation 220 and the second frequency domain representation 225. In some embodiments, the frequency domain difference determination unit 230 may determine the elemental differences between the spectral elements in the first frequency domain representation 220 and the corresponding spectral elements in the second frequency domain representation 225, and determine the frequency domain difference 235 based on the elemental differences.

[0088] In some embodiments, the spectral difference 235 may indicate the difference between each spectral element in the first frequency domain representation 220 and each corresponding spectral element in the second frequency domain representation.

[0089] In some embodiments, the spectral difference 235 may further include multiple regional spectral differences corresponding to multiple spectral regions. For example... Figure 2 As shown, taking the first frequency domain representation 220 and the second frequency domain representation 225 obtained based on DCT transform as examples, since the origin of the spectrum obtained based on DCT transform is at the upper left corner of the spectrum, the spectrum can be divided into multiple sector-shaped spectrum regions 240 with the origin as the center. Each annular spectrum region 240 corresponds to a spectrum element within a specific frequency domain range.

[0090] In some embodiments, the spectral difference determination unit 230 may, for example, divide the spectrogram into a specified number of spectral regions. Alternatively, the number of spectral regions may also be based on the resolution of the first image 205 or the second image 210. For example, the spectral difference determination unit 230 may divide the spectrogram into r spectral rings, wherein... H represents the height of the first image 205 or the second image 210, and W represents the width of the first image 205 or the second image 210.

[0091] Accordingly, the spectral difference determination unit 230 can determine the regional spectral difference for each annular spectral region 240. For example, the regional spectral difference can be expressed as:

[0092]

[0093] Where N r This represents the number of spectral elements on the r-th spectral ring. The first frequency domain represents the spectral element of 220 in the r-th spectral ring. The second frequency domain represents the corresponding spectral element of 225 in the r-th spectral ring.

[0094] For the frequency domain representation obtained based on DFT transform, the origin of the generated spectrum is at the center of the spectrum. The spectrum difference determination unit 230 can divide the spectrum into multiple annular spectrum regions centered on the origin in a similar manner, and determine the regional spectrum difference of each spectrum region based on a method similar to formula (1).

[0095] It should be understood that any other suitable spectral region can also be used to determine the spectral difference 235 between the first spectral image 220 and the second spectral image 225.

[0096] like Figure 1 As shown, the sample generation device 120 may further include a frequency domain representation update unit 250. In some embodiments, the frequency domain representation update unit 250 may determine a first set of spectral elements from a second frequency domain representation 225 based on the determined spectral differences 235.

[0097] In some embodiments, as described above, the spectral difference 235 can indicate the elemental difference between corresponding spectral elements in the first frequency domain representation 220 and the second frequency domain representation 225. Accordingly, the frequency domain representation update unit 250 can, for example, compare the elemental difference with a predetermined threshold and determine the spectral elements with elemental differences greater than the threshold as a first group of spectral elements.

[0098] In some other embodiments, the spectral difference 235 may include, for example, multiple regional spectral differences. Accordingly, the frequency domain representation update unit 250 may, for example, determine a predetermined number of spectral regions from multiple spectral regions based on the multiple regional spectral differences. Exemplarily, the frequency domain representation update unit 250 may select the K spectral regions (K being a positive integer) with the largest spectral differences from the multiple spectral regions.

[0099] Alternatively, to improve the randomness of the generated sample images, the frequency domain representation update unit 250 can also first determine the selection probability corresponding to multiple spectral regions based on the spectral differences of multiple regions. Taking the DCT transform discussed above as an example, the selection probability corresponding to each sector spectral region can be expressed as:

[0100]

[0101] Where e is the natural constant, g r The difference in the spectral region of the r-th sector ring is determined according to formula (1), where H represents the height of the first image 205 or the second image 210, and W represents the width of the first image 205 or the second image 210.

[0102] Accordingly, the frequency domain representation update unit 250 can select a predetermined number of frequency regions from multiple frequency regions based on the selection probability corresponding to each frequency region. For example, the frequency domain representation update unit 250 can perform K samplings without replacement (K is a positive integer) based on the selection probability of each frequency region to obtain K frequency regions.

[0103] In some embodiments, the frequency domain representation update unit 250 may further determine spectral elements corresponding to a predetermined number (e.g., K) of spectral regions from the second frequency domain representation 225 as a first set of spectral elements. Based on this, the frequency domain representation update unit 250 can determine the region with the larger region in the first frequency domain representation 220 and the second frequency domain representation 225.

[0104] In some embodiments, the frequency domain representation update unit 250 may further utilize the determined first set of spectral elements to update the second set of spectral elements corresponding to the first set of spectral elements in the first frequency domain representation 220. The frequency domain representation update unit 250 may, for example, update the second set of spectral elements based on an interpolation method. In some embodiments, the frequency domain representation update unit 250 may determine a weighted sum of the first set of spectral elements and the second set of spectral elements, and use this weighted sum to replace the corresponding spectral elements in the first frequency domain representation 220, thereby obtaining an updated first frequency domain representation 255. Exemplarily, the weighted sum may be expressed as:

[0105]

[0106] in This represents the updated spectrum element. This represents the second group of spectral elements. Let α represent the first set of spectral elements and α represent the weight value. Based on this, the frequency domain representation update unit 250 can effectively mix the first frequency domain representation 220 and the second frequency domain representation 225, and can enrich the diversity in regions where the two differ significantly.

[0107] In some implementations, to further enhance the randomness of the frequency domain representation mixing and thus increase the diversity of the generated sample images, the weight value α can be, for example, a random number.

[0108] like Figure 2 As shown, the sample generation device 120 also includes an inverse transform unit 260, which is configured to generate a sample image 125 based on an updated first frequency domain representation 255 generated by the frequency domain representation update unit 250. It should be understood that the inverse transform unit 260 may employ an inverse transform method corresponding to the method used to generate the first frequency domain representation 220 and the second frequency domain representation 225, such as an inverse DCT transform or an inverse DFT transform.

[0109] Based on the methods discussed above, the sample generation device 120 can generate a sample image 125 that is different from the first image 205 and has a lower image quality than the second image 210, based on the input first image 205 and second image 210. By converting the image to a frequency domain representation and determining the differences between the frequency domain representations, the embodiments of this disclosure can both ensure the authenticity of the generated sample image 125 and enrich the diversity of regions with large differences, thereby enabling the generated sample images to be used more effectively for training image processing models.

[0110] Construction of training dataset

[0111] Figure 3A and Figure 3B Schematic diagrams 300A and 300B illustrate the construction of a training dataset 130 according to embodiments of the present disclosure. (Refer to the above text.) Figure 1 The appropriate equipment discussed (e.g., Figure 1 The sample generation device 120 can construct a training dataset 130 based on the generated sample images 125.

[0112] In some embodiments, the sample generation device 120 may construct the training dataset 130 in a manner corresponding to the type of image processing model to be used for training in the training dataset 130.

[0113] In one example, the image processing model may be for high-level vision tasks (e.g., classification, object detection), which receives input sample images and corresponding annotation information (i.e., ground truth) during training. Accordingly, the sample generation device 120 can use the sample images 125 to directly construct the training dataset 130 as input sample images for training the image processing model.

[0114] In some embodiments, the annotation information corresponding to the sample image 125 may be determined based on the second image 210 with higher image quality. Taking a classification model as an example, the standard information of the sample image 125 may inherit the original annotation information "Category: Panda" from the second image 210.

[0115] Alternatively, the annotation information could also be the processing result of the second image 210 based on the image processing model. Since the second image 210 has higher image quality, the image processing model is able to obtain an accurate prediction result of the second image 210, which can be further used as annotation information corresponding to the sample image 125.

[0116] In another example, the image processing model could be trained on pairs of images with the same resolution but different image qualities. Examples of such image processing models include, but are not limited to, image denoising models, image dehazing models, or image artifact removal models. Accordingly, such as Figure 3A As shown, the sample generation device 120 can use the generated sample image 125 and the second image 210 to construct training image pairs in the training dataset 130.

[0117] In this way, the generated sample pairs can enhance the trained image processing model's ability to process regions with large differences, thereby improving the performance of the image processing model in a targeted manner.

[0118] In yet another example, the image processing model could be trained on pairs of images with different resolutions, such as an image super-resolution model. Accordingly, as... Figure 3B As shown, in order to obtain image pairs for training the image super-resolution model, the sample generation device 120 can first downsample the generated sample image 115 to obtain a low-resolution image 310, and use the low-resolution image 310 and the second image 210 to construct training image pairs in the training dataset 130.

[0119] Based on this approach, the generated sample pairs can enhance the recovery ability of the trained image super-resolution model for regions with large frequency domain differences, thereby specifically improving the performance of the trained image super-resolution model.

[0120] Model training and use

[0121] After obtaining the training dataset 130 based on the methods discussed above, as follows Figure 1 As shown, the model training device 140 can further train the image processing model based on the training dataset 130 to obtain a trained image processing model 150. In some embodiments, the model training device 140 may be a computing device with strong computing power, such as a cloud server. In this way, the model training device 140 can obtain an image processing model 150 with better performance.

[0122] Such as combination Figure 1 The image processing model 150 discussed here can be a machine learning model for performing high-level vision tasks (e.g., classification, object detection, or object segmentation). Alternatively, the image processing model 150 can also be a machine learning model for performing low-level vision tasks (e.g., super-resolution, denoising, dehazing, or artifact removal).

[0123] In some embodiments, such as Figure 1As shown, the trained image processing model 150 can be further deployed to the model deployment device 160. The model deployment device 160 can receive the input image 170 and use the trained image processing model 150 to process the input image 170 to obtain the corresponding processing result 180.

[0124] In some embodiments, the model deployment device 160 may be an appropriate type of terminal device, such as a smartphone, tablet, smartwatch, laptop, etc.

[0125] In some embodiments, the input image 170 may be input by a user of the model deployment device 160. Taking the image processing model 150 as an image super-resolution model as an example, the user may upload the input image 170 through the input interface provided by the model deployment device 160, thereby obtaining a higher resolution and clearer image generated using the image super-resolution model. The input image 170 may include, for example, medical images, thereby enabling the acquisition of more accurate medical image information.

[0126] In some embodiments, the input image 170 may be automatically acquired by the model deployment device 160, for example. Taking the image processing model 150 as an image super-resolution model as an example, the model deployment device 160 may, for example, acquire the image captured by the camera as the input image 170 while the user is taking a picture with the camera, and process the input image 170 using the image super-resolution model to obtain an image with higher resolution and clearer image.

[0127] In some embodiments, the input image 170 may be received by the model deployment device 160 from another device. Taking the image processing model 150 as an example, which is an image super-resolution model, considering the cost of bitstream transmission, the sending end typically compresses a higher-resolution image into a lower-resolution image when sending the image. The model deployment device 160 may, for example, receive a low-resolution image sent from another device and use the image super-resolution model to generate a corresponding image with a higher resolution and clearer image. In this way, the user experience can be improved while reducing transmission costs.

[0128] It should be understood that the above scenarios are merely illustrative, and the image processing model trained based on the method of this disclosure can also be applied to other appropriate scenarios, which will not be listed in this disclosure.

[0129] Furthermore, experiments have shown that the training dataset 130 constructed using the method disclosed herein can significantly improve the performance of image super-resolution models. Figure 4A , Figure 4B and Figure 4CSchematic diagrams 400A, 400B, and 400C illustrate test results according to some embodiments of the present disclosure.

[0130] like Figure 4A As shown, the method based on this disclosure has a significant improvement in training CARN, RCAN and EDSR super-resolution models. Figure 4A The data circled in the box (DCTmix) represents the PSRN (Peak Signal-to-Noise Ratio) / SSIM (Structural Similarity) performance metrics obtained by the image processing model trained using the model training method of this disclosure (based on DCT frequency domain representation update) on different datasets (DIV2K dataset and natural image dataset). It can be seen that the method of this disclosure has a significant performance improvement over the baseline model, and the improvement is particularly significant on the natural image dataset.

[0131] also, Figure 4B The generalization performance of the model training method according to this disclosure is shown. Generalization performance represents the ability to directly transfer a model trained on one dataset to other datasets. For example... Figure 4B As shown in the box, the method disclosed herein can further improve the generalization performance of the trained image processing model.

[0132] Figure 4C The visual effect of the image processing model using the model training method of this disclosure (frequency domain representation update based on DCT) is further demonstrated. Figure 4C The visual differences in image super-resolution results between two samples from the Urban100 and RealSR datasets are illustrated using the X4 image super-resolution model as an example. Figure 4C As can be seen, the embodiments of this disclosure achieve better visual effects compared to the baseline model.

[0133] Example processes, apparatus and equipment

[0134] Figure 5 A flowchart of a process 500 for generating sample images according to an embodiment of the present disclosure is shown. Process 500 may, for example, be generated by... Figure 1 The sample generation device 120 is used for implementation. For ease of description, the following references... Figure 1 and Figure 2 To describe process 500.

[0135] In block 502, the sample generation device 120 acquires a first image 205 and a second image 210 corresponding to the first image 205, wherein the first image 205 and the second image 210 have the same resolution, and the second image 210 has a higher image quality than the first image 205. In block 504, the sample generation device 120 determines a spectral difference 235 between a first frequency domain representation 220 of the first image 205 and a second frequency domain representation 225 of the second image 210. In block 506, the sample generation device 120 updates the first frequency domain representation 220 based on the spectral difference 235, using at least a portion of the second frequency domain representation 225. In block 508, the sample generation device 120 generates a sample image 125 based on the updated first frequency domain representation 255.

[0136] Figure 7 A block diagram of an apparatus 600 for generating sample images according to an embodiment of the present disclosure is further shown. The apparatus 600 may include multiple modules for performing, for example... Figure 5 The corresponding steps in process 500 discussed herein. For example... Figure 6 As shown, the apparatus 600 includes an acquisition unit 610 configured to acquire a first image and a second image corresponding to the first image, the first image and the second image having the same resolution, and the second image having a higher image quality than the first image. The apparatus 600 also includes a determination unit 620 configured to determine a spectral difference between a first frequency domain representation of the first image and a second frequency domain representation of the second image. The apparatus 600 further includes an update unit 630 configured to update the first frequency domain representation based on the spectral difference using at least a portion of the second frequency domain representation. Furthermore, the apparatus 600 includes a generation unit 640 configured to generate a sample image based on the updated first frequency domain representation.

[0137] In some embodiments, the apparatus 600 further includes a dataset construction unit configured to construct a training dataset for training an image processing model based on sample images.

[0138] In some embodiments, the training dataset includes at least training image pairs constructed based on the second image and the sample image.

[0139] In some embodiments, the training dataset includes at least a training image pair constructed based on a second image and a third image, wherein the third image is generated based on downsampling of the sample images.

[0140] In some embodiments, the first frequency domain representation and the second frequency domain representation are generated based on the Discrete Cosine Transform (DCT) or the Discrete Fourier Transform (DFT).

[0141] In some embodiments, the determining unit 620 is configured to: determine the elemental differences between spectral elements in the first frequency domain representation and corresponding spectral elements in the second frequency domain representation, so as to determine the spectral differences.

[0142] In some embodiments, the updating unit 630 is configured to: determine a first set of spectral elements from a second frequency domain representation based on spectral differences; and update a second set of spectral elements in the first frequency domain representation corresponding to the first set of spectral elements using the first set of spectral elements.

[0143] In some embodiments, the spectral differences include multiple region spectral differences corresponding to multiple spectral regions, and the update unit 630 is configured to: determine a predetermined number of spectral regions from the multiple spectral regions based on the multiple region spectral differences; and determine a first set of spectral elements corresponding to the predetermined number of spectral regions from a second frequency domain representation.

[0144] In some embodiments, the updating unit 630 is configured to: determine multiple selection probabilities corresponding to multiple spectral regions based on the spectral differences of multiple regions; and select a predetermined number of spectral regions from the multiple spectral regions based on the multiple selection probabilities.

[0145] In some embodiments, the plurality of spectral regions are determined based on the resolution of the second image.

[0146] In some embodiments, the regional spectral difference is determined based on the mean of the differences between spectral elements in the first frequency domain represented in the corresponding spectral region and spectral elements in the second frequency domain represented in the corresponding spectral region.

[0147] In some embodiments, the update unit 630 is configured to replace the second set of spectrum elements in the first frequency domain representation with a weighted sum of the values ​​of the first set of spectrum elements and the values ​​of the second set of spectrum elements.

[0148] In some embodiments, the weighted sum is determined based on random weights.

[0149] In some embodiments, the first image is generated based on upsampling of a fourth image corresponding to the second image, the fourth image having a lower resolution than the second image.

[0150] In some embodiments, the first image and the second image present at least partially the same visual content.

[0151] Figure 7A schematic block diagram of an example device 700 that can be used to implement embodiments of the present disclosure is shown. Device 700 can be used to implement sample generation device 120, model training device 140, and / or model deployment device 160. As shown, device 700 includes a computing unit 701, which can perform various appropriate actions and processes according to computer program instructions stored in random access memory (RAM) and / or read-only memory (ROM) 702 or loaded from storage unit 707 into RAM and / or ROM 702. Various programs and data required for the operation of device 700 may also be stored in RAM and / or ROM 702. The computing unit 701 and RAM and / or ROM 702 are interconnected via bus 703. Input / output (I / O) interface 704 is also connected to bus 703.

[0152] Multiple components in device 700 are connected to I / O interface 704, including: input unit 705, such as keyboard, mouse, etc.; output unit 706, such as various types of monitors, speakers, etc.; storage unit 707, such as disk, optical disk, etc.; and communication unit 708, such as network card, modem, wireless transceiver, etc. Communication unit 708 allows device 700 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0153] The computing unit 701 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the computing unit 701 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 701 performs the various methods and processes described above, such as process 600. For example, in some embodiments, process 600 may be implemented as a computer software program tangibly contained in a machine-readable medium, such as storage unit 707. In some embodiments, part or all of the computer program may be loaded and / or installed on device 700 via RAM and / or ROM and / or communication unit 708. When the computer program is loaded into RAM and / or ROM and executed by computing unit 701, one or more steps of process 600 described above may be performed. Alternatively, in other embodiments, computing unit 701 may be configured to perform process 1000 by any other suitable means (e.g., by means of firmware).

[0154] The program code used to implement the methods of this disclosure may be written in any combination of one or more programming languages. This program code may be provided to a processor or controller of a general-purpose computer, special-purpose computer, or other programmable data processing apparatus, such that when executed by the processor or controller, the program code causes the functions / operations specified in the flowcharts and / or block diagrams to be implemented. The program code may be executed entirely on a machine, partially on a machine, as a standalone software package partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0155] In the context of this disclosure, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. Machine-readable media can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0156] Furthermore, although the operations are described in a specific order, this should be understood as requiring that such operations be performed in the specific order shown or in sequential order, or requiring that all illustrated operations be performed to achieve the desired result. In certain environments, multitasking and parallel processing may be advantageous. Similarly, although several specific implementation details are included in the above discussion, these should not be construed as limiting the scope of this disclosure. Certain features described in the context of individual embodiments may also be implemented in combination in a single implementation. Conversely, various features described in the context of a single implementation may also be implemented individually or in any suitable sub-combination in multiple implementations.

[0157] Although the subject matter has been described using language specific to structural features and / or methodological logic, it should be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or actions described above. Rather, the specific features and actions described above are merely illustrative examples of implementing the claims.

Claims

1. A method for generating sample images, comprising: Acquire a first image and a second image corresponding to the first image, wherein the first image and the second image have the same resolution, and the second image has a higher image quality than the first image; Determine the spectral difference between the first frequency domain representation of the first image and the second frequency domain representation of the second image; Based on the spectral differences, the first frequency domain representation is updated using at least a portion of the second frequency domain representation; as well as Based on the updated first frequency domain representation, a sample image is generated; Determining the spectral differences includes: The elemental differences between the spectral elements in the first frequency domain representation and the corresponding spectral elements in the second frequency domain representation are determined to identify the spectral differences.

2. The method according to claim 1, further comprising: Based on the sample images, a training dataset is constructed for training the image processing model.

3. The method of claim 2, wherein the training dataset includes at least a training image pair constructed based on the second image and the sample image.

4. The method according to claim 2, wherein the training dataset includes at least a training image pair constructed based on the second image and the third image, wherein the third image is generated based on downsampling of the sample image.

5. The method according to claim 1, wherein the first frequency domain representation and the second frequency domain representation are generated based on the Discrete Cosine Transform (DCT) or the Discrete Fourier Transform (DFT).

6. The method of claim 1, wherein updating the first frequency domain representation using at least a portion of the second frequency domain representation comprises: The first set of spectral elements is determined from the second frequency domain representation based on the spectral differences; as well as The second set of spectrum elements corresponding to the first set of spectrum elements in the first frequency domain representation is updated using the first set of spectrum elements.

7. The method of claim 6, wherein the spectral differences include spectral differences corresponding to multiple spectral regions, and determining the first set of spectral elements from the second frequency domain representation based on the spectral differences includes: Based on the spectral differences among the multiple regions, a predetermined number of spectral regions are determined from the multiple spectral regions; as well as The first set of spectral elements corresponding to the predetermined number of spectral regions are determined from the second frequency domain representation.

8. The method of claim 7, wherein determining a predetermined number of spectral regions from the plurality of spectral regions based on the plurality of spectral differences comprises: Based on the spectral differences of the multiple regions, determine multiple selection probabilities corresponding to the multiple spectral regions; as well as Based on the multiple selection probabilities, a predetermined number of spectrum regions are selected from the multiple spectrum regions.

9. The method of claim 8, wherein the plurality of spectral regions are determined based on the resolution of the second image.

10. The method of claim 7, wherein the regional spectral difference is determined based on the mean of the differences between the spectral elements represented in the first frequency domain in the corresponding spectral region and the spectral elements represented in the second frequency domain in the corresponding spectral region.

11. The method of claim 6, wherein updating the second set of spectral elements corresponding to the first set of spectral elements in the first frequency domain representation using the first set of spectral elements comprises: The second group of spectrum elements in the first frequency domain representation is replaced by a weighted sum of the values ​​of the first group of spectrum elements and the values ​​of the second group of spectrum elements.

12. The method of claim 11, wherein the weighted sum is determined based on random weights.

13. The method of claim 1, wherein the first image is generated based on upsampling of a fourth image corresponding to the second image, the fourth image having a lower resolution than the second image.

14. The method of claim 1, wherein the first image and the second image present at least partially the same visual content.

15. The method of claim 1, wherein the image quality indicates at least one of the following: image sharpness, the number of noise points in the image, or the degree of artifacts in the image.

16. A method for training an image processing model, comprising: Obtain the training dataset; as well as The image processing model is trained using the training dataset. The training dataset is constructed based on sample images, which are generated using the following process: A first image and a second image corresponding to the first image are acquired by an image generation device. The first image and the second image have the same resolution, and the second image has a higher image quality than the first image. The spectral difference between the first frequency domain representation of the first image and the second frequency domain representation of the second image is determined by the image generation device; The image generation device updates the first frequency domain representation based on the spectral differences using at least a portion of the second frequency domain representation; as well as The sample image is generated by an image generation device based on the updated first frequency domain representation; The determination of the spectral difference by the image generation device includes: The image generating device determines the elemental differences between the spectral elements in the first frequency domain representation and the corresponding spectral elements in the second frequency domain representation to determine the spectral differences.

17. An image processing method, comprising: Receive input image; as well as The input image is processed using an image processing model to obtain the processing result for the input image; The image processing model is trained using a training dataset constructed from sample images, which are generated based on the following process: A first image and a second image corresponding to the first image are acquired by an image generation device. The first image and the second image have the same resolution, and the second image has a higher image quality than the first image. The spectral difference between the first frequency domain representation of the first image and the second frequency domain representation of the second image is determined by the image generation device; The image generation device updates the first frequency domain representation based on the spectral differences using at least a portion of the second frequency domain representation; as well as The sample image is generated by an image generation device based on the updated first frequency domain representation; The determination of the spectral difference by the image generation device includes: The image generating device determines the elemental differences between the spectral elements in the first frequency domain representation and the corresponding spectral elements in the second frequency domain representation to determine the spectral differences.

18. An apparatus for generating sample images, comprising: The acquisition unit is configured to acquire a first image and a second image corresponding to the first image, wherein the first image and the second image have the same resolution, and the second image has a higher image quality than the first image; The determining unit is configured to determine the spectral difference between a first frequency domain representation of the first image and a second frequency domain representation of the second image; The update unit is configured to update the first frequency domain representation based on the spectral difference using at least a portion of the second frequency domain representation; as well as The generation unit is configured to generate sample images based on the updated first frequency domain representation; The determining unit is further configured to: determine the elemental differences between the spectral elements in the first frequency domain representation and the corresponding spectral elements in the second frequency domain representation, thereby determining the spectral differences.

19. An electronic device comprising: At least one computing unit; At least one memory coupled to the at least one computing unit and storing instructions for execution by the at least one computing unit, the instructions, when executed by the at least one computing unit, causing the device to perform the method according to any one of claims 1-15, 16 or 17.

20. A computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the method according to any one of claims 1-15, 16 or 17.

21. A computer program product comprising computer-executable instructions, wherein the computer-executable instructions, when executed by a processor, implement the method according to any one of claims 1-15, 16 or 17.

Images