Method for bilateral interpolation of images

Abstract

The invention provides a method for bilateral interpolation of an image, equipment for implementing the method and a software storage product. The method comprises: providing an input image to be magnified by bilateral interpolation; providing an initial image; providing a guide image; for each pixel position in the final image or a corresponding pixel position in the initial image, selecting a region of interest ROIspat around the corresponding pixel position in the input image, and determining a spatial distance dspat from the pixel position to each pixel in the ROIspat in the input image; for each pixel position in the initial image, selecting a region of interest ROIint around the corresponding pixel position in the guide image, and determining an intensity distance dint from the pixel position to each pixel in the ROIint in the guide image; and determining a pixel value at the pixel position on the basis of the weighted average value of all pixels in the ROIspat based on the spatial distance and the weighted average value of all pixels in the ROIint based on the intensity distance to generate a final image.

Description

Technical Field

[0001] The present invention relates to a method for bilateral interpolation of an image, and in particular to upscaling an image by bilateral interpolation. In addition, the present invention relates to a device and a software storage product for implementing the method. Background Art

[0002] Digital images continue to grow in size, evolving from 250,000-pixel video frames to multi-million-pixel digital photographs. Simultaneously, image processing operations—including local tone mapping (LTM), image quality enhancement (IQE), spatial automatic white balance (AWB), dense optical flow (DOF), bokeh (Bokeh), depth super-resolution (SR), and colorization—have proven effective in enhancing photographs. However, the ubiquitous use of mobile phones to capture and process most photographs presents a challenge, as stringent power budgets limit the computational power of edge processors (i.e., on-device processing units). This limitation makes many common algorithms in the literature unsuitable for deployment on mobile devices, especially for real-time applications. A common strategy for accelerating image processing operations is to first apply them to a low-resolution image and then upscale the result to produce the final image. Ideally, this upscaling should reintroduce details present in the high-resolution original image that were lost during downscaling. Consequently, image processing operations only require processing the low-resolution image, reducing computational requirements and enabling fast or even real-time processing.

[0003] As the field of image processing continues to advance, the development of upscaling algorithms is becoming increasingly important for enhancing the resolution and quality of digital images. Upscaling, also known as image interpolation or enlargement, involves generating a high-resolution image from a low-resolution image.

[0004] Early upscaling algorithms were characterized by simple methods such as nearest neighbor interpolation, bilinear interpolation, and bicubic interpolation. Nearest neighbor interpolation selects the value of the nearest pixel, bilinear interpolation considers a weighted average of the four nearest pixels, and bicubic interpolation expands it to a 4×4 neighborhood. While these methods provide basic upscaling capabilities, they often result in blurry and artifact-ridden images.

[0005] In the realm of hardware-friendly upscaling algorithms, bilinear interpolation and nearest neighbor interpolation represent particularly efficient options. Bilinear interpolation involves a simple weighted average of neighboring pixels, making it computationally lightweight and suitable for real-time applications. Nearest neighbor interpolation, while less complex, is even more computationally efficient because it directly selects the value of the nearest pixel without requiring complex calculations. These algorithms are well-suited for implementation on hardware with limited computational resources, such as embedded systems and low-power devices.

[0006] Furthermore, line buffers play a crucial role in optimizing the efficiency of the amplification algorithms in hardware. Line buffers store rows of pixel values, allowing for parallel processing of multiple pixels at the same time. By fully utilizing line buffers, hardware implementations can efficiently manage data flow, thereby reducing the need for large amounts of random access memory (RAM) and reducing overall computational speed and power. On the other hand, these line buffers increase the area cost of the algorithm and are therefore undesirable. Therefore, algorithms without line buffers are desirable because the cost of the hardware algorithm can be kept manageable without compromising power and computational speed. The combination of simple interpolation algorithms and the efficient use of line buffers makes these hardware-friendly approaches ideal for scenarios with limited computational resources. Summary of the Invention

[0007] The object of the present invention is to provide an upscaling or interpolation method that is efficient and has low hardware requirements.

[0008] This problem is solved by a method for bilateral interpolation of an image, a device and a software storage product implementing such a method according to the following aspects.

[0009] In one aspect of the present invention, a method for bilateral interpolation of an image, and in particular for upscaling an image, is provided. The method comprises:

[0010] Provide the input image to be upscaled using bilateral interpolation;

[0011] Provide an initial image;

[0012] Provide guiding imagery;

[0013] For each pixel position in the final image or the corresponding pixel position in the initial image, a region of interest (ROI) is selected around the corresponding pixel position in the input image. spat ), and determine the distance from the pixel position to the ROI in the input image spat The spatial distance d of each pixel within spat ;

[0014] For each pixel position in the initial image, a region of interest (ROI) is selected around the corresponding pixel position in the guidance image. int ), and determine the distance from the pixel position to the ROI in the guidance image int The intensity of each pixel within distance d int ;

[0015] ROI based on spatial distance spat The weighted average of all pixels in the ROI and the intensity distance int The pixel value at the pixel location is determined based on the weighted average of all pixels in to generate the final image.

[0016] Therefore, an input image is provided, where the input image is an image to be upscaled via bilateral interpolation. The input image may be a mask. In addition, a guide image is provided as additional information to be used during bilateral interpolation. The initial image is raw or pre-processed image data captured by an image sensor, where the input image is derived from the initial image through downscaling and additional image manipulation / processing. Therefore, the initial image can be downscaled to apply one or more image processing operations, such as LTM, IQE, spatial AWB, DOF, bokeh, depth SR, colorization, etc. One or more of these image processing operations can be applied to the downscaled initial image to generate input for an upscaling algorithm according to the present invention. Therefore, due to the reduced image size and reduced data volume, the image processing operations can be applied more quickly, in real time, and in a hardware-efficient manner. The resulting input image is then upscaled via bilateral interpolation according to the present invention to obtain a final image.

[0017] For bilateral interpolation, calculate the spatial distance d spat Wherein, the spatial distance may be the Euclidean distance between the pixel position and the pixel obtained from the input image. However, the present invention is not limited to the Euclidean distance as the spatial distance, and is not limited to the distance that depends on the pixel position and the ROI in the input image. spat The pixel position may be the location / position of the pixel in the final image to be determined by bilateral interpolation. If the final image and the initial image have the same resolution, then the pixel position of the final image directly corresponds to the pixel position in the initial image. If the final image has a different resolution than the initial image, then the ROI may be determined around the pixel position to be determined in the final image or the pixel position in the initial image that corresponds to the pixel position to be determined in the final image. spat .

[0018] Furthermore, for each pixel position in the initial image, the intensity distance d is determined int , the intensity distance indicates the intensity at the pixel location and the intensity at the ROI int The intensity difference between corresponding pixels within the guidance image.

[0019] Based on the spatial distance and intensity distance, the pixel value at the pixel location is calculated as a weighted average. Thus, the spatial distance information used in the guide image, as well as the intensity distance information, is used to perform bilateral interpolation or amplification on the input image to obtain the final image. This method can be executed pixel by pixel or as a pixel-in-pixel-out algorithm, thus not requiring any line buffers. Consequently, the hardware cost of implementing this method can be significantly reduced.

[0020] Preferably, the guide image is monochrome.

[0021] Preferably, the guidance image is generated using a conventional downscaling algorithm or a learning-based approach.

[0022] Preferably, the size of the guide image is equal to the size of the input image to be upscaled by bilateral interpolation. Thus, the number of pixels in the guide image and the input image is the same.

[0023] Preferably, before determining the intensity distance, the initial image is transformed into a monochrome initial image, wherein the transformation of the initial image into the monochrome initial image may be the same as the transformation used to generate the monochrome guide image.

[0024] Preferably, the size of the initial image is the same as the size of the final image.

[0025] Preferably, the intensity distance is the difference in intensity values ​​of the corresponding considered pixels of the initial image / monochromatic initial image and the guidance image.

[0026] Preferably, the ROI used to determine the intensity distance int ROI used to determine spatial distance spat same.

[0027] Preferably, ROI int The size of ROI is 2×2 pixels, 4×4 pixels, 8×8 pixels, 16×16 pixels, 32×32 pixels, 64×64 pixels or one of larger sizes. Additionally or alternatively, ROI spat The size of is one of 2×2 pixels, 4×4 pixels, 8×8 pixels, 16×16 pixels, 32×32 pixels, 64×64 pixels, or larger.

[0028] Preferably, ROI spat The size of is equal for the entire image. Therefore, for each pixel position in the original image, a ROI of the same size is selected spat Additionally or alternatively, ROI int The size of is equal for the entire image. Therefore, for each pixel position in the original image, a ROI of the same size is selected int .

[0029] Alternatively, ROI spat The size of ROI is changed for the corresponding pixel position in the initial image, where spat The size of ROI can be adjusted according to the initial image and in particular according to the structural size of the object in the initial image, the input image or the guidance image. int The size of ROI is changed for the corresponding pixel position in the initial image, whereint The size of may be adjusted according to the initial image, and in particular according to the structural size of the object in the initial image (ie, the scene of the initial image), the input image or the guidance image.

[0030] Preferably, the spatial distance d spat and intensity distance d int By combining the following

[0031] d comb =d spat w spat +d int ·(1-w spat ),

[0032] where w spat is the interpolation weight, w spat ∈[0,1]. In particular, w spat can be chosen to be between 0.4 and 0.6. In particular, the interpolation weight w spat is the same for every pixel position. Instead, the interpolation weights w spat can be adjusted for the corresponding pixel position. In particular, the interpolation weight w spat The adjustment is performed based on the structural size of the objects in the initial image (ie, the scene of the initial image), the guidance image, or the input image.

[0033] If w spat = 1.0, then interpolation uses only spatial position information to estimate the output pixel value. Therefore, the image content is omitted in the interpolation. spat = 0.0, then interpolation uses only intensity information to estimate the output pixel value. Therefore, no spatial information is used in interpolation.

[0034] Thus, w between 0.4 and 0.6 spat It is a good compromise for using spatial position information and image data.

[0035] Preferably, the weighted average value for determining the pixel value at the pixel position is obtained by spat and / or intensity distance d int and / or combined distance d comb exponentially decaying weight w pix In particular, w pix With exp(-d spat )、exp(-d int ) and exp(-d comb) is proportional to one or more of the following. The exponential decay can be provided by an exponential function, a Gaussian function, etc. Therefore, for the pixel values ​​in the final image, pixels that are further away in terms of distance or intensity are less considered. More specifically, the weight of the pixel value is calculated by w pix =exp(-d comb *k), where k is a configurable parameter. By changing k, more or less weight can be obtained for the central part of the corresponding ROI. k can be selected between 1 and 100, in particular between 20 and 50. By controlling a faster decay, the weights of pixels further away are taken into account. The parameter k can be fixed for the entire final image. Alternatively, k can be adjusted for the corresponding pixel position in the final image, and in particular according to the structure of the object in the initial image, the guide image or the input image.

[0036] Preferably, for each pixel position, the pixel value is determined by:

[0037]

[0038] HR pix is the pixel value in the final image and LR pix is the pixel value of a pixel in the input image. It sums up all the pixels in the corresponding region of interest, whether ROI int ROI spat Or is it a combined ROI of the two? Therefore, the weight w can be adjusted accordingly pix Therefore, for a 4×4 pixel ROI, in order to calculate the combined distance d comb , the pixel values ​​in the final image are obtained by using the weights w pix The interpolation is performed by calculating the weighted average of 16 pixels in a 4×4 ROI.

[0039] Preferably, the number of pixels of the final image is 16 times, preferably 32 times, more preferably 64 times or more the number of pixels of the input image.Thus, the scaling factor of the input image may be 16x, 32x or 64x or more.

[0040] In one aspect of the present invention, a device is provided, which includes a processor and a storage memory, wherein the storage memory stores instructions, and when the instructions are executed by the processor, the steps of the method described above are performed. The device may be a user terminal, such as a smart phone, a tablet computer, etc. In particular, the device may include a camera or an image sensor to obtain an initial image. The device may also include an ISP pipeline to generate an input image to be enlarged by bilateral interpolation. The initial image may be provided by a camera sensor in the input image, and the guide image is provided by an ISP or a processor. The initial image is reduced and one or more image processing operations (such as LTM, IqE, spatial AWB, DOF, bokeh, depth SR, coloring, etc.) are applied to a low-resolution image. The low-resolution input image may be a mask. After applying the image processing operation, the input image is enlarged to a final image by the method described above.

[0041] In one aspect of the present invention, a software storage product is provided that stores instructions, which, when executed by a processor, perform the steps of the method described above. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] The present invention will be described in more detail below with reference to the accompanying drawings.

[0043] The accompanying drawings show:

[0044] Figure 1 shows a schematic flow chart of the method according to the present invention,

[0045] Figure 2 An exemplary embodiment of the method according to the invention is shown,

[0046] Figure 3a and Figure 3b Details for determining spatial distance and intensity distance are shown,

[0047] Figure 4 shows a comparative example of the method according to the present invention applied to an image,

[0048] Figure 5 FIG. 4 shows a comparative example of applying the method according to the present invention to a mask.

[0049] Figure 6 shows a device according to the invention, and

[0050] Figure 7 A device according to the invention is shown. DETAILED DESCRIPTION

[0051] refer to Figure 1, shows a schematic flow chart of a method according to the present invention. The method comprises:

[0052] In step S01 , an input image to be upscaled by bilateral interpolation is provided.

[0053] In step S02 , an initial image is provided.

[0054] In step S03 , a guidance image is provided.

[0055] In step S04, for each pixel position in the final image or the corresponding pixel position in the initial image, a region of interest (ROI) is selected around the corresponding pixel position in the input image. spat ) and determine the distance from the pixel position to the ROI in the input image spat The spatial distance d of each pixel within spat .

[0056] In step S05, for each pixel position in the initial image, a region of interest (ROI) is selected around the corresponding pixel position in the guide image. int ) and determine the distance from the pixel position to the ROI in the guidance image int The intensity of each pixel within distance d int .

[0057] In step S06, the ROI based on the spatial distance spat The weighted average of all pixels in the ROI and the intensity distance int The pixel value at the pixel location is determined based on the weighted average of all pixels in to generate the final image.

[0058] The order of the steps can be adjusted and is not limited to the order indicated above. Therefore, it will be understood by those skilled in the art that steps S01 to S03 can be performed in any order or simultaneously. Similarly, steps S04 and S05 can be performed in any order or simultaneously. In addition, the spatial distance d is calculated according to steps S01 and S04. spat The intensity distance d can be calculated independently of the intensity distance d calculated according to steps S02, S03 and S05. int to execute.

[0059] refer to Figure 2 An exemplary embodiment of the method shown in . Figure 2 A common use case is to upscale a mask from a low-resolution input mask. The low-resolution input mask is used as the input image to be upscaled using bilateral interpolation. Furthermore, a low-resolution monochrome guidance image is provided along with the high-resolution initial image. The algorithm outputs a high-resolution output mask, which serves as the final image. This output mask can be used for spatial AWB, for example.

[0060] The high-resolution initial image may have a size typically provided by a conventional camera sensor and may include 2 MP or more, preferably 5 MP or more, and more preferably 10 MP or more. The high-resolution initial image is converted into a monochrome initial image. Each pixel in the high-resolution initial image is converted from its RGB value to a monochrome pixel. The present invention is not limited to a particular type of conversion. For example, the conversion may be performed using the following formula:

[0061] M=w R ·R+w G ·G+w B B,

[0062] Where M is the output monochrome pixel value, and R, G, and B are the red, green, and blue channels of the incoming pixel. If a standard transformation from RGB to grayscale (brightness) is desired, then the weights w in the above formula are R 、w G and w B For R, G, and B channels, they should be 0.2126, 0.7152, and 0.0722, respectively. In particular, the same transformation should be used to generate a low-resolution monochrome guidance image.

[0063] To calculate the spatial distance d of a given pixel position in the high-resolution initial image spat , select the region of interest (ROI). Figure 3a In the example of , the ROI is provided by the nearest 4×4 pixels (16 pixels) from the low-resolution monochrome guidance image and the low-resolution input image. The spatial distance d spat is the Euclidean distance between the pixel position of the high-resolution initial image and the pixel space coordinate of the low-resolution input image. Figure 3a The indication in shows the spatial position of a pixel in the initial image and the center position of the pixel within the ROI provided by the 4×4 pixels around the pixel position in the input image. Therefore, for each pixel position in the example with an ROI of size 4×4 pixels, the spatial distance d is calculated spat 16 values.

[0064] Similarly, to calculate the intensity distance d int , calculate the ROI of the nearest 16 pixels in a 4×4 pattern around a given pixel location in the initial image. The intensity-based distance d is calculated using the absolute difference between the pixel intensity values ​​of the monochrome initial image and the intensity values ​​of the low-resolution guidance image. int Therefore, for a 4 × 4 sized ROI, 16 intensity distances d are also calculated. int . Then, d spat and d int By combining the following

[0065] dcomb =d spat w spat +d int ·(1-w spat ),

[0066] w spat is the interpolation weight. If w spat = 1.0, then interpolation uses only spatial position information to estimate the output pixel value. Therefore, the image content is omitted in the interpolation. spat = 0.0, then interpolation uses only intensity information to estimate the output pixel value. Therefore, no spatial information is used in interpolation.

[0067] From the combined distance d comb , through w pix =exp(-d comb *k) weights are calculated, where k is a configurable parameter. By changing k, more or less weight can be given to the center of the ROI. If different ROIs are used for intensity distance and spatial distance, then Calculate the individual weights and contributions, where k int and k spat is the scaling factor. In particular, k int =k spat .

[0068] In the last step, the final image is interpolated, where the low-resolution input image is provided and the pixel weights w are used pix The high-resolution output image is determined as the final image. Here, the weight w calculated for each pixel is calculated by the following formula pix Calculate the pixel value by taking a weighted average of the 16 pixels in the 4×4 ROI

[0069] HR pix =∑ pix∈ROI w pix *LR pix .

[0070] If the ROIs used to determine the spatial distance and the intensity distance are different, then the formula used to calculate the value in the final image is given by:

[0071]

[0072] Therefore, by providing a low-resolution monochrome guide image, a high-resolution final image interpolated from a low-resolution input image is generated. A magnification factor of 32x, 64x, and greater can be achieved. The algorithm is implemented as a pixel-in, pixel-out type algorithm, and therefore does not require any line buffer. Therefore, in the ISP hardware (HW) design, the HW cost of the algorithm can be significantly reduced. The bilateral interpolation of the present invention uses both position and intensity information of a given pixel. For bilateral interpolation of an RGB image, each color channel can be individually amplified by bilateral interpolation. Therefore, the method is applied, for example, to each of the RGB channels individually to amplify an RBG image.

[0073] This method according to the invention can be used for a variety of purposes. Here are some examples of use cases for the algorithm:

[0074] a. Local Tone Mapping (LTM), where the user can define regional tone mapping parameters as a low-resolution grid.

[0075] b. Image Quality Enhancement (IqE) algorithm localized style manipulation parameters need to be scaled up to the original image resolution.

[0076] c. Spatial automatic white balance (AWB), which requires estimating a low-resolution white balance gain map and upscaling it to full image resolution by obtaining edge information inside the image. Different types of lighting transitions need to be handled separately based on local statistical information.

[0077] d. Dense Optical Flow (DOF), where DOF data is estimated from low-resolution images (this is the typical way to run DOF algorithms in an ISP due to algorithmic complexity) and needs to be upscaled to the target resolution. DOF is often used for multi-frame algorithms such as noise reduction, super-resolution, and high dynamic range (HDR) processing.

[0078] e. Algorithms using semantic segmentation, such as face enhancement (e.g., face beautification, tone correction, face smoothing, etc.), bokeh algorithms, region and context manipulation (such as sky replacement) can require our algorithm to upscale a given low-resolution semantic segmentation image.

[0079] Amplification / interpolation can be combined with pre-processing or post-processing algorithms to further improve the amplification accuracy. These algorithms can be learning-based algorithms and / or traditional algorithms.

[0080] The algorithm in this invention is related to the upscaling algorithm. The proposed solution can be used alone or inside a digital image processing pipeline (also called ISP pipeline). In the case of mobile digital camera imaging, the use as part of the image processing pipeline is typical.

[0081] The main application of this algorithm is (but not limited to) upscaling mask images, which is an integral part of many image processing and computer vision algorithms. These applications include various applications such as bokeh effects, background scene changes, local tone mapping, AWB, etc., where preserving regional spatial information is crucial.

[0082] Furthermore, this method according to the invention is used in cases where an embedded device needs to process statistical and spatial information and the HW does not have efficient processing capabilities for higher resolutions. The processing is done for low resolution and the output is upscaled to the original resolution. Figure 5 A small demonstration is shown in , where a low-resolution input mask is successfully upscaled to the original image.

[0083] The lower-resolution input does not necessarily have to be a mask; it can be any 2D image, such as the Y channel of YUV or the output from a neural network. This adaptability removes obstacles, making the algorithm applicable to a variety of scenarios.

[0084] refer to Figure 4 , shows a comparative example, where Figure 4 The ground truth grayscale image of size 512×512 is provided in (a). Figure 4 (b) represents a low-resolution input grayscale image with a size of 16×16. Figure 4 (c) is the enlarged result by the nearest neighbor algorithm, which again has the size of 512×512 as the initial image. Figure 4 (d) is the amplified result by bilinear interpolation. Figure 4 (e) is the enlarged result through bicubic interpolation. Figure 4 (f) shows the enlarged result of the bilateral interpolation according to the present invention. As can be clearly seen, the objects within the original image are reconstructed with sufficient accuracy.

[0085] exist Figure 5 Another comparative example is provided in Figure 5 (a) shows an RGB image with a resolution of 2247×2663. Figure 5 (b) The initial RGB image is scaled down and the input mask is determined. Figure 5 (f) Determine the low-resolution guidance image from the downscaled initial RGB image. Figure 5 (c), 5(d), and 5(g) show comparative examples where the mask is enlarged by the nearest neighbor algorithm ( Figure 5 (c)), amplified by bilinear interpolation ( Figure 5 (d)) and upscaling by bicubic interpolation ( Figure 5 (g)). In addition, Figure 5(h) shows an enlarged low-resolution input mask, where the enlargement is performed by bilateral interpolation according to the present invention. Figure 5 (e) shows the direct Figure 5 (a) The ground truth mask calculated from the RGB high-resolution initial image. As can be clearly seen, small structures can be reconstructed with high reliability. Figure 5 The ground truth mask image of (e) requires processing the image data of the full initial image, but the low-resolution input mask can be computed faster based on a scaled-down version of the initial image, while reducing computational requirements.

[0086] Figure 6 The device 100 includes a processor 102 and a storage device 104. The storage device 104 stores instructions, and when the instructions are executed by the processor 102, the steps of the method according to the present invention are performed. The device 100 may be a user terminal and may include the following: Figure 7 , the camera module 106 is shown in FIG. The user terminal may be a smartphone, tablet, or the like. In particular, these devices have limited computing power. Therefore, it is advantageous to implement the bilateral interpolation method according to the present invention in order to quickly apply image processing operations to images, particularly in real time, using low-cost hardware.

[0087] In particular, the processor 102 may include more than one processor. Alternatively, the processor 102 may include an image signal processor (ISP) implemented in an ISP pipeline to handle, manipulate, and process image data provided by the camera 106.

Claims

1. A method for bilateral interpolation of an image, comprising: Provide the input image to be upscaled by bilateral interpolation; Provide an initial image; Provide guiding imagery; For each pixel position in the final image or the corresponding pixel position in the initial image, a region of interest (ROI) is selected around the corresponding pixel position in the input image. spat , and determine the distance from the pixel position to the ROI in the input image spat The spatial distance d of each pixel within spat ; For each pixel position in the initial image, a region of interest (ROI) is selected around the corresponding pixel position in the guide image. int , and determine the pixel position from the ROI in the guide image int The intensity of each pixel within distance d int ; The ROI based on the spatial distance spat The weighted average of all pixels in the ROI based on the intensity distance int The pixel value at the pixel position is determined based on the weighted average of all pixels in to generate the final image. The method of claim 1 , wherein the guide image is monochrome.

3. The method according to claim 1 or 2, wherein the initial image is transformed into a monochrome initial image before determining the intensity distance. The method according to claim 2 , wherein the transformation of the initial image and the generation of the guide image are performed by the same transformation.

5. The method according to claim 1 or 2, wherein the intensity distance is a difference in intensity values; and / or wherein the input image is derived from the initial image by downscaling and / or manipulation; and / or The intensity distance d is used to determine the int The ROI int Equal to the distance used to determine the spatial distance d spat The ROI spat and / or The ROI int and / or the ROI spat One of 2×2 pixels, 4×4 pixels, 8×8 pixels, 16×16 pixels, 32×32 pixels, or 64×64 pixels.

6. The method according to claim 1 or 2, wherein the spatial distance d spat and the intensity distance d int By combining the following formula: d comb =d spat ·w spat +d int ·(1-w spat ), where w spat are the interpolation weights.

7. The method according to claim 1 or 2, wherein the weighted average is obtained by multiplying the weighted average by the distance d. spat and / or the intensity distance d int and / or combined distance d comb exponentially decaying weight w pix Sure; Preferably, for each pixel position, the pixel value is determined by HR pix is the pixel value in the final image, and LR pix is the pixel value of the pixel in the input image.

8. The method according to claim 1 or 2, wherein the number of pixels of the final image is 16 times, preferably 32 times, more preferably 64 times or more the number of pixels of the input image.

9. A device comprising a processor and a memory, wherein the memory stores instructions that, when executed by the processor, perform the steps of the method according to any one of claims 1 to 8.

10. A software storage product storing instructions, which, when executed by a processor, perform the steps of the method according to any one of claims 1 to 8.

Images