An efficient interpolation method for image upsampling

Abstract

This invention provides an efficient interpolation method for image upsampling. The method comprises the following steps: Step 1, upscaling the original low-resolution image LR by a factor of k using conventional bicubic interpolation to obtain a high-resolution image HR; Step 2, sharpening the images before and after upscaling to obtain the corresponding edge images LR. eg and HR eg Then, the high-resolution edge image HR eg Downsample k times and LR from low-resolution edge images eg Step 3 involves subtracting the lost sharpening information from the image edges during upsampling to obtain the portion of sharpening information lost during the upsampling process. This lost information is then added to the original image's LR (Light Rendering) matrix at a certain ratio. Simultaneously, the offset of the image center point during scaling is considered, and the interpolated pixels are fine-tuned. Finally, the optimized bicubic interpolation is used to enlarge the image's LR matrix by a factor of k to obtain the final high-resolution image. This invention offers a significant performance improvement compared to traditional upsampling methods and possesses a certain degree of versatility.

Description

Technical Field

[0001] This invention relates to an efficient interpolation method for upsampling color images, belonging to the technical field of image processing. Background Technology

[0002] Image upsampling, which converts low-resolution (LR) images to high-resolution (HR) images, plays a crucial role in the field of graphics processing. Upsampled HR images offer more detail and a better view, and are widely used in digital high-definition television, medical imaging, and terminal devices. Currently, traditional interpolation methods include nearest-neighbor interpolation, bilinear interpolation, bicubic interpolation, and Lanczos interpolation.

[0003] Traditional interpolation methods mostly use a weighted average of neighboring pixels to obtain the value of the pixel to be interpolated. These methods are based on the assumption that the image has regional smoothness across all areas. However, this assumption does not hold true for edges and textures in the image, failing to fully consider local feature factors. Therefore, interpolation methods can damage high-frequency components of the image, causing blurring and edge distortion. Currently, most proposed edge-guided image interpolation methods improve the subjective quality of the interpolated image at the cost of higher complexity. Compared with classical interpolation methods, edge-guided HR images have sharper edges. However, they are often prone to causing artifacts in textures. In addition, the image center point will shift during image scaling, which will also lead to a non-uniform distribution of pixels after interpolation. Summary of the Invention

[0004] In order to preserve the details of the original image to the greatest extent, improve the similarity between the interpolated HR image and the original image, solve the blurring problem at the edges of the HR image and simplify the complexity, this invention provides an efficient interpolation method for image upsampling.

[0005] To achieve the above-mentioned objectives, the technical solution adopted by the method of the present invention is as follows:

[0006] An efficient interpolation method for image upsampling includes the following steps:

[0007] Step 1: Magnify the original low-resolution image LR by a factor of k using bicubic interpolation to obtain the high-resolution image HR;

[0008] Step 2: Sharpen the images before and after magnification to obtain the corresponding edge images (LR). eg and HR eg Then, the high-resolution edge image HR eg Downsampling k times and LR from low-resolution edge images egSubtracting from the middle yields the edge sharpening information L lost in bicubic interpolation. eg ;

[0009] Step 3, extract edge sharpening information L eg A certain proportion is added to the original low-resolution image LR to obtain the low-resolution image LR1 after the missing edge information is filled. Then, the image LR1 is enlarged using an optimized bicubic interpolation method to obtain the final high-resolution image. The optimized bicubic interpolation method is as follows: the geometric center point of the image that is generally skewed to the lower right after interpolation is adjusted to make it evenly distributed, and each pixel point after interpolation is adjusted by the weighted average ratio of the pixels in a 4*4 region before and after interpolation.

[0010] In this method, the image is processed by dividing it into three channel matrices: R, G, and B. Each of the three channels is then processed separately, and finally, the three processed channels are combined into one image.

[0011] Compared with the traditional bicubic interpolation method, the present invention has the following advantages:

[0012] (1) This invention proposes an efficient interpolation method for image upsampling, which not only preserves the detailed information of the original image, but also increases the similarity between the interpolated image and the original image, and has a significant performance improvement compared with traditional interpolation methods.

[0013] (2) The method of the present invention addresses the principle of traditional bicubic interpolation, which uses the weights of the 16 nearest points around a pixel in the original image to weight and superimpose the 16 pixels to obtain the corresponding pixel in the target image. It makes fine adjustments to the interpolated pixel so that the pixel value is closer to the original image.

[0014] (3) The method of the present invention uses sharpened convolution kernel to detect the edge information of the image, which solves the edge blurring problem in the traditional method. The edge image of HR contains certain texture details, and compared with other edge detection methods, it greatly reduces the computational complexity.

[0015] (4) The method of the present invention is applicable to almost all traditional interpolation methods. Only the fine-tuning coefficients need to be modified according to the principle of the corresponding method, which has strong versatility. Attached Figure Description

[0016] Figure 1 This is a flowchart of the image interpolation method of the present invention.

[0017] Figure 2 This is a pixel correspondence diagram of the bicubic interpolation method in this embodiment of the invention.

[0018] Figure 3 It is a graph of the bicubic interpolation weight function.

[0019] Figure 4 It is a geometric center mapping diagram.

[0020] Figure 5 This is a schematic diagram of the fine-tuning and improvement method of the present invention. Detailed Implementation

[0021] The technical solution of the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0022] This embodiment provides an efficient interpolation method for image upsampling. Figure 1 This is the interpolation process of this method.

[0023] In this embodiment, k is set to 4, meaning the LR image is magnified by a factor of 4. The specific implementation process of the method is as follows.

[0024] First, the original low-resolution image LR is enlarged by 4 times using the traditional bicubic interpolation method to obtain the high-resolution image HR.

[0025] The original image LR is of size m*n, and the image HR after being enlarged by 4 times is of size M*N, such as... Figure 2 As shown, since the number of pixels in image LR is known, each pixel HR(i,j) in image HR has a corresponding pixel LR(i,j). The 16 pixels in image LR closest to pixel LR(i,j) are used as parameters to calculate HR(i,j). The weights W of these 16 points are obtained using the bicubic weight function in equation (1), where a is usually taken as -0.5, and x1 is the distance from the point to the pixel to be calculated. The corresponding function graph is shown below. Figure 3 As shown, HR(i,j) can be obtained by weighting and summing them.

[0026]

[0027] Depend on Figure 2 It can be seen that the position coordinates of LR(i,j) will contain decimals. Let's assume its coordinates are LR(x+u,y+v), where x and y represent the integer parts, and u and v represent the decimal parts. The positions of the 16 nearest surrounding pixels are as follows: Figure 2 To ensure that the 16 nearest points can be found when the interpolation point is an edge point, the image to be interpolated (LR) matrix is ​​expanded by two rows and two columns at the beginning and end, for a total of four rows and four columns. The expanded pixels are the pixels at the edge of the image (LR).

[0028] Since pixels are two-dimensional, the rows and columns are calculated separately to obtain the horizontal and vertical coordinate weights A and C as shown in equations (2) and (3), and the corresponding coordinates B of the 16 pixels in equation (4). Finally, the calculation method for the pixel points is shown in equation (5).

[0029] A=[W(u+1)W(u)W(1-u)W(2-u)] (2)

[0030] C = [W(v+1)W(v)W(1-v)W(2-v)] T (3)

[0031]

[0032] HR(i,j)=A*B*C (5)

[0033] After obtaining the image HR using the traditional bicubic method, the images before and after magnification are sharpened according to equations (6) and (7) to obtain the corresponding edge images LR. eg and HR eg , where rh represents the sharpening convolution kernel, which is used to sharpen the image, making the image edges sharper. The image edges contain most of the image's detailed information, i.e., the high-frequency part of the image, which varies greatly. In this method, the convolution kernel is used as a high-pass filter, as shown in Equation (8). This convolution kernel mainly highlights the edge features of the image. In the convolution, a circle of zeros is padded around the image to obtain the difference between the center pixel and the surrounding pixels. The larger the calculated result, the greater the change in the pixels near that location in the image, and the higher the frequency.

[0034]

[0035]

[0036]

[0037] Next, the HR edge image is downsampled by a factor of 4 to make it equal in size to the LR edge image. The downsampling method can be either bicubic interpolation or direct pixel extraction to obtain the HR. eg1 The image edge information lost during bicubic interpolation is calculated using equation (9).

[0038] L eg =LR eg -HR eg1 (9)

[0039] According to equation (10), the lost edge sharpening information is added to the image LR. The value of α should theoretically be as small as possible, with a range of 0.01 to 0.1. After testing, it is taken as 0.04 in this invention. At this time, the image LR1 already contains a lot of information and the edges are sharper. The optimized bicubic interpolation method is used to amplify it by 4 times to obtain the final high-resolution image.

[0040] LR1=LR+α*L eg (10)

[0041] The optimized bicubic algorithm of this invention is as follows:

[0042] During interpolation, the origin (0,0) is usually chosen as the top left corner, such as... Figure 4 As shown, suppose the original 3*3 image is interpolated into a 5*5 image, where Δ is the corresponding coordinate point of the target image on the original image. The geometric center point of the original 3*3 image is (1,1), and the geometric center point of the target 5*5 image is (2,2). The corresponding point mapped to the original image is 2*3 / 5, i.e., (1.2,1.2), not (1,1). The interpolated pixels are obviously concentrated in the lower right of the image rather than evenly distributed throughout the image. To address this, the image coordinates can be calculated starting from 0.5 to achieve geometric center alignment. Since the computer stores images starting from coordinate 0, it is necessary to subtract 0.5 after calculating the target image point coordinates according to formula (11), i.e., (2+0.5)*3 / 5-0.5=1. With geometric center alignment, the pixel distribution is more uniform.

[0043] src=(dst+0.5) / k-0.5 (11)

[0044] Where src is the coordinate of the image to be interpolated, dst is the coordinate of the interpolated image, and k is the magnification factor.

[0045] Furthermore, the traditional bicubic interpolation method uses a weighted average of 16 neighboring pixels to obtain the value of the pixel to be determined. Based on this principle, the present invention makes a fine adjustment to each interpolated pixel according to equation (12):

[0046] HR1(i,j)=(LR1(i1,j1)*k1 / sum)*bic(i,j) (12)

[0047] like Figure 5 As shown, HR1(i,j) is the pixel after fine-tuning, LR1(i1,j1) is the pixel before interpolation, corresponding to a 4*4 region after interpolation, bic(i,j) is a pixel in the 4*4 region before fine-tuning, k1 is the fine-tuning coefficient. Since bicubic interpolation uses the 16 nearest points around a pixel to find the target image pixel, k1 is set to 16 here, and sum is the sum of the pixels of the 16 pixels around bic(i,j). Through this adjustment, a connection can be established between the interpolated pixel and the corresponding point in the original image, improving the similarity.

[0048] This embodiment improves the overall accuracy of the algorithm by analyzing the principles of classic interpolation algorithms, making the interpolated pixels closer to the original image pixels. The method reduces the complexity of conventional edge detection methods by performing sharpening detection on image edges. In terms of software implementation, compared to other complex edge detection methods, the time required for upsampling a single color image can be reduced by more than 50%. In terms of hardware implementation, the hardware implementation of bicubic interpolation, compared to methods such as Lanczos interpolation, significantly reduces hardware resource consumption due to the lower computational load in the weight function and sharpening convolution, and can be implemented in parallel across multiple paths. In terms of functionality, artifacts at image textures almost disappear after upsampling, and the blurring problem at edges is also improved.

[0049] Taking a 4K (3840*2160) image as an example, an image with a downsampled resolution of 960*540 is upscaled to 4K (3840*2160) using this method. The performance of the method is comprehensively evaluated using three metrics: Peak Signal-to-Noise Ratio (PSNR), Structural Similarity (SSIM), and Learned Perceptual Patch Similarity (LPIPS). After testing with multiple example images using the proposed technique at a 4x upsampling ratio, the average PSNR value reaches 34.18, the average SSIM value reaches 0.914, and the average LPIPS value reaches 0.188, meeting industry requirements. As shown in Table 1, using six color images as an example for testing, compared with the classic bilinear method, PSNR is improved by 4.29 and SSIM by 0.063; compared with the traditional bicubic method, PSNR is improved by 4.12 and SSIM by 0.054; compared with the Lanczos interpolation method, PSNR is improved by 1.86 and SSIM by 0.021. In addition, the LPIPS index is more in line with human perception, and the lower the value, the more similar the two images are.

[0050] For other classic interpolation algorithms, such as nearest neighbor interpolation, bilinear interpolation, and Lanczos interpolation, the edge detection and interpolation algorithm improvements in this invention are still applicable, and the performance is improved compared to many original interpolation methods.

[0051] Table 1 Performance comparison of different upsampling methods

[0052]

[0053] The above description is merely a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in the present invention should be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. An efficient interpolation method for image upsampling, characterized in that, Includes the following steps: Step 1: Magnify the original low-resolution image LR by a factor of k using bicubic interpolation to obtain the high-resolution image HR. Specifically: Given the values ​​of each pixel in the original low-resolution image LR, each pixel HR(i,j) in the high-resolution image HR has a corresponding pixel LR(i,j). Use the 16 pixels in the original low-resolution image LR that are closest to pixel LR(i,j) as parameters to calculate pixel HR(i,j). Use the bicubic weight function to calculate the weights of these 16 pixels, and then sum the weighted values ​​of the 16 pixels to obtain the interpolation point. Step 2: Sharpen the images before and after magnification to obtain the corresponding edge images. and Then, high-resolution edge images Downsample by k times and extract from low-resolution edge images Subtracting from the middle yields the edge sharpening information lost in bicubic interpolation. ; Step 3, extract edge sharpening information Add the image proportionally to the original low-resolution image in Lightroom (LR) to obtain the low-resolution image after restoring the lost edge information. Then, the image is enlarged using an optimized bicubic interpolation method. The final high-resolution image is obtained. The optimized bicubic interpolation method is as follows: the geometric center point of the image, which is generally located in the lower right corner after interpolation, is adjusted to make it evenly distributed. The weighted average ratio of pixels in a 4x4 region before and after interpolation is used to adjust each pixel after interpolation. Specifically, each pixel after interpolation is adjusted using the following formula: in, These are the interpolated pixels. These are the pixels before interpolation, corresponding to a 4x4 region after interpolation. To adjust a specific pixel in the previous 4x4 region, Take 16, It is a pixel. The sum of the pixels of the surrounding 16 pixels.

2. The efficient interpolation method for image upsampling according to claim 1, characterized in that, The coordinates of the 16 pixels are as follows: The horizontal and vertical weights of two-dimensional pixels are calculated using the following two formulas: Where x and y represent the integer parts of the horizontal and vertical coordinates of pixel LR(i,j), u and v represent the fractional parts of the horizontal and vertical coordinates of pixel LR(i,j), A is the weight of the horizontal coordinates of 16 pixels, and C is the weight of the vertical coordinates of 16 pixels.

3. The efficient interpolation method for image upsampling according to claim 2, characterized in that, In step 1, the edges of the image to be interpolated are expanded by two rows and two columns, for a total of four rows and four columns. Then, the final interpolation points are calculated using the following formula:

4. The efficient interpolation method for image upsampling according to claim 1, characterized in that, In step 2, the corresponding edge images are obtained by sharpening the images before and after magnification using the following two formulas. and : Where rh represents the sharpened convolution kernel.

5. The efficient interpolation method for image upsampling according to claim 4, characterized in that, The sharpened convolution kernel rh is:

6. The efficient interpolation method for image upsampling according to claim 1, characterized in that, In step 2, the high-resolution edge image is obtained using the following formula. From low-resolution edge images Subtract from: in, It is a high-resolution edge image Obtained by direct downsampling. It is the edge sharpening information lost during bicubic interpolation.

7. The efficient interpolation method for image upsampling according to claim 6, characterized in that, In step 3, the edge sharpening information is... Add the following formula to the original low-resolution image in Lightroom: in, The value range is 0.01 to 0.

1.

8. The efficient interpolation method for image upsampling according to claim 1, characterized in that, In step 3, the geometric center point of the image, which is shifted to the lower right after interpolation, is adjusted using the following formula: in dst represents the coordinates of the image to be interpolated, and dst represents the coordinates of the interpolated image. This is the magnification factor.

Images