Image up-sampling zoom display method

Abstract

The application discloses an image up-sampling and enlarging display method in the technical field of image processing, and comprises the following steps: S100, constructing a three-dimensional scene and importing an image; S200, rendering the scene in a small size to generate an initial frame buffer as the input of the next step; S300, performing time anti-aliasing on the input frame buffer to generate a new frame buffer after eliminating jaggies as the input of the next step; S400, performing fast resampling calculation based on a CAS algorithm on the input frame buffer to generate a large-size frame buffer as a display frame; and S500, sampling and outputting the input frame to a viewport. The application has the beneficial effects that the original image sampling up-sampling algorithm is optimized, the algorithm efficiency of the image up-sampling display is improved through multi-point sampling and optimized calculation, and the display effect of the image up-sampling is optimized.

Description

Technical Field

[0001] The present invention relates to the field of image processing, and in particular to an image upsampling, enlarging and displaying method. Background Art

[0002] With the development of immersive display technology, various large-scale displays have become increasingly popular. Multi-channel projection domes and large LED domes are increasingly used in exhibitions, flight training, and virtual reality. Ensuring the efficient, real-time display of three-dimensional scenes with excellent visual quality on large-scale displays has become a topic of great concern. This paper proposes an adaptive fast resampling algorithm based on the CAS algorithm, which can improve the display quality of scenes on projection domes while maintaining rendering frame rate. Downsampling is the process of reducing a high-resolution original image to accommodate smaller display areas on various devices, while upsampling is generally used to render images on large-scale displays. Many games employ image resampling techniques to ensure the desired image quality without sacrificing frame rate. NVIDIA has developed the DLSS algorithm based on its graphics card framework, but it relies on certain devices and has limited compatibility. Its proposed NIS algorithm is compatible with all platforms and devices, but uses the older prewitt operator and requires 36 sampling calculations, resulting in low efficiency and failing to meet our frame rate requirements. The CAS algorithm proposed by AMD can solve the current problem and has a certain degree of adaptability, but its only drawback is that it uses the most traditional bilinear interpolation for interpolation, and the effect cannot reach the ideal state of push to display. Its image display effect and sampling calculation efficiency are low. Therefore, this application proposes an image upsampling and magnification display method. Summary of the Invention

[0003] The purpose of the present invention is to provide an image upsampling and magnifying display method, which improves the algorithm efficiency of image upsampling display by optimizing the original image sampling upsampling algorithm, through multi-point sampling and optimized calculation, and at the same time, optimizes the display effect of image upsampling to solve the above-mentioned AMD proposed CAS algorithm. It can solve the current problem and has a certain degree of adaptability, but its only drawback is that its interpolation adopts the most traditional bilinear interpolation, and the effect cannot reach the ideal state of pushing to display, and its image display effect and sampling calculation efficiency are relatively low.

[0004] To achieve the above object, the present invention provides the following technical solution: the method can be implemented by a smart device with a display screen and a processor, comprising the following steps:

[0005] S100: Build a 3D scene and import images.

[0006] S200: render the scene in small size, generate an initial frame buffer as the input of the next step.

[0007] S300: perform temporal anti-aliasing on the input frame buffer, generate a new frame buffer after removing the jaggies as the input of the next step.

[0008] S400: perform fast resampling calculation based on CAS algorithm on the input frame buffer, generate a large size frame buffer as the display frame.

[0009] S500: sample and output the input frame to the viewport.

[0010] The step S100 constructs a convolution kernel based on the Laplace operator and imports the image, the step S200 replaces the bilinear interpolation with bicubic interpolation to provide more accurate pixel value completion, the step S300 optimizes the bicubic interpolation with multiple sampling, optimizes the optimized bicubic interpolation, and universally rewrites the bicubic interpolation to the form of bilinear difference, further optimizes the sampling times, and finally optimizes the original 16 sampling times to 5 sampling times, so as to obtain the weight value of the bilinear interpolation, the step S400 obtains the weight value meeting the condition of the step S300, and performs secondary optimization through bicubic resampling, and calculates the local display parameters, and finally convolves the parameters, and the step S500 outputs the values obtained in the step S400.

[0011] As a further scheme of the application, in the step S200, the bicubic interpolation is optimized to improve the running efficiency, the weight is calculated through a one-dimensional interpolation function, and the sampling value is determined, the sampling value is expanded to two dimensions, one cubic interpolation is performed in the x and y directions, and the results of the interpolation points are obtained by synthesizing the multiple sampling values and the corresponding weight values.

[0012] As a further scheme of the application, in the step S300, the original optimization formula is rewritten to obtain the universal rewriting of the numerical value, so as to obtain the cubic interpolation.

[0013] As a further scheme of the application, in the step S400, the pixel value is calculated according to the obtained cubic interpolation, so as to obtain the pixel value of the sampling point, and the sampling point is assigned with a weight value.

[0014] As a further scheme of the application, in the step S400, when the obtained weight value has a negative value, the negative value is excluded, the weight value meeting the condition is selected, and the sampling value is calculated according to the weight value.

[0015] Compared with the prior art, the application has the following advantages:

[0016] 1. In the application, the algorithm efficiency of image up-sampling display is improved by optimizing the up-sampling algorithm of the original image, through multi-point sampling and optimized calculation, and the display effect of image up-sampling is optimized. BRIEF DESCRIPTION OF DRAWINGS

[0017] Figure 1 It is an algorithm flow diagram of the application.

[0018] Figure 2 It is a CAS algorithm convolution kernel structure diagram in the application.

[0019] Figure 3 It is a pixel position diagram in the CAS algorithm in the application.

[0020] Figure 4 It is a small size rendering pixel position diagram in the application.

[0021] Figure 5 It is a bicubic interpolation sampling process diagram in the application.

[0022] Figure 6 It is a weight value size change trend diagram under different distances in the application.

[0023] Figure 7 It is a quadratic interpolation sampling diagram in the application.

[0024] Figure 8 It is a first contrast image diagram.

[0025] Figure 9 It is a first contrast image enlargement diagram.

[0026] Figure 10 It is a contrast image diagram without algorithm.

[0027] Figure 11 It is a contrast image enlargement diagram without algorithm.

[0028] Figure 12 It is a CAS algorithm contrast image diagram.

[0029] Figure 13 It is a first CAS algorithm contrast image enlargement diagram.

[0030] Figure 14 It is a second CAS algorithm contrast image enlargement diagram.

[0031] Figure 15 It is a contrast image diagram of the application algorithm.

[0032] Figure 16 It is a first contrast image enlargement diagram of the application algorithm.

[0033] Figure 17 Figure 2 is a second enlarged view of the contrast image according to the algorithm of the present application. DETAILED DESCRIPTION

[0034] The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by a person of ordinary skill in the art without creative work fall within the protection scope of the present application.

[0035] The CAS (Contrast Adaptive Sharpening) algorithm is a contrast adaptive sharpening algorithm based on Laplacian operator proposed by AMD Company. The algorithm mainly calculates the basic sharpening amount and the remapped sharpening parameter according to the local contrast of the pixel neighborhood, so as to adjust the convolution kernel to achieve the adaptive purpose.

[0036] The algorithm derivation and the construction of the convolution kernel are as follows Figure 2

[0037] The CAS algorithm is based on Laplacian operator sharpening, and the construction of the convolution kernel is as follows Figure 3

[0038] The calculation of the weight w is as follows:

[0039] ω = amp * peak

[0040] The amp is the basic sharpening amount, which is determined by the local contrast of the pixel, and the peak is the remapping of the sharpening parameter.

[0041] The local contrast is calculated by taking the minimum value mnR, mnG, mnB and the maximum value mxR, mxG, mxB of each color channel in the center pixel e and its four directly adjacent pixels.

[0042] LocalContrast = min((mn, 1.0-mx))

[0043]

[0044] The clamp function restricts the parameter between 0.0 and 1.0

[0045]

[0046] The remapping of the sharpening parameter is to map the value in the range [0.0, 1.0] to [-0.125, -0.2], and the calculation is as follows:

[0047]

[0048] When the small size rendering graphics display to large size window is needed to resample the texture, CAS algorithm in the bilinear interpolation and sharpening algorithm fusion, so that the use of a color to complete the graphics zoom and sharpening, its approach as Figure 4 .

[0049] Using adaptive contrast sharpening algorithm to calculate the center four pixels (f, g, j, k) no scaling sharpening results and bilinear interpolation to get the final results, first coordinate conversion, the output pixel corresponding to the source pixel position

[0050] y(s,t)=>x(u,v)

[0051] u f =u-u i

[0052] v f =v-v i

[0053] (s, t) is the source pixel position, (u, v) is the pixel position corresponding to the enlarged image, u i , v i is the integer part of the coordinate value, u f , v f is the decimal part of the coordinate value. The calculation of bilinear interpolation results as follows, the calculation of four weights is:

[0054]

[0055] At the same time in order to reduce the bilinear interpolation artifacts, the weight is processed

[0056]

[0057] Where c is a constant, generally take the value of 1 / 32, mx and mn are the calculated field pixel single color single channel minimum and maximum in the previous step, used to reflect its local contrast.

[0058] By multiplying and accumulating the bilinear interpolation weight and CAS sharpening convolution kernel, the final result can be obtained, and the final convolution kernel is as follows:

[0059]

[0060] In order to improve the clarity of display, this paper uses the bicubic interpolation which has good processing efficiency and output result combination to replace bilinear interpolation, provides more accurate pixel value completion, and optimizes the bicubic interpolation to improve its running efficiency, the algorithm is as follows:

[0061] One-dimensional cubic interpolation can be described as the following formula:

[0062] f(x,y) = w -1 f(x i-1 , y i ) + w i f(x i , y +1 ) + w i+1 f(x i , y +2 ) + w i+2 f(x i , y i )

[0063] where w l is the weight calculated by the interpolation function, and f(x,y) is the sampling value obtained at (x,y). Extending it to two dimensions, that is, performing cubic interpolation in both x and y directions, the process can be represented as the following formula:

[0064]

[0065] As the above formula shows, the bicubic sampling method needs to integrate 16 sampling values and their corresponding weight values to obtain the result of the interpolation point. Among them, the bicubic interpolation sampling process is as follows Figure 5 .

[0066] The Catmull-Rom function was originally used for the construction of graphics curves and surfaces, but it also performs well in texture resampling, and its function is as follows:

[0067]

[0068] The independent variable d is the Euclidean distance from the sampling point to the interpolation target point:

[0069]

[0070] Substituting the value of d into the formula can obtain the expression of each weight with respect to the distance:

[0071]

[0072] Using the function graph can more intuitively see the size and trend of the weight value at different distances, see Figure 6 .

[0073] An optimization idea of bicubic interpolation is to use hardware-accelerated bilinear sampling to optimize 16 bicubic interpolations to 4 bilinear interpolations, and the principle is as follows:

[0074] The one-dimensional linear interpolation calculation formula is:

[0075] f l = (1 - a)f(x i,y i )+αf(x i+1 ,y i )

[0076] The one-dimensional cubic interpolation formula is rewritten as follows:

[0077] f c =w -1 f(x i-1 ,y i )+w0f(x i ,y i )+w +1 f(x i+1 y i )+w +2 f(x i+2 ,y i )

[0078] where x i is the integer part of the coordinate x, x i+1 =x i +1, the vertical coordinate is the same, a is the decimal part of the x coordinate.

[0079] Then we can rewrite the two formulas in a universal way by making a series of changes to them - taking part of the cubic interpolation formula:

[0080] C=w +1 f(x i+1 ,y i )+w +2 f(x i+2 ,y i )

[0081] Let s(w +1 +w +2 )=1, we can get sw +1 =1-sw +2 , by observing that its form is the same as the weight of linear interpolation

[0082] sC=(1-sw +2 )f(x i+1 ,y i )+sw +2 f(x i+2 ,y i )=(1-t)f(x i ,y i )+tf(x i+1 ,y i )

[0083] We can get a portion of the pixel values ​​of the cubic interpolation to calculate C=s -1 f l , where s-1 =w +1 +w +2

[0084] This can be written as a universal formula as follows:

[0085]

[0086] in This means we can transform the linear interpolation of two samples into one sample The texture at the position is assigned a certain weight. In this way, the entire one-dimensional cubic interpolation formula can be simplified to the following form

[0087]

[0088] in

[0089] g0=w -1 +w0g1=w +1 +w +2

[0090]

[0091] However, we can see from the figure that since the first and last weights in the catmul l-rom curve are always negative, this does not meet the conditions for using this optimization method, that is,

[0092]

[0093] Although the first and last weights do not show the expected results, the weights in the middle can meet the conditions. Therefore, according to this idea, catmul l-rom bicubic resampling can be optimized to 9-sample bicubic interpolation, and only the two samples in the middle can be optimized to one hardware-accelerated linear sampling.

[0094]

[0095] According to this idea, catmul l-rom can only be optimized to 9 samplings. Compared with 16 samplings, the efficiency has also been improved to a certain extent. The formula for the unidirectional bicubic interpolation of 9 sampling catmul l-rom is

[0096]

[0097] On this basis, we further simplify it and take into account the convolution kernel shape of the CAS algorithm, that is, ignore the bilinear sampling results on the four corners, and perform 5 samplings for catmul l-rom interpolation.

[0098] The sampling position is summarized as

[0099]

[0100] Wherein the offset

[0101]

[0102] The weight value corresponding to each sampling point is

[0103]

[0104] Combined with the improved catmul l-rom bicubic interpolation.

[0105] Through the optimized CAS algorithm, certain rendering efficiency can be ensured, and the rendering effect is improved.

[0106] Please refer to Figure 8-9 , 1920*1080 display size of 1248*702 resolution rendering scene;

[0107] Please refer to Figure 10-11 , 1920*1080 display size of 1920*1080 resolution rendering scene without using any algorithm effect;

[0108] Please refer to Figure 12-14 , 1920*1080 display size of 1248*702 resolution rendering scene using original CAS algorithm effect;

[0109] Please refer to Figure 15-17 , 1920*1080 display size of 1248*702 resolution rendering scene using the algorithm effect.

[0110] Wherein, the efficiency comparison table is as follows:

[0111]

[0112]

[0113] To sum up, the algorithm flow of the present application is as follows: inputting an image, obtaining interpolation pixel coordinates and calculating the four neighborhood coordinates, respectively mapping to the original image and performing bicubic interpolation, taking the obtained result as the input of the CAS sharpening algorithm, calculating local contrast, sharpening parameters and the like, finally performing convolution, and outputting the result.

[0114] The above is only the preferred specific implementation of the present application, but the protection scope of the present application is not limited to this, any skilled person in the art can make equivalent replacement or change according to the technical scheme and the inventive concept of the present application within the technical range disclosed by the present application, which should be covered in the protection scope of the present application.

Claims

1. A method for upsampling and magnifying an image, which can be implemented by a smart device equipped with a display and a processor, and is characterized by: The steps include: S100: Build a 3D scene and import images; S200: Render the scene at a small size and generate an initial frame buffer as input for the next step: S300: performing temporal anti-aliasing on the input frame buffer to generate a new anti-aliased frame buffer as input for the next step; S400: Perform fast resampling calculation based on the CAS algorithm on the input frame buffer to generate a large-size frame buffer as a display frame: S500: sampling the input frame and outputting it to the viewport; The step S100 constructs a convolution kernel based on the Laplace operator and imports the image. The step S200 replaces the bilinear interpolation with bicubic interpolation to provide more accurate pixel value completion. The step S300 optimizes the multi-sampled bicubic interpolation through bicubic interpolation optimization and universally rewrites the optimized bicubic interpolation to the form of bilinear difference, that is, 16-times Catmull-ROM bicubic resampling is optimized to 9-sample bicubic interpolation. The single-direction formula of the 9-times Catmull-ROM bicubic interpolation is: in; w -1 、w +1 、w0、w +2 is the weight calculated by the difference function, and f(x, y) is the sample value obtained at xy; The number of sampling times is further optimized, the bilinear sampling results on the four corners are ignored, 5 samplings are performed for Catmull-ROM interpolation, and finally the original 16 samplings are finally optimized to 5 samplings, thereby obtaining the weights of bilinear interpolation. Step S400 obtains the weights that meet the conditions in step S300, performs secondary optimization through bicubic resampling, calculates local display parameters, and finally convolves the parameters. Step S500 outputs the values ​​obtained in step S400.

2. The image upsampling and magnifying display method according to claim 1, wherein: In step S200, bicubic interpolation is optimized to improve its operating efficiency. The weight is calculated by a one-dimensional interpolation function, and the sampling value is determined. The sampling value is extended to two dimensions, and a cubic interpolation is performed in both the x and y directions. The result of the interpolation point is obtained by combining multiple sampling values ​​and their corresponding weight values.

3. The image upsampling and magnifying display method according to claim 1, wherein: The step S400 calculates pixel values ​​based on the obtained cubic interpolation, thereby obtaining pixel values ​​of sampling points and assigning weights to the sampling points.

4. The image upsampling and magnifying display method according to claim 1, wherein: In step S400, when the obtained weights have negative values, the negative values ​​are excluded, and then weights that meet the conditions are selected, and the sampling values ​​are calculated according to the weights.

Images