Upsampling method, upsampling device, electronic device, and storage medium

Abstract

This application discloses an upsampling method, upsampling device, electronic device, and computer-readable storage medium based on multi-type edge detection. The upsampling method includes: performing edge detection on an input image using multiple different types of edge detection methods to obtain detection results; determining the weight value corresponding to each original pixel in the input image within the interpolation range based on the detection results; determining the interpolation method for interpolating the input image based on the detection results; and performing interpolation processing on the input image based on the interpolation method and the weight value corresponding to each original pixel to obtain a target image. The upsampling method, upsampling device, electronic device, and computer-readable storage medium based on multi-type edge detection of this application employ multiple different types of edge detection methods to perform edge detection on the input image, which can improve the accuracy of edge detection and reduce artifacts in the upsampled image.

Description

Technical Field

[0001] This application relates to the field of image processing technology, and in particular to an upsampling method, upsampling device, electronic device, and computer-readable storage medium based on multi-type edge detection. Background Technology

[0002] Image upsampling is widely used in various image processing procedures, such as digital zooming or enlarging an image to a specified size. However, current industry-standard upsampling methods, by increasing the number of pixels and the sampling frequency, can lead to the creation of artifacts in the upsampled image, such as jagged edges. Summary of the Invention

[0003] This application provides an upsampling method, upsampling device, electronic device, and computer-readable storage medium based on multi-type edge detection.

[0004] The upsampling method based on multi-type edge detection in this application is used to output a target image based on an input image. The upsampling method includes:

[0005] The input image is edge detected using various different edge detection methods to obtain detection results;

[0006] Based on the detection results, determine the weight value corresponding to each original pixel of the input image within the interpolation range;

[0007] Based on the detection results, an interpolation method is determined for interpolating the input image.

[0008] The target image is obtained by interpolating the input image according to the interpolation method and the weight value corresponding to each original pixel.

[0009] The upsampling device based on multi-type edge detection in this application is used to output a target image based on an input image. The upsampling device includes:

[0010] The edge detection module is used to perform edge detection on the input image using various different types of edge detection methods to obtain detection results;

[0011] The weight determination module is used to determine the weight value corresponding to each original pixel of the input image within the interpolation range based on the detection result;

[0012] An interpolation selection module is used to determine an interpolation method for interpolating the input image based on the detection results.

[0013] An interpolation processing module is used to perform interpolation processing on the input image according to the interpolation method and the weight value corresponding to each original pixel to obtain the target image.

[0014] The electronic device according to the embodiments of this application includes one or more processors and a memory, wherein the memory stores a computer program, and when the computer program is executed by the processor, it implements the upsampling method based on multi-type edge detection according to the embodiments of this application.

[0015] The computer-readable storage medium of the present application embodiment stores a computer program thereon, which, when executed by a processor, implements the upsampling method based on multi-type edge detection of the present application embodiment.

[0016] The upsampling method, upsampling device, electronic device, and computer-readable storage medium based on multi-type edge detection in this application employ multiple different types of edge detection methods to perform edge detection on the input image, which can improve the accuracy of edge detection and reduce artifacts in the upsampled image.

[0017] Additional aspects and advantages of embodiments of this application will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of embodiments of this application. Attached Figure Description

[0018] The above and / or additional aspects and advantages of this application will become apparent and readily understood from the description of the embodiments taken in conjunction with the following drawings, wherein:

[0019] Figure 1 This is a flowchart illustrating an upsampling method based on multi-type edge detection according to certain embodiments of this application;

[0020] Figure 2 This is a schematic diagram of a module of an upsampling device based on a variable filter kernel size according to certain embodiments of this application;

[0021] Figure 3 This is a schematic diagram illustrating the principle of an upsampling method based on multi-type edge detection in certain embodiments of this application;

[0022] Figure 4 This is a flowchart illustrating an upsampling method based on multi-type edge detection according to certain embodiments of this application;

[0023] Figure 5 This is a flowchart illustrating an upsampling method based on multi-type edge detection according to certain embodiments of this application;

[0024] Figure 6This is a schematic diagram illustrating the principle of an upsampling method based on multi-type edge detection in certain embodiments of this application;

[0025] Figure 7 This is a schematic diagram illustrating the principle of an upsampling method based on multi-type edge detection in certain embodiments of this application;

[0026] Figure 8 This is a flowchart illustrating an upsampling method based on multi-type edge detection according to certain embodiments of this application;

[0027] Figure 9 This is a schematic diagram illustrating the principle of an upsampling method based on multi-type edge detection in certain embodiments of this application;

[0028] Figure 10 This is a schematic diagram illustrating the principle of an upsampling method based on multi-type edge detection in certain embodiments of this application;

[0029] Figure 11 This is a schematic diagram illustrating the principle of an upsampling method based on multi-type edge detection in certain embodiments of this application;

[0030] Figure 12 This is a flowchart illustrating an upsampling method based on multi-type edge detection according to certain embodiments of this application;

[0031] Figure 13 This is a schematic diagram illustrating the principle of an upsampling method based on multi-type edge detection in certain embodiments of this application;

[0032] Figure 14 This is a schematic diagram illustrating the principle of an upsampling method based on multi-type edge detection in certain embodiments of this application;

[0033] Figure 15 This is a flowchart illustrating an upsampling method based on multi-type edge detection according to certain embodiments of this application;

[0034] Figure 16 This is a schematic diagram illustrating the principle of an upsampling method based on multi-type edge detection in certain embodiments of this application;

[0035] Figure 17 This is a schematic diagram illustrating the principle of an upsampling method based on multi-type edge detection in certain embodiments of this application;

[0036] Figure 18 This is a flowchart illustrating an upsampling method based on multi-type edge detection according to certain embodiments of this application;

[0037] Figure 19 This is a flowchart illustrating an upsampling method based on multi-type edge detection according to certain embodiments of this application;

[0038] Figure 20 This is a schematic diagram illustrating the principle of an upsampling method based on multi-type edge detection in certain embodiments of this application;

[0039] Figure 21 This is a flowchart illustrating an upsampling method based on multi-type edge detection according to certain embodiments of this application;

[0040] Figure 22 This is a flowchart illustrating an upsampling method based on multi-type edge detection according to certain embodiments of this application;

[0041] Figure 23 This is a flowchart illustrating an upsampling method based on multi-type edge detection according to certain embodiments of this application;

[0042] Figure 24 This is a flowchart illustrating an upsampling method based on multi-type edge detection according to certain embodiments of this application;

[0043] Figure 25 This is a schematic diagram illustrating the principle of an upsampling method based on multi-type edge detection in certain embodiments of this application;

[0044] Figure 26 This is a flowchart illustrating an upsampling method based on multi-type edge detection according to certain embodiments of this application;

[0045] Figure 27 This is a flowchart illustrating an upsampling method based on multi-type edge detection according to certain embodiments of this application;

[0046] Figure 28 This is a schematic diagram illustrating the effect of an upsampling method based on multi-type edge detection in certain embodiments of this application;

[0047] Figure 29 This is a schematic diagram illustrating the effect of an upsampling method based on multi-type edge detection in certain embodiments of this application;

[0048] Figure 30 This is a schematic diagram illustrating the effect of an upsampling method based on multi-type edge detection in certain embodiments of this application;

[0049] Figure 31 This is a schematic diagram of the structure of an electronic device according to certain embodiments of this application;

[0050] Figure 32 This is a schematic diagram illustrating the connection state between a computer-readable storage medium and a processor according to certain embodiments of this application. Detailed Implementation

[0051] The embodiments of this application will be further described below with reference to the accompanying drawings. The same or similar reference numerals in the drawings denote the same or similar elements or elements having the same or similar functions throughout. Furthermore, the embodiments of this application described below with reference to the accompanying drawings are exemplary and are only used to explain the embodiments of this application, and should not be construed as limiting this application.

[0052] Please see Figure 1 This application provides an upsampling method based on multi-type edge detection. The upsampling method is used to output a target image based on an input image. The upsampling method includes:

[0053] 01: Employ various edge detection methods to perform edge detection on the input image to obtain detection results;

[0054] 02: Determine the weight value corresponding to each original pixel in the input image within the interpolation range based on the detection results;

[0055] 03: Determine the interpolation method used for interpolating the input image based on the detection results;

[0056] 04: The input image is interpolated according to the interpolation method and the weight value corresponding to each original pixel to obtain the target image.

[0057] Please see Figure 2 This application also provides an upsampling device 100 based on multi-type edge detection. The upsampling device 100 is used to output a target image based on an input image. The upsampling device 100 includes an edge detection module 10, a weight determination module 20, an interpolation selection module 30, and an interpolation processing module 40. The upsampling method based on multi-type edge detection in this application can be implemented by the upsampling device 100 based on multi-type edge detection in this application. Specifically, the edge detection module 10, weight determination module 20, interpolation selection module 30, and interpolation processing module 40 can be used to implement the methods in 01, 02, 03, and 04, respectively. That is, the edge detection module 10 can be used to perform edge detection on the input image using multiple different types of edge detection methods to obtain detection results. The weight determination module 20 can be used to determine the weight value corresponding to each original pixel in the interpolation range of the input image based on the detection results. The interpolation selection module 30 can be used to determine the interpolation method used for interpolating the input image based on the detection results. The interpolation processing module 40 can be used to interpolate the input image according to the interpolation method and the weight value corresponding to each original pixel to obtain the target image.

[0058] The upsampling method and upsampling device 100 based on multi-type edge detection in this application employ multiple different types of edge detection methods to perform edge detection on the input image, which can improve the accuracy of edge detection and reduce artifacts in the upsampled image.

[0059] Specifically, the upsampling method of this application is used to upsample an input image to obtain a target image. Image upsampling can be understood as enlarging an image, also known as upsampling. The main purposes of image upsampling include: performing digital zoom, enlarging the image to a specified size, and enabling the image to be displayed on a higher resolution display device.

[0060] Please see Figure 3 Assume the input image is A and the target image is B. The size of input image A is m*n, and the size of target image B is M*N. Then, the scaling ratio K between target image B and input image A is K = M / m = N / n. When obtaining target image B from input image A, the size of input image A is known, and at least one parameter, either the size of target image B or the scaling ratio K, is known. The other parameter can be calculated. For example, the user can manually set the desired size of target image B, or the system can automatically set a suitable size. Then, according to K = M / m = N / n, the scaling ratio K can be calculated. Similarly, the user can manually set the desired scaling ratio K, or the system can automatically set a suitable scaling ratio K. Then, according to K = M / m = N / n, the size of target image B can be calculated. In this embodiment, the target image is obtained by upsampling the input image, which is equivalent to interpolating the input image. For example, when obtaining an M*N target image B from an m*n input image A, one or more pixels can be inserted between adjacent pixels in the input image A using an interpolation algorithm, increasing the number of pixels from m*n to M*N, thereby obtaining the target image B.

[0061] In this embodiment, the edge detection module 10 employs various edge detection methods to perform edge detection on the input image, achieving high accuracy. These methods may include, for example, the Roberts edge detection operator, the Sobel edge detection operator, the Prewitt edge detection operator, the Canny edge detection operator, and the Gaussian-Laplace (LOG) edge detection operator. Several different edge detection methods will be described in detail later. After edge detection, the edge detection module 10 obtains detection results, which may include edge type, image gradient, etc. The weight determination module 20 and the interpolation selection module 30 determine the weight value and interpolation method corresponding to each original pixel within the interpolation range based on these detection results. Specifically, the weight determination module 20 determines the weight value corresponding to each original pixel within the interpolation range based on the detection results, and the interpolation selection module 30 determines the interpolation method based on the detection results; the order of these determinations is not restricted. For example, the weight determination module 20 can execute 02 first, followed by the interpolation selection module 30 executing 03; or, the interpolation selection module 30 can execute 03 first, followed by the weight determination module 20 executing 02; or, the weight determination module 20 can execute 02, while the interpolation selection module 30 executes 03 simultaneously. Finally, the interpolation processing module 40 performs interpolation processing on the input image according to the interpolation method and the weight value corresponding to each original pixel to obtain the target image. Because the edge detection accuracy is high, the weight values ​​and interpolation methods determined based on the detection results are more reasonable, resulting in fewer artifacts in the obtained target image, and reducing the likelihood of jagged edges or blurry images after magnification.

[0062] Please see Figure 4 In some implementations, various edge detection methods are used to perform edge detection on the input image to obtain the detection result (i.e., 01), including:

[0063] 011: The first edge detection method is used to perform edge detection on the input image to obtain the edge type;

[0064] 012: The second edge detection method is used to perform edge detection on the input image to obtain the first image gradient;

[0065] 013: Use a third edge detection method to perform edge detection on the input image to obtain the second image gradient;

[0066] Based on the detection results, determine the weight value (i.e., 02) corresponding to each original pixel in the input image within the interpolation range, including:

[0067] 021: Determine the weight value corresponding to each original pixel based on the gradient of the first image and the gradient of the second image;

[0068] Based on the detection results, the interpolation method (i.e., 03) used for interpolation processing of the input image is determined, including:

[0069] 031: Determine the interpolation method based on the edge type.

[0070] Please see Figure 2 In some embodiments, the edge detection module 10 can be used to implement the methods in 011, 012, and 013, the weight determination module 20 can be used to implement the method in 021, and the interpolation selection module 30 can be used to implement the method in 031. That is, the edge detection module 10 can be used to: perform edge detection on the input image using a first edge detection method to obtain the edge type; perform edge detection on the input image using a second edge detection method to obtain a first image gradient; and perform edge detection on the input image using a third edge detection method to obtain a second image gradient. The weight determination module 20 can be used to determine the weight value corresponding to each original pixel based on the first image gradient and the second image gradient. The interpolation selection module 30 can be used to determine the interpolation method based on the edge type.

[0071] Specifically, the first, second, and third edge detection methods are all different. These methods can be selected from the aforementioned Roberts edge detection operator, Sobel edge detection operator, Praveit edge detection operator, Canney edge detection operator, Gaussian-Laplacian edge detection operator, or other edge detection methods. When performing edge detection on the input image, the detection ranges selected by the first, second, and third edge detection methods can be the same or different. For example, in the following text, the first edge detection method selects a first local region as the detection range, the second edge detection method also selects a first local region, and the third edge detection method selects a second local region. The first edge detection method, the second edge detection method, and the third edge detection method select detection ranges for the input image that at least partially overlap, so that the weight values ​​and interpolation methods can be determined based on the various detection results, and the weight values ​​and interpolation methods can be combined for interpolation processing.

[0072] Please see Figure 5 In some implementations, the target image includes multiple target pixels. The corresponding position of the target pixel in the input image is the center pixel. Edge detection is performed on the input image using a first edge detection method to obtain the edge type (i.e., 011), including:

[0073] 0111: Determine a first local region within a first predetermined range surrounding the center pixel;

[0074] 0112: Calculate the current gradient magnitude of the first local region;

[0075] 0113: Calculate the current edge direction of the first local region;

[0076] 0114: Determine the edge type based on the current gradient magnitude and the current edge direction.

[0077] Please see Figure 2 In some embodiments, the target image includes multiple target pixels. The corresponding position of the target pixel in the input image is the center pixel. The edge detection module 10 can be used to implement the methods in 0111, 0112, 0113, and 0114. That is, the edge detection module 10 can be used to: determine a first local region within a first predetermined range around the center pixel; calculate the current gradient magnitude of the first local region; calculate the current edge direction of the first local region; and determine the edge type based on the current gradient magnitude and the current edge direction.

[0078] Specifically, in this embodiment, edge detection is performed on a first local region to obtain the edge type. Figure 6 For example, input image A includes 8*8 raw pixels (i.e., m=8, n=8, for illustrative purposes only; in reality, the number of raw pixels is much larger), and target image B includes 64*64 target pixels (i.e., M=64, N=64, for illustrative purposes only, due to limitations in the attached image). Figure 6 The target image B is not drawn; the arrangement of the target pixels can be referenced from the original pixels. The pixel value of each original pixel in the input image A is known, while the pixel value of the target pixel in the target image B is unknown. Therefore, to find the pixel value of each target pixel in the target image B, we need to first find the corresponding position of the target pixel in the input image A. Here, the pixel corresponding to the target pixel in the input image is called the center pixel. Finding the corresponding position of the target pixel in the input image A is equivalent to finding the position of the center pixel. Figure 6 For example, the target pixel B11 corresponds to the center pixel C in the input image A. Figure 6 The position of the center pixel C (shown for illustrative purposes only) is the position of the target pixel B11 in the target image B within the input image A. The position of the center pixel C can be determined based on the position of the target pixel in the target image (represented by rows and columns) and the scaling factor K between the target image B and the input image A. For example... Figure 6In this example, the scaling factor K = 64 / 8 = 8. Therefore, the position of the center pixel C corresponding to the target pixel B11 can be the reciprocal of the scaling factor K (1 / 8), which is the position of the 1 / 8th row and 1 / 8th column in the input image A. Similarly, the position of the center pixel corresponding to each target pixel in the target image in the input image can be obtained. Of course, other methods can be used to determine the position of the center pixel in other examples, and this is not a limitation here.

[0079] Please see Figure 7 In this embodiment, a first local region within a predetermined range surrounding the center pixel is first defined. For example, the first predetermined range can be 3*3, 4*4, 5*5, 8*8, etc., and is not limited here. The first predetermined range can be set based on empirical values. Correspondingly, the first local region is a 3*3 region covering the center pixel, or a 4*4 region covering the center pixel, or a 5*5 region covering the center pixel, or an 8*8 region covering the center pixel, etc. Figure 7 The first local region is an 8x8 area covering the center pixel. Then, the current gradient magnitude and current edge direction of the first local region are calculated separately. The order of calculating the current gradient magnitude and current edge direction is not restricted; the current gradient magnitude can be calculated first, then the current edge direction; or the current edge direction can be calculated first, then the current gradient magnitude; or both can be calculated simultaneously. Finally, the edge type is determined based on the current gradient magnitude and current edge direction. It can be understood that the current gradient magnitude and current edge direction of the first local region reflect the edge type of the first local region. For example, if the first local region has an edge type with a certain edge direction (such as 45 degrees, 90 degrees, 135 degrees), an appropriate interpolation method can be used to ensure that the interpolated target image has better smoothness and reduces jagged edges.

[0080] It is understandable that since the center pixel positions corresponding to different target pixels are different, the first local region defined by the center pixel is also different. Thus, the edge type is independently determined for the first local region corresponding to each center pixel. Each center pixel can use an appropriate interpolation method to obtain the pixel value, instead of using a uniform interpolation method for all center pixels. The interpolation method can be adaptively selected according to the edge type, resulting in fewer artifacts in the target image.

[0081] Please see Figure 8 In some implementations, the first local region includes multiple local sub-regions. Calculating the current gradient magnitude (i.e., 0112) of the first local region includes:

[0082] 01121: Calculate the horizontal gradient magnitude and vertical gradient magnitude of each local sub-region based on the preset image gradient algorithm and the pixel values ​​of multiple original pixels in each local sub-region;

[0083] 01122: Determine the sub-region gradient magnitude of a local sub-region based on the horizontal gradient magnitude and the vertical gradient magnitude of the sub-region, and use the sub-region gradient magnitudes of multiple local sub-regions as the current gradient magnitude;

[0084] Calculate the current edge direction (i.e., 0113) of the first local region, including:

[0085] 01131: Determine the sub-region edge direction of a local sub-region based on the horizontal gradient magnitude and the vertical gradient magnitude of the sub-region, and use the sub-region edge directions of multiple local sub-regions as the current edge direction;

[0086] The edge type (i.e., 0114) is determined based on the current gradient magnitude and the current edge direction, including:

[0087] 01141: Determine the edge type based on the sub-region gradient magnitude and sub-region edge direction of multiple local sub-regions.

[0088] Please see Figure 2 In some embodiments, the first local region includes multiple local sub-regions. The edge detection module 10 can be used to implement the methods in 01121, 01122, 01131, and 01141. That is, the edge detection module 10 can be used to: calculate the sub-region horizontal gradient magnitude and sub-region vertical gradient magnitude of each local sub-region according to a preset image gradient algorithm and the pixel values ​​of multiple original pixels in each local sub-region; determine the sub-region gradient magnitude of the local sub-region according to the sub-region horizontal gradient magnitude and sub-region vertical gradient magnitude, and use the sub-region gradient magnitudes of multiple local sub-regions as the current gradient magnitude; determine the sub-region edge direction of the local sub-region according to the sub-region horizontal gradient magnitude and sub-region vertical gradient magnitude, and use the sub-region edge direction of multiple local sub-regions as the current edge direction; and determine the edge type according to the sub-region gradient magnitudes and sub-region edge directions of multiple local sub-regions.

[0089] Specifically, the preset image gradient algorithm can be the Sobel operator. The Sobel operator consists of two sets of 3*3 matrices, a horizontal matrix and a vertical matrix, as shown below:

[0090]

[0091] By performing planar convolutions of the horizontal and vertical matrices with the pixel values ​​of multiple original pixels in each local sub-region, the horizontal gradient magnitude Gx and vertical gradient magnitude Gy of each local sub-region can be obtained. Then, the sub-region gradient magnitude Gmag can be calculated based on the horizontal gradient magnitude Gx and vertical gradient magnitude Gy. Alternatively, it can be simplified to Gmag = |Gx| + |Gy|. The sub-region edge direction Gdir can be determined based on the sub-region horizontal gradient magnitude Gx11 and the sub-region vertical gradient magnitude Gy11, where Gdir = arctan2(Gx, Gy). Furthermore, the edge type can be determined based on the sub-region gradient magnitude Gmag and the sub-region edge direction Gdir of multiple local sub-regions.

[0092] like Figure 9 As shown, taking an 8x8 region with the first local region as the center pixel as an example, the original pixels within the first local region are A11, A12, A13...A18, A21, A22, A23...A86, A87, A88. The first local region includes multiple local sub-regions. Each local sub-region is a 3x3 region, and the multiple local sub-regions overlap with each other. For example, the local sub-region in the upper left corner of the first local region includes 9 original pixels: A11, A12, A13, A21, A22, A23, A31, A32, A33. Using this local sub-region as a reference, and translating it sequentially in the horizontal direction and in the vertical direction with a step size of 1, we can obtain 6x6 local sub-regions. For each local sub-region, the horizontal gradient magnitude Gx and the vertical gradient magnitude Gy can be calculated. Then, based on these values, the sub-region gradient magnitude Gmag and the sub-region edge direction Gdir can be calculated. Thus, 6*6 sub-region gradient magnitudes Gmag can be obtained for each of the 6*6 local sub-regions (e.g., ...). Figure 10 As shown), and the 6*6 sub-region edge directions Gdir corresponding to the 6*6 local sub-regions (as shown). Figure 11 As shown in the figure, the edge type can be determined based on the gradient magnitude Gmag of the 6*6 sub-regions and the edge direction Gdir of the 6*6 sub-regions.

[0093] Taking the upper left sub-region of the first local region as an example, the calculation methods for the horizontal gradient magnitude Gx11 and the vertical gradient magnitude Gy11 of the sub-region are as follows:

[0094]

[0095]

[0096] That is, Gx11 = (A11-A13) + 2*(A21-A23) + (A31-A33), Gy11 = (A11-A31) + 2*(A12-A32) + (A13-A33). Further, the sub-region gradient magnitude Gmag11 = |(A11-A13) + 2*(A21-A23) + (A31-A33)| + |(A11-A31) + 2*(A12-A32) + (A13-A33)|.

[0097] Similarly, the gradient magnitudes of 6*6 sub-regions can be calculated, namely Gmag11, Gmag12, Gmag13...Gmag16, Gmag21, Gmag22, Gmag23...Gmag64, Gmag65, Gmag66. These 6*6 sub-region gradient magnitudes are used as the current gradient magnitudes of the first local region.

[0098] Taking the upper left sub-region of the first local region as an example, the edge direction Gdir11 of the sub-region can be determined based on the horizontal gradient magnitude Gx11 and the vertical gradient magnitude Gy11 of the sub-region, as shown below:

[0099] Gdir11 = arctan2(Gx11, Gy11)

[0100] Similarly, the edge directions of 6*6 sub-regions can be determined, namely Gdir11, Gdir12, Gdir13...Gdir16, Gdir21, Gdir22, Gdir23...Gdir64, Gdir65, Gdir66. These 6*6 sub-region edge directions serve as the current edge directions of the first local region. There is a one-to-one correspondence between the gradient magnitude Gmag of these 6*6 sub-regions and the edge directions Gdir of these 6*6 sub-regions. Based on the gradient magnitude Gmag and the edge directions Gdir of these 6*6 sub-regions, the edge type of the first local region can be determined.

[0101] It is understood that, in the embodiments of this application, the preset image gradient algorithm may use other operators besides the Sobel operator, such as the Laplacian operator, the Prewitt operator, the Roberts operator, the Canny operator, etc., and no limitation is made here.

[0102] Please see Figure 12 In some implementations, the edge type (i.e., 01141) is determined based on the sub-region gradient magnitude and sub-region edge direction of multiple local sub-regions, including:

[0103] 011411: Compare the gradient magnitude of the sub-region with the gradient magnitude threshold;

[0104] 011412: When the gradient magnitude of a sub-region is greater than the gradient magnitude threshold, the local sub-region corresponding to the gradient magnitude of the sub-region is taken as an edge pixel;

[0105] 011413: Determine the edge region and / or extended region to which the edge direction of the sub-region corresponding to the edge pixel belongs;

[0106] 011414: Count the number of edge pixels in each edge region and / or extended region to obtain the edge region and / or extended region with the most edge pixels;

[0107] 011415: Determine the percentage of pixels in the edge region and / or extended region with the largest number of edge pixels;

[0108] 011416: When the pixel ratio of the edge region is greater than the first ratio threshold and / or the pixel ratio of the extended region is greater than the second ratio threshold, the edge type is determined to be the edge direction type corresponding to the edge region and / or the extended region.

[0109] Please see Figure 2 In some embodiments, the edge detection module 10 can be used to implement the methods in 011411, 011412, 011413, 011414, 011415, and 011416. That is, the edge detection module 10 can be used to: compare the gradient magnitude of a sub-region with a gradient magnitude threshold; when the gradient magnitude of a sub-region is greater than the gradient magnitude threshold, take the local sub-region corresponding to the gradient magnitude of the sub-region as an edge pixel; determine the edge region and / or extended region to which the edge direction of the sub-region corresponding to the edge pixel belongs; count the number of edge pixels in each edge region and / or extended region to obtain the edge region and / or extended region with the most edge pixels; determine the pixel ratio of the edge region and / or extended region with the most edge pixels; when the pixel ratio of the edge region is greater than a first ratio threshold and / or the pixel ratio of the extended region is greater than a second ratio threshold, determine that the edge type is the edge direction type corresponding to the edge region and / or extended region.

[0110] by Figure 10 and Figure 11 For example, the gradient magnitude Gmag of each of the 6*6 local sub-regions is compared with the gradient magnitude threshold egde thres. If the gradient magnitude Gmag of a certain sub-region is greater than the gradient magnitude threshold egde thres, then the local sub-region corresponding to the gradient magnitude Gmag is regarded as an edge pixel. Assuming that there are 12 edge pixels, the edge region and / or extended region to which the edge direction Gdir of the sub-regions corresponding to these 12 edge pixels belongs are determined.

[0111] The edge region and extended region are divided as follows: First, the image space of 0°-360° is transformed into a regional space of 0°-180°, and then the regional space of 0°-180° is divided into multiple regions. For example... Figure 13 As shown, in the first partitioning method, adjacent regions do not overlap, and each region serves as an edge region. Figure 13 In the example, the spatial area from 0° to 180° is divided into 8 regions, each with an angle range of 22.5°. For example... Figure 14 As shown, in the second partitioning method, adjacent regions partially overlap, and each region serves as an extended region. For example... Figure 14 In the example, the 0°-180° area is divided into 8 regions, each with an angle range of 40°. Compared to the edge regions, the extended regions have a larger area, meaning a wider angle range. The extended regions cover and extend beyond the edge regions, thus allowing for the collection of more edge pixels.

[0112] When determining the edge region and / or extended region to which the edge direction Gdir of the sub-regions corresponding to the above 12 edge pixels belong, each region can be determined individually and its value accumulated in a counter. For example, if the edge direction Gdir of a sub-region corresponding to a certain edge pixel is 15 degrees, it is counted in the first edge region (0–22.5 degrees); if the edge direction Gdir of a sub-region corresponding to a certain edge pixel is 40 degrees, it is counted in the second edge region (22.5–45 degrees). This process continues until the edge regions to which the edge directions Gdir of the sub-regions corresponding to these 12 edge pixels belong are determined. Similarly, the extended regions to which the edge directions Gdir of the sub-regions corresponding to these 12 edge pixels belong can also be determined. It is understood that since there is overlap between adjacent extended regions, there may be cases where the edge direction Gdir of the same sub-region corresponding to an edge pixel belongs to two extended regions simultaneously. In this case, both extended regions are counted simultaneously.

[0113] Then, the number of edge pixels in each edge region and / or extended region is counted to identify the edge region and / or extended region with the most edge pixels, and the pixel percentage of the edge region and / or extended region with the most edge pixels is determined. For example, if the first edge region (0–22.5 degrees) contains the most edge pixels (7), then comparing the number of edge pixels (5) with the total number of pixels (6*6) gives the pixel percentage of the edge region with the most edge pixels as 7 / 36. Similarly, the pixel percentage of the extended region with the most edge pixels can also be obtained.

[0114] Finally, the pixel percentage of the edge region with the most edge pixels is compared with a first ratio threshold, and / or the pixel percentage of the extended region with the most edge pixels is compared with a second ratio threshold to determine the edge type. Taking a first ratio threshold of 5 / 36 as an example, the pixel percentage of the first edge region (0-22.5 degree interval) of 7 / 36 is greater than the first ratio threshold of 5 / 36. Therefore, the edge type is determined to be the edge direction type corresponding to the first edge region, i.e., the edge direction of 0-22.5 degrees.

[0115] It should be noted that, in the embodiments of this application, the edge type can be determined by choosing one of the following: the relationship between the pixel proportion of the edge region and the first proportional threshold, or the relationship between the pixel proportion of the extended region and the second proportional threshold. Alternatively, both can be used to determine the edge type simultaneously, without limitation. For example, when the pixel proportion of the edge region is greater than the first proportional threshold, the edge type is determined to be the edge direction type corresponding to that edge region; or, when the pixel proportion of the extended region is greater than the second proportional threshold, the edge type is determined to be the edge direction type corresponding to that extended region; or, when the pixel proportion of the edge region is greater than the first proportional threshold and the pixel proportion of the extended region is greater than the second proportional threshold, the edge type is determined to be the edge direction type corresponding to either the edge region or the extended region, without limitation.

[0116] Please see Figure 15 In some implementations, the target image includes multiple target pixels. The corresponding position of the target pixel in the input image is the center pixel. Edge detection of the input image using a second edge detection method to obtain the first image gradient (i.e., O12) includes:

[0117] 0121: Determine a first local region within a first predetermined range surrounding the center pixel;

[0118] 0122: Select local directional regions in multiple different directions within the first local region;

[0119] 0123: Determine the first direction image gradient corresponding to the local direction region in each direction, and use the first direction image gradients corresponding to the local direction regions in multiple different directions as the first image gradient;

[0120] A third edge detection method is used to perform edge detection on the input image to obtain the second image gradient (i.e., 013), including:

[0121] 0131: Determine a second local region within a second predetermined range surrounding the center pixel; wherein the second predetermined range is smaller than the first predetermined range;

[0122] 0132: Determine the second image gradient of the second local region in multiple different directions, and use the second image gradients in multiple different directions as the second image gradient;

[0123] The weight value (i.e., 021) corresponding to each original pixel is determined based on the gradient of the first image and the gradient of the second image, including:

[0124] 0211: Determine the weight value of the original pixel in each direction based on the first direction image gradient and the second direction image gradient corresponding to multiple different directions.

[0125] Please see Figure 2 In some implementations, the target image includes multiple target pixels. The corresponding position of the target pixel in the input image is the center pixel. The edge detection module 10 can be used to implement the methods in 0121, 0122, 0123, 0131, and 0132, and the weight determination module 20 can be used to implement the method in 0211.

[0126] In other words, the edge detection module 10 can be used to: determine a first local region within a first predetermined range around the center pixel; select local directional regions in multiple different directions within the first local region; determine the first directional image gradient corresponding to the local directional region in each direction, and use the first directional image gradients corresponding to the local directional regions in multiple different directions as the first image gradient; determine a second local region within a second predetermined range around the center pixel; wherein the second predetermined range is smaller than the first predetermined range; determine the second directional image gradient of the second local region in multiple different directions, and use the second directional image gradients in multiple different directions as the second image gradient. The weight determination module 20 can be used to determine the weight value corresponding to the original pixel in each direction based on the first directional image gradient and the second directional image gradient corresponding to multiple different directions.

[0127] Specifically, the explanation of 0111 in the foregoing embodiments also applies to 0121 in the embodiments of this application, and will not be elaborated further here. Please refer to [link to relevant documentation]. Figure 16 In this embodiment, after determining a first local region within a predetermined range around the center pixel, local directional regions are selected within the first local region in multiple different directions. That is, the first local region includes local directional regions. These local directional regions in multiple different directions may overlap, and the multiple different directions may be, for example, upward, downward, leftward, and rightward.

[0128] by Figure 16For example, the first local region is an 8x8 area covering the center pixel. Within this 8x8 region, 5x4 local directional regions are selected upwards and downwards, and 4x5 local directional regions are selected to the left and right. It can be understood that switching from the up-down direction to the left-right direction involves a 90-degree rotation, thus changing the range of the local directional regions from 5x4 to 4x5. This allows for the calculation of the first-direction image gradient corresponding to each local directional region in a similar manner.

[0129] Taking the local region in the upward direction as an example, the method for calculating the image gradient in the first direction is as follows: traverse each original pixel in rows 3 to 5 of the 5*4 local region, and calculate the difference Diff for each original pixel. The formula for calculating the difference Diff of the original pixels (i, j) is as follows:

[0130] Diff=abs(pixel(i,j)-pixel(i,j-2))

[0131] Where pixel is the pixel value of the original pixel, i is the x-coordinate of the original pixel, j is the y-coordinate of the pixel, and abs is the absolute value. Following this pattern, we can obtain the difference Diff for each original pixel in rows 3 to 5, resulting in a total of 12 difference Diff values. Summing these 12 difference Diff values ​​yields the first-direction image gradient sum diff corresponding to the local upward direction region.

[0132] Similarly, for the local region in the downward direction, the method for calculating the image gradient in the first direction is as follows: traverse each original pixel in rows 1 to 3 of the 5*4 local region, and calculate the difference Diff for each original pixel. The formula for calculating the difference Diff of the original pixel (i, j) is as follows:

[0133] Diff=abs(pixel(i,j)-pixel(i,j+2))

[0134] Similarly, we can obtain the difference Diff for each original pixel in rows 1 to 3, resulting in a total of 12 difference Diff values. Summing these 12 difference Diff values ​​yields the first-direction image gradient sumdiff corresponding to the local downward-direction region.

[0135] For the local region to the left, the gradient of the first direction image is calculated as follows: traverse each original pixel in columns 3 to 5 of the 4*5 local region, and calculate the difference Diff for each original pixel. The formula for calculating the difference Diff of the original pixels (i, j) is as follows:

[0136] Diff=abs(pixel(i,j)-pixel(i-2,j))

[0137] Similarly, we can obtain the difference Diff for each original pixel in columns 3 to 5, resulting in a total of 12 difference Diff values. Summing these 12 difference Diff values ​​yields the first-direction image gradient sumdiff corresponding to the local region to the left.

[0138] For the local region to the right, the gradient of the first direction image is calculated as follows: traverse each original pixel in columns 1 to 3 of the 4*5 local region, and calculate the difference Diff for each original pixel. The formula for calculating the difference Diff of the original pixels (i, j) is as follows:

[0139] Diff=abs(pixel(i,j)-pixel(i+2,j))

[0140] Similarly, we can obtain the difference Diff for each original pixel in columns 1 to 3, resulting in a total of 12 difference Diff values. Summing these 12 difference Diff values ​​yields the first-direction image gradient sumdiff corresponding to the local region to the right.

[0141] Thus, the first-direction image gradients sum diff corresponding to the local directional regions in the four directions of upward, downward, leftward, and rightward can be obtained respectively, denoted as sum diff top, sum diff down, sum diff left, and sum diff right. The first-direction image gradients corresponding to the local directional regions in these four directions are used as the first image gradient.

[0142] Please see Figure 17 This application embodiment further defines a second local region within a second predetermined range surrounding the center pixel. The second predetermined range is smaller than the first predetermined range. After determining the second local region within the second predetermined range surrounding the center pixel, second-direction image gradients are determined for each of the second local regions in multiple different directions, such as upward, downward, leftward, and rightward directions. Figure 17 For example, the second predetermined range is 4*4, and correspondingly, the second local region is a 4*4 region covering the center pixel. The second image gradient of the second local region in multiple different directions is: the second image gradient of the 4*4 second local region in the upward, downward, leftward, and rightward directions.

[0143] Taking the upward gradient of the second local region in the second direction as an example, the method for calculating the gradient of the second direction image is as follows: traverse each original pixel in rows 1 to 3 of the 4*4 second local region, and calculate the gradient grad for each original pixel. The formula for calculating the gradient grad of the original pixel (i, j) is as follows:

[0144] grad=abs(pixel(i,j)-pixel(i,j+1))

[0145] Where pixel is the pixel value of the original pixel, i is the x-coordinate of the original pixel, j is the y-coordinate of the original pixel, and abs is the absolute value. Following this pattern, the gradient grad for each original pixel in rows 1-3 can be obtained, resulting in a total of 12 gradients grad. Summing these 12 gradients grad yields the second-direction image gradient gradV of the second local region upwards.

[0146] Similarly, for the downward second-direction image gradient of the second local region, the method for calculating the second-direction image gradient is as follows: traverse each original pixel in rows 2 to 4 of the 4*4 second local region, and calculate the gradient grad for each original pixel. The formula for calculating the gradient grad of the original pixel (i, j) is as follows:

[0147] grad=abs(pixel(i,j)-pixel(i,j-1))

[0148] Similarly, we can obtain the gradient grad for each original pixel in rows 2 to 4, resulting in a total of 12 gradients grad. Summing these 12 gradients grad yields the downward gradient gradV of the second local region. Expanding the calculation reveals that the upward gradient gradV of the second local region is equal to the downward gradient gradV of the second local region; both can be used as the vertical gradient gradV of the second local region.

[0149] The gradient of the image in the second direction to the left of the second local region is calculated as follows: Traverse each original pixel in columns 1 to 3 of the 4*4 second local region, and calculate the gradient grad for each original pixel. The formula for calculating the gradient grad of the original pixel (i, j) is as follows:

[0150] grad=abs(pixel(i,j)-pixel(i+1,j))

[0151] Similarly, we can obtain the gradient grad for each original pixel in columns 1 to 3, resulting in a total of 12 gradients grad. Summing these 12 gradients grad yields the second image gradient gradH in the second direction to the left of the second local region.

[0152] The second-direction image gradient for the second local region is calculated as follows: Traverse each original pixel in columns 2 to 4 of the 4x4 second local region, and calculate the gradient grad for each original pixel. The formula for calculating the gradient grad of the original pixel (i, j) is as follows:

[0153] grad=abs(pixel(i,j)-pixel(i-1,j))

[0154] Similarly, we can obtain the gradient grad for each original pixel in columns 2 to 4, resulting in a total of 12 gradients grad. Summing these 12 gradients grad yields the second-direction image gradient gradH of the second local region to the right. Expanding the calculation reveals that the second-direction image gradient gradH of the second local region to the left is equal to the second-direction image gradient gradH of the second local region to the right; both can be used as the second-direction image gradient gradH in the horizontal direction of the second local region.

[0155] Thus, the second-direction image gradients in the four directions of upward, downward, leftward, and rightward of the second local region can be obtained, denoted as gradV, gradV, gradH, and gradH respectively. These four second-direction image gradients are used as the second image gradient.

[0156] Finally, based on the first-direction image gradients sum diff top, sum diff down, sum diff left, and sum diff right corresponding to the local directional regions in the four directions, and the second-direction image gradients gradV, gradV, gradH, and gradH in the four directions, the weight values ​​Weight top, Weight down, Weight left, and Weight right corresponding to the original pixels in the four directions can be determined. The specific calculation formula is as follows:

[0157] Weight top=sum diff down / GradV

[0158] Weight down=sum diff top / GradV

[0159] Weight left=sum diff right / GradH

[0160] Weight right=sum diff left / GradH

[0161] Please see Figure 18In some implementations, determining the interpolation method (i.e., 031) based on the edge type includes:

[0162] 0311: After determining the edge type, determine the current confidence level corresponding to the edge type based on the pixel ratio of the edge region and / or the pixel ratio of the extended region;

[0163] 0312: When the current confidence level is within the first confidence level interval, the bicubic interpolation algorithm is used.

[0164] 0313: When the current confidence level is in the second confidence level interval, an interpolation method that interpolates along multiple different directions is adopted;

[0165] 0314: When the current confidence level is in the third confidence level interval, the interpolation method is adopted to interpolate along the edge direction determined by the edge type;

[0166] The confidence levels for the first confidence interval, the second confidence interval, and the third confidence interval increase sequentially.

[0167] Please see Figure 2 In some embodiments, the interpolation selection module 30 can be used to implement the methods in 0311, 0312, 0313, and 0314. That is, the interpolation selection module 30 can be used to: after determining the edge type, determine the current confidence level corresponding to the edge type based on the pixel ratio of the edge region and / or the pixel ratio of the extended region; when the current confidence level is in the first confidence level interval, use the bicubic interpolation algorithm; when the current confidence level is in the second confidence level interval, use the interpolation method that interpolates along multiple different directions; when the current confidence level is in the third confidence level interval, use the interpolation method that interpolates along the edge direction determined by the edge type; wherein the confidence levels corresponding to the first confidence level interval, the second confidence level interval, and the third confidence level interval increase sequentially.

[0168] Specifically, in this embodiment, after determining the edge type, the current confidence level corresponding to that edge type is determined based on the pixel proportion of the edge region and / or the pixel proportion of the extended region. That is, this embodiment determines the current confidence level corresponding to the edge type based on the condition that the pixel proportion of the edge region is greater than a first proportional threshold and / or the pixel proportion of the extended region is greater than a second proportional threshold. When determining the current confidence level corresponding to the edge type, it can be determined solely based on the pixel proportion of the edge region, solely based on the pixel proportion of the extended region, or simultaneously based on both the pixel proportions of the edge region and the pixel proportions of the extended region.

[0169] Taking the determination of the current confidence level corresponding to the edge type based on the pixel proportion of the edge region as an example, combined with the case in the previous example where the pixel proportion of the first edge region (0-22.5 degree interval) is 7 / 36, which is greater than the first proportion threshold of 5 / 36, and the edge type is determined to be the edge direction type corresponding to the first edge region (i.e., the edge direction of 0-22.5 degrees), the current confidence level corresponding to the edge direction type of the first edge region is further determined based on the pixel proportion of the first edge region (7 / 36). For example, the current confidence level can be quantified based on the pixel proportion of the edge region. The higher the pixel proportion of the edge region, the higher the current confidence level; the lower the pixel proportion of the edge region, the lower the current confidence level.

[0170] In one example, the first confidence interval is 0-30%, the second confidence interval is 30-60%, and the third confidence interval is 70-100%. Assuming that the current confidence level quantized based on the aforementioned pixel ratio of 7 / 36 is 10%, the current confidence level is determined to be in the first confidence interval, which is a case of extremely low confidence. Assuming that the current confidence level quantized based on the pixel ratio is 50%, the current confidence level is determined to be in the second confidence interval, which is a case of moderate confidence. Assuming that the current confidence level quantized based on the pixel ratio is 80%, the current confidence level is determined to be in the third confidence interval, which is a case of extremely high confidence. (1) For cases with extremely low confidence, it indicates that the confidence of the determined edge direction type is very low. In this case, the bicubic interpolation algorithm can be used. In addition, for some cases in the embodiments of this application where the edge type cannot be determined, it can also be understood as having extremely low confidence, and the bicubic interpolation algorithm can be used. (2) For cases with moderate confidence, indicating that the edge direction type cannot be accurately determined, an interpolation method that interpolates along multiple different directions can be used (the specific interpolation algorithm is not limited). For example, interpolation can be performed along the aforementioned four directions: upward, downward, leftward, and rightward, combined with the weight values ​​corresponding to the original pixels in the four directions. (3) For cases with extremely high confidence, indicating that the determined edge direction type is highly reliable, an interpolation method that interpolates along the determined edge direction can be used (the specific interpolation algorithm is not limited). For example, interpolation can be performed along the aforementioned edge direction of 0 to 22.5 degrees, combined with the weight values ​​corresponding to the original pixels in that edge direction. The implementation method of this application reasonably selects the corresponding interpolation method according to the confidence range to which the current confidence level belongs, which can reduce the number of artifacts generated in the target image and make it less likely to produce jagged edges or blurry images after magnification.

[0171] Please see Figure 19 In some implementations, multiple different types of edge detection methods are used to perform edge detection on the input image to obtain the detection result (i.e., 01), and the method further includes:

[0172] 014: Determine the edge severity based on the second-direction image gradient of the second local region in multiple different directions;

[0173] The input image is interpolated according to the interpolation method and the weight value corresponding to each original pixel to obtain the target image (i.e., 04), including:

[0174] 041: The first interpolation result is obtained by interpolating the input image using a bicubic interpolation algorithm and the weight value corresponding to each original pixel, to output the target image; or

[0175] 042: The input image is interpolated using an interpolation method that interpolates along multiple different directions and the weight value corresponding to each original pixel to obtain the second initial interpolation result;

[0176] 043: Determine the second interpolation result based on the second initial interpolation result, the edge severity, and the first interpolation result to output the target image; or

[0177] 044: The third initial interpolation result is obtained by interpolating the input image using an interpolation method that interpolates along the edge direction determined by the edge type and the weight value corresponding to each original pixel;

[0178] 045: Determine the third interpolation result based on the third initial interpolation result, the edge degree, and the first interpolation result to output the target image.

[0179] Please see Figure 2 In some embodiments, the edge detection module 10 can be used to implement the method in 014, and the interpolation processing module 40 can be used to implement the methods in 041, 042, 043, 044, and 045. That is, the edge detection module 10 can be used to determine the edge degree based on the second local region's second-direction image gradient towards multiple different directions. The interpolation processing module 40 can be used to: interpolate the input image using a bicubic interpolation algorithm and the weight value corresponding to each original pixel to obtain a first interpolation result, and output a target image; or interpolate the input image using an interpolation method that interpolates along multiple different directions and the weight value corresponding to each original pixel to obtain a second initial interpolation result; determine a second interpolation result based on the second initial interpolation result, the edge degree, and the first interpolation result, and output a target image; or interpolate the input image using an interpolation method that interpolates along the edge direction determined by the edge type and the weight value corresponding to each original pixel to obtain a third initial interpolation result; determine a third interpolation result based on the third initial interpolation result, the edge degree, and the first interpolation result, and output a target image.

[0180] In this embodiment, after obtaining the second-direction image gradients (gradV, gradV, gradH, gradH) of the second local region in the four directions of upward, downward, leftward, and rightward using the aforementioned method, the edge strength can be determined based on the second-direction image gradients (gradV, gradV, gradH, gradH). Specifically, the edge strength can be represented by the difference between the second-direction image gradient gradV in the vertical direction and the second-direction image gradient gradH in the horizontal direction of the second local region, calculated using the following formula:

[0181] EdgeStrength=abs(gradH-gradV)

[0182] To further reduce image artifacts, the interpolation results of the second and third interpolation methods can be adjusted based on the edge strength and the interpolation result of the first interpolation method (i.e., the fusion ratio between directional and non-directional interpolation results is controlled by the edge strength). The calculation formula is as follows:

[0183] result2=result2(0)*EdgeStrength+(1-EdgeStrength)*result1

[0184] result3=result3(0)*EdgeStrength+(1-EdgeStrength)*result1

[0185] Wherein, result1 is the first interpolation result, result2 is the second interpolation result, result3 is the third interpolation result, result2(0) is the second initial interpolation result, and result3(0) is the third initial interpolation result.

[0186] The second initial interpolation result, result2(0), uses an interpolation method that interpolates along multiple different directions. result2(0) fuses the interpolation results in the upward, downward, leftward, and rightward directions using the corresponding weight values ​​in each of the four directions. For example... Figure 20 As shown, within the first local region of 8*8, 4*4 interpolation windows are selected upwards, downwards, leftwards, and rightwards respectively. For each 4*4 interpolation window, bicubic interpolation can be used to obtain the interpolation results (result top, result down, result left, result right) in each direction. Then, the results are fused using the corresponding weight values ​​(weight top, weight down, weight left, weight right) in the four directions. The calculation formula is as follows:

[0187] result2(0)=result top*weight top+result down*weight down+resultleft*weight left+result right*weight right

[0188] Among them, result top is the interpolation result upward, result down is the interpolation result downward, result left is the interpolation result to the left, and result right is the interpolation result to the right.

[0189] Please see Figure 21 In some implementations, the target image includes multiple target pixels. The corresponding position of the target pixel in the input image is the center pixel. The upsampling method further includes:

[0190] 05: Determine a first local region within a first predetermined range surrounding the center pixel;

[0191] 06: Perform pixel statistical analysis on the first local region to obtain pixel information;

[0192] Based on the detection results, determine the weight value (i.e., 02) corresponding to each original pixel in the input image within the interpolation range, including:

[0193] 022: Determine the weight value corresponding to each original pixel based on the detection results and pixel information.

[0194] Please see Figure 2 In some embodiments, the target image includes multiple target pixels. The corresponding position of the target pixel in the input image is the center pixel. The upsampling device 100 also includes a region analysis module 50. The region analysis module 50 can be used to implement the methods in 05 and 06, and the weight determination module 20 can be used to implement the method in 022. That is, the region analysis module 50 can be used to: determine a first local region within a first predetermined range around the center pixel; and perform pixel statistical analysis on the first local region to obtain pixel information. The weight determination module 20 can be used to determine the weight value corresponding to each original pixel based on the detection results and the pixel information.

[0195] Specifically, the explanation of 0111 in the foregoing embodiments also applies to 05 of the embodiments of this application, and will not be elaborated further here. Please refer to Figure 7 In this application embodiment, pixel statistical analysis is performed on the original pixels within a first local region to obtain pixel information. The pixel information may include any one or more of the following: the mean pixel value, the variance pixel value, and the difference pixel value among multiple original pixels within the first local region. For example, for... Figure 7 Pixel statistical analysis is performed on the 8*8 original pixels to calculate the mean, variance, and contrast of the pixel values. The contrast is the difference between the maximum and minimum pixel values.

[0196] The formula for calculating the mean of pixel values ​​is as follows:

[0197]

[0198] The formula for calculating the pixel value variance is shown below:

[0199]

[0200] The formula for calculating the maximum pixel value (max) is as follows:

[0201] max = max(pixelvalue(i,j))

[0202] The formula for calculating the minimum pixel value (min) is shown below:

[0203] min = min(pixelvalue(i,j))

[0204] The formula for calculating the pixel value difference (contrast) is shown below:

[0205] contrast = max - min

[0206] Where pixelvalue is the pixel value of the original pixel, i is the x-coordinate of the original pixel, j is the y-coordinate of the original pixel, and abs is the absolute value.

[0207] The aforementioned pixel information reflects the state of the first local region. Specifically, the mean pixel value reflects the brightness of the first local region. A larger mean pixel value indicates higher brightness in the first local region. The variance pixel value represents the flatness of the first local region. A larger variance pixel value indicates lower flatness in the first local region. The contrast pixel value represents the contrast of the first local region. A larger contrast pixel value indicates higher contrast in the first local region. After performing pixel statistical analysis on the first local region, the weight value corresponding to each original pixel can be adjusted using the pixel information to obtain a more reasonable and accurate weight value, thereby effectively reducing artifacts generated by interpolation along the edge direction.

[0208] Please see Figure 22 In some implementations, determining the weight value (i.e., 021) corresponding to each original pixel based on the first image gradient and the second image gradient further includes:

[0209] 0212: After determining the weight value of the original pixel in each direction based on the first direction image gradient and the second direction image gradient corresponding to multiple different directions, the weight value of the original pixel in each direction is adjusted according to the pixel information of the first local region.

[0210] Please see Figure 2 In some implementations, the weight determination module 20 can be used to implement the method in 0212. That is, the weight determination module 20 can be used to adjust the weight values ​​of the original pixels in each direction according to the pixel information of the first local region after determining the weight values ​​of the original pixels in each direction based on the first direction image gradient and the second direction image gradient corresponding to multiple different directions.

[0211] For example, after determining the weight values ​​(Weight top = sum diff down / GradV; Weight down = sum difftop / GradV; Weight left = sum diff right / GradH; Weight right = sum diff left / GradH) of the original pixels in the four directions (up, down, left, and right) based on the first and second direction image gradients respectively, the weight values ​​(Weight top, Weight down, Weight left, and Weight right) of the original pixels in the four directions can be adjusted using the pixel mean, pixel variance, and pixel difference, respectively. In subsequent interpolation processing based on the weight values ​​of each original pixel, the adjusted weight values ​​for each original pixel are used.

[0212] Please see Figure 23 In some implementations, various edge detection methods are used to perform edge detection on the input image to obtain the detection result (i.e., 01), including:

[0213] 015: Determine the attenuation factor of the edge severity based on the pixel information of the first local region;

[0214] 016: Adjust the edge intensity based on the attenuation factor;

[0215] The second interpolation result (i.e., 043) is determined based on the second initial interpolation result, the edge degree, and the first interpolation result, including:

[0216] 0431: Determine the second interpolation result based on the second initial interpolation result, the adjusted edge severity, and the first interpolation result;

[0217] The third interpolation result (i.e., 045) is determined based on the third initial interpolation result, the edge degree, and the first interpolation result, including:

[0218] 0451: Determine the third interpolation result based on the third initial interpolation result, the adjusted edge degree, and the first interpolation result.

[0219] Please see Figure 2 In some embodiments, the edge detection module 10 can be used to implement the methods in 015 and 016, and the interpolation processing module 40 can be used to implement the methods in 0431 and 0451. That is, the edge detection module 10 can be used to: determine an edge attenuation factor based on the pixel information of the first local region; and adjust the edge degree based on the attenuation factor. The interpolation processing module 40 can be used to: determine a second interpolation result based on a second initial interpolation result, the adjusted edge degree, and the first interpolation result; and determine a third interpolation result based on a third initial interpolation result, the adjusted edge degree, and the first interpolation result.

[0220] Specifically, after determining the edge strength (EdgeStrength) based on the second-direction image gradients of the second local region in multiple different directions, the edge strength can be adjusted using a decay factor (gain). This adjustment can be achieved by multiplying the edge strength by the decay factor (gain). The magnitude of the decay factor (gain) is determined by pixel information. For different pixel mean, pixel variance, and pixel difference, the corresponding value of the decay factor (gain) can be different, resulting in varying degrees of edge strength decay. The specific formula is shown below:

[0221] EdgeStrength(after adjustment)=EdgeStrength*gain(mean)*gain(contrast)*gain(variance)

[0222] Here, EdgeStrength(adjusted) represents the adjusted edge strength, EdgeStrength represents the original edge strength, gain(mean) represents the attenuation factor corresponding to the pixel value mean, gain(contrast) represents the attenuation factor corresponding to the pixel value variance, and gain(variance) represents the attenuation factor corresponding to the pixel value difference contrast. For the relationships between the pixel value mean and attenuation factor gain, the pixel value variance and attenuation factor gain, and the pixel value difference contrast and attenuation factor gain, gain curves can be pre-set to call the corresponding attenuation factors gain(mean), gain(contrast), and gain(variance) based on the pixel value mean, pixel value variance, and pixel value difference contrast, respectively.

[0223] It is understandable that after adjusting the edge degree according to the attenuation factor, the adjusted edge degree is used when calculating the second and third interpolation results to make the interpolation results more accurate. This will not be explained further here.

[0224] Please see Figure 24 In some implementations, the target image includes multiple target pixels. The corresponding position of the target pixel in the input image is the center pixel. Edge detection is performed on the input image using a first edge detection method to obtain the edge type (i.e., 011), including:

[0225] 0115: Determine a third local region within a third predetermined range surrounding the center pixel; wherein the third local region includes the center region, and the center pixel is located within the center region;

[0226] 0116: Within the third local region, multiple reference regions are selected from the central region in multiple different directions;

[0227] 0117: Calculate the sum of multiple absolute differences in multiple directions based on the pixel values ​​of multiple original pixels in the central region and the pixel values ​​of multiple original pixels in multiple reference regions;

[0228] 0118: Determine the edge type based on multiple absolute differences in multiple different directions.

[0229] Please see Figure 2In some embodiments, the target image includes multiple target pixels. The corresponding position of the target pixel in the input image is the center pixel. The edge detection module 10 can be used to implement the methods in 0115, 0116, 0117, and 0118. That is, the edge detection module 10 can be used to: determine a third local region within a third predetermined range around the center pixel; wherein the third local region includes a central region, and the center pixel is located within the central region; select multiple reference regions in the third local region from the central region toward multiple different directions; calculate multiple absolute differences in multiple different directions based on the pixel values ​​of multiple original pixels in the central region and the pixel values ​​of multiple original pixels in the multiple reference regions; and determine the edge type based on the multiple absolute differences in multiple different directions.

[0230] Specifically, embodiments of this application can determine the edge type by summing multiple absolute differences in multiple directions within a third local region. See also... Figure 25 First, a third local region is defined within a predetermined range surrounding the center pixel. For example, the third predetermined range can be 3*3, 4*4, 7*5, 7*8, etc., without restriction. The third predetermined range can be set based on empirical values. Correspondingly, the third local region is a 3*3 region covering the center pixel, or a 4*4 region covering the center pixel, or a 7*7 region covering the center pixel, or an 8*8 region covering the center pixel, etc.

[0231] Figure 25 The third local region is a 7x7 area covering the center pixel. This third local region includes a 3x3 central region, from which multiple 3x3 reference regions can be selected in different directions. Figure 25Taking the 45-degree direction passing through the central region as an example, moving along this direction with a step size of 1, using the central region as a reference, two 3*3 reference regions can be obtained by moving along the upper right corner, and two more 3*3 reference regions can be obtained along the lower left corner, resulting in a total of four 3*3 reference regions along the 45-degree direction. Based on the pixel values ​​of the nine original pixels in each reference region and the nine original pixels in the central region, the absolute difference (SAD) corresponding to that reference region can be calculated. For example, the absolute difference (SAD) corresponding to the reference region moved by a step size of 1 from the central region to the upper right corner is: |A33-A24|+|A34-A25|+A35-A26|+|A43-A34|+|A44-A35|+|A45-A36|+|A53-A44|+|A54-A45|+|A55-A46|. Therefore, a total of four absolute difference (SAD) can be calculated in each direction. Similarly, for other directions, four absolute differences and SAD can be calculated for each direction. The edge type can be determined based on multiple absolute differences and SAD in multiple different directions. For example, by judging the relationship between multiple absolute differences and SAD in multiple different directions and the absolute difference and threshold, it can be determined whether the corresponding reference area is an edge pixel (e.g., if the absolute difference and SAD are greater than the absolute difference and threshold, the reference area is considered an edge pixel), and the edge type can be determined based on the pixel proportion of edge pixels in each direction (e.g., the edge type is determined as the edge direction type corresponding to the direction with the highest pixel proportion). Of course, the methods for determining the edge type based on multiple absolute differences and SAD in multiple different directions are not limited to this, and will not be illustrated here.

[0232] Please see Figure 26 In some implementations, various edge detection methods are used to perform edge detection on the input image to obtain the detection result (i.e., 01), including:

[0233] 017: Employ various edge detection methods based on brightness values ​​to perform edge detection on the input image to obtain the first detection result;

[0234] Based on the detection results, determine the weight value (i.e., 02) corresponding to each original pixel in the input image within the interpolation range, including:

[0235] 023: Determine the weight value corresponding to each original pixel within the interpolation range based on the first detection result.

[0236] Please see Figure 2In some embodiments, the edge detection module 10 can be used to implement the method in 017, and the weight determination module 20 can be used to implement the method in 023. That is, the edge detection module 10 can be used to perform edge detection on the input image based on the brightness value using various different types of edge detection methods to obtain a first detection result. The weight determination module 20 can be used to determine the weight value corresponding to each original pixel within the interpolation range based on the first detection result.

[0237] Specifically, the pixel value of each original pixel can include a luminance value (Y value) and a chromaticity value (UV value). In this embodiment, when performing edge detection on the input image using various edge detection methods, the luminance value can be used as the basis. The luminance value can well reflect the grayscale changes of the image, thus obtaining a reasonable first detection result (e.g., a first image gradient, a second image gradient), and the weight value corresponding to each original pixel determined accordingly is also more reasonable. It can be understood that when the input image is not a YUV image, for example, when the input image is an RGB image, it can be converted to a YUV image, thereby obtaining the YUV values.

[0238] Please see Figure 27 In some implementations, various edge detection methods are used to perform edge detection on the input image to obtain the detection result (i.e., 01), including:

[0239] 018: Employ various edge detection methods based on brightness values ​​to perform edge detection on the input image to obtain the first detection result;

[0240] 019: Perform edge detection on the input image based on chromaticity values ​​to obtain a second detection result;

[0241] Based on the detection results, determine the weight value (i.e., 02) corresponding to each original pixel in the input image within the interpolation range, including:

[0242] 024: Determine the weight value corresponding to each original pixel within the interpolation range based on the first and second detection results.

[0243] Please see Figure 2 In some embodiments, the edge detection module 10 can be used to implement the methods in 018 and 019, and the weight determination module 20 can be used to implement the method in 024. That is, the edge detection module 10 can be used to: perform edge detection on the input image based on brightness values ​​using multiple different types of edge detection methods to obtain a first detection result; and perform edge detection on the input image based on chromaticity values ​​to obtain a second detection result. The weight determination module 20 can be used to determine the weight value corresponding to each original pixel within the interpolation range based on the first detection result and the second detection result.

[0244] Specifically, in this embodiment, when performing edge detection on the input image using various edge detection methods, not only luminance values ​​are considered, but also chrominance values ​​are added. On one hand, various edge detection methods can be used to perform edge detection on the input image based on luminance values ​​to obtain a first detection result; on the other hand, edge detection can be performed on the input image based on chrominance values ​​to obtain a second detection result (the edge detection method based on chrominance values ​​is not limited). Finally, the weight value corresponding to each original pixel is determined by combining the first and second detection results, so that the weight value corresponding to each original pixel is more accurate.

[0245] In summary, the upsampling method and upsampling device 100 based on multi-type edge detection according to the embodiments of this application have at least the following advantages:

[0246] First, using various edge detection methods to perform edge detection on the input image can improve the accuracy and robustness of edge detection, resulting in fewer artifacts in the upsampled image.

[0247] Second, during interpolation, the interpolation results of the second and third interpolation methods are adjusted by using the EdgeStrength and the interpolation results of the first interpolation method (i.e., the fusion ratio between directional and non-directional interpolation results is controlled by the EdgeStrength) to further reduce the generation of artifacts.

[0248] Third, by performing pixel statistical analysis on the first local region to obtain pixel information (pixel mean, pixel variance, pixel difference), the weight value corresponding to each original pixel can be adjusted, which can effectively reduce the artifacts generated by interpolation processing along the edge direction.

[0249] like Figure 28 , Figure 29 , Figure 30 As shown in the figures, (a) is a schematic diagram of the effect of the upsampling method according to the embodiment of this application, and (b) is a schematic diagram of the effect of a commonly used upsampling method in the industry. It can be seen from the figures that the upsampling method according to the embodiment of this application has better smoothness at the edges and fewer artifacts.

[0250] Please see Figure 31 This application also provides an electronic device 200. The electronic device 200 includes one or more processors 210 and a memory 220. The memory 220 stores a computer program that, when executed by the processor 210, implements the upsampling method based on multi-type edge detection according to any of the above embodiments.

[0251] For example, when a computer program is executed by processor 210, the following upsampling method is implemented:

[0252] 01: Employ various edge detection methods to perform edge detection on the input image to obtain detection results;

[0253] 02: Determine the weight value corresponding to each original pixel in the input image within the interpolation range based on the detection results;

[0254] 03: Determine the interpolation method used for interpolating the input image based on the detection results;

[0255] 04: The input image is interpolated according to the interpolation method and the weight value corresponding to each original pixel to obtain the target image.

[0256] For example, when a computer program is executed by processor 210, the following upsampling method is implemented:

[0257] 011: The first edge detection method is used to perform edge detection on the input image to obtain the edge type;

[0258] 012: The second edge detection method is used to perform edge detection on the input image to obtain the first image gradient;

[0259] 013: Use a third edge detection method to perform edge detection on the input image to obtain the second image gradient;

[0260] 021: Determine the weight value corresponding to each original pixel based on the gradient of the first image and the gradient of the second image;

[0261] 031: Determine the interpolation method based on the edge type.

[0262] The electronic devices 200 described in this application include, but are not limited to, mobile phones, tablets, cameras, personal digital assistants, wearable devices, smart robots, smart vehicles, and terminal devices. Wearable devices include smart bracelets, smartwatches, and smart glasses.

[0263] It should be noted that the explanations of the upsampling method and upsampling device 100 based on multi-type edge detection in the foregoing embodiments are also applicable to the electronic device 200 of the embodiments of this application, and will not be elaborated here.

[0264] Please see Figure 32 This application also provides a computer-readable storage medium 300 on which a computer program 310 is stored. When the program 310 is executed by the processor 320, it implements the upsampling method based on multi-type edge detection of any of the above embodiments.

[0265] For example, when program 310 is executed by processor 320, the following upsampling method is implemented:

[0266] 01: Employ various edge detection methods to perform edge detection on the input image to obtain detection results;

[0267] 02: Determine the weight value corresponding to each original pixel in the input image within the interpolation range based on the detection results;

[0268] 03: Determine the interpolation method used for interpolating the input image based on the detection results;

[0269] 04: The input image is interpolated according to the interpolation method and the weight value corresponding to each original pixel to obtain the target image.

[0270] For example, when program 310 is executed by processor 320, the following upsampling method is implemented:

[0271] 011: The first edge detection method is used to perform edge detection on the input image to obtain the edge type;

[0272] 012: The second edge detection method is used to perform edge detection on the input image to obtain the first image gradient;

[0273] 013: Use a third edge detection method to perform edge detection on the input image to obtain the second image gradient;

[0274] 021: Determine the weight value corresponding to each original pixel based on the gradient of the first image and the gradient of the second image;

[0275] 031: Determine the interpolation method based on the edge type.

[0276] It should be noted that the explanations and descriptions of the upsampling method and upsampling device 100 based on multi-type edge detection in the foregoing embodiments also apply to the computer-readable storage medium 300 of the embodiments of this application, and will not be elaborated here.

[0277] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., refer to specific features, structures, materials, or characteristics described in connection with that embodiment or example, which are included in at least one embodiment or example of this application. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of different embodiments or examples.

[0278] Any process or method described in the flowchart or otherwise herein can be understood as representing a module, segment, or portion of code comprising one or more executable instructions for implementing a particular logical function or process, and the scope of the preferred embodiments of this application includes additional implementations in which functions may be performed not in the order shown or discussed, including substantially simultaneously or in reverse order depending on the function involved, as will be understood by those skilled in the art to which embodiments of this application pertain.

[0279] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as a sequenced list of executable instructions for implementing logical functions, and can be embodied in any computer-readable storage medium for use by, or in conjunction with, an instruction execution system, apparatus, or device (such as a computer-based system, a processor-included system, or other system that can fetch and execute instructions from, an instruction execution system, apparatus, or device). For the purposes of this specification, a computer-readable storage medium can be any means that can contain, store, communicate, propagate, or transmit programs for use by, or in conjunction with, an instruction execution system, apparatus, or device. More specific examples (a non-exhaustive list) of computer-readable storage media include: an electrical connection having one or more wires (electronic device), a portable computer disk drive (magnetic device), random access memory (RAM), read-only memory (ROM), erasable and programmable read-only memory (EPROM or flash memory), fiber optic devices, and portable optical disc read-only memory (CDROM). Alternatively, the computer-readable storage medium could be paper or other suitable media on which the program can be printed, since the program can be obtained electronically, for example, by optically scanning the paper or other medium, followed by editing, interpreting, or otherwise processing as necessary, and then stored in a computer memory.

[0280] It should be understood that various parts of this application can be implemented using hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented using software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, it can be implemented using any one or a combination of the following techniques known in the art: discrete logic circuits having logic gates for implementing logical functions on data signals, application-specific integrated circuits (ASICs) having suitable combinational logic gates, programmable gate arrays (PGAs), field-programmable gate arrays (FPGAs), etc.

[0281] Those skilled in the art will understand that all or part of the steps of the methods in the above embodiments can be implemented by a program instructing related hardware. The program can be stored in a computer-readable storage medium, and when executed, it includes one or a combination of the steps of the method embodiments. Furthermore, the functional units in the various embodiments of this application can be integrated into a processing module, or each unit can exist physically separately, or two or more units can be integrated into a module. The integrated module can be implemented in hardware or as a software functional module. If the integrated module is implemented as a software functional module and sold or used as an independent product, it can also be stored in a computer-readable storage medium. The storage medium mentioned above can be a read-only memory, a disk, or an optical disk, etc.

[0282] Although embodiments of this application have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting this application. Those skilled in the art can make changes, modifications, substitutions and variations to the above embodiments within the scope of this application, the scope of which is defined by the claims and their equivalents.

Claims

1. An upsampling method based on multi-type edge detection, used to output a target image from an input image, characterized in that, The upsampling method includes: The input image is edge-detected using multiple different edge detection methods to obtain detection results, including using a first edge detection method to perform edge detection on the input image to obtain edge type; using a second edge detection method to perform edge detection on the input image to obtain a first image gradient; and using a third edge detection method to perform edge detection on the input image to obtain a second image gradient. Determining the weight value corresponding to each original pixel of the input image within the interpolation range based on the detection results includes determining the weight value corresponding to each original pixel based on the first image gradient and the second image gradient; Determining the interpolation method for interpolating the input image based on the detection results includes determining the interpolation method based on the edge type; The target image is obtained by interpolating the input image according to the interpolation method and the weight value corresponding to each original pixel.

2. The upsampling method according to claim 1, characterized in that, The target image includes multiple target pixels, and the corresponding position of each target pixel in the input image is a center pixel. The step of performing edge detection on the input image using a first edge detection method to obtain the edge type includes: Determine a first local region within a first predetermined range surrounding the center pixel; Calculate the current gradient magnitude of the first local region; Calculate the current edge direction of the first local region; The edge type is determined based on the current gradient magnitude and the current edge direction.

3. The upsampling method according to claim 2, characterized in that, The first local region includes multiple local sub-regions, and calculating the current gradient magnitude of the first local region includes: The horizontal gradient magnitude and vertical gradient magnitude of each local sub-region are calculated based on the preset image gradient algorithm and the pixel values ​​of the original pixels in each local sub-region. The sub-region gradient magnitude of the local sub-region is determined based on the horizontal gradient magnitude and the vertical gradient magnitude of the sub-region, and the sub-region gradient magnitudes of multiple local sub-regions are used as the current gradient magnitude. The calculation of the current edge direction of the first local region includes: The sub-region edge direction of the local sub-region is determined based on the horizontal gradient magnitude and the vertical gradient magnitude of the sub-region, and the sub-region edge directions of multiple local sub-regions are used as the current edge direction. Determining the edge type based on the current gradient magnitude and the current edge direction includes: The edge type is determined based on the gradient magnitude of the sub-regions and the edge direction of the sub-regions.

4. The upsampling method according to claim 3, characterized in that, Determining the edge type based on the gradient magnitude of the sub-regions and the edge direction of the sub-regions in multiple local sub-regions includes: The gradient magnitude of the sub-region is compared with the gradient magnitude threshold; When the gradient magnitude of the sub-region is greater than the gradient magnitude threshold, the local sub-region corresponding to the gradient magnitude of the sub-region is regarded as an edge pixel; Determine the edge region and / or extended region to which the edge direction of the sub-region corresponding to the edge pixel belongs; Count the number of edge pixels in each edge region and / or extended region to obtain the edge region and / or extended region with the most edge pixels; Determine the pixel percentage of the edge region and / or the extended region that have the largest number of edge pixels; When the pixel percentage of the edge region is greater than a first ratio threshold and / or the pixel percentage of the extended region is greater than a second ratio threshold, the edge type is determined to be the edge direction type corresponding to the edge region and / or the extended region.

5. The upsampling method according to claim 1, characterized in that, The target image includes multiple target pixels, and the corresponding position of each target pixel in the input image is a center pixel. The step of performing edge detection on the input image using a second edge detection method to obtain a first image gradient includes: Determine a first local region within a first predetermined range surrounding the center pixel; Within the first local region, local directional regions are selected in multiple different directions; The first direction image gradient corresponding to the local direction region in each direction is determined respectively, and the first direction image gradients corresponding to the local direction regions in multiple different directions are used as the first image gradient; The step of performing edge detection on the input image using a third edge detection method to obtain the second image gradient includes: A second local region is determined within a second predetermined range around the center pixel; wherein the second predetermined range is smaller than the first predetermined range; The second local region is determined to be oriented towards multiple different directions in the second direction image gradient, and the multiple different directions of the second direction image gradient are used as the second image gradient; The step of determining the weight value corresponding to each original pixel based on the first image gradient and the second image gradient includes: The weight value corresponding to the original pixel in each direction is determined based on the image gradients of the first direction and the second direction corresponding to multiple different directions.

6. The upsampling method according to claim 4, characterized in that, Determining the interpolation method based on the edge type includes: After determining the edge type, the current confidence level corresponding to the edge type is determined based on the pixel ratio of the edge region and / or the pixel ratio of the extended region; When the current confidence level is within the first confidence level interval, the bicubic interpolation algorithm is used for interpolation. When the current confidence level is within the second confidence interval, an interpolation method that interpolates along multiple different directions is adopted; When the current confidence level is in the third confidence level interval, an interpolation method is adopted that interpolates along the edge direction determined by the edge type; The confidence levels corresponding to the first confidence interval, the second confidence interval, and the third confidence interval increase sequentially.

7. The upsampling method according to claim 5, characterized in that, The method of performing edge detection on the input image using multiple different types of edge detection methods to obtain detection results also includes: The edge degree is determined based on the second local region's image gradient in multiple different directions. The step of interpolating the input image according to the interpolation method and the weight value corresponding to each original pixel to obtain the target image includes: The input image is interpolated using a bicubic interpolation algorithm and a weight value corresponding to each original pixel to obtain a first interpolation result, which is then used to output the target image; or The input image is interpolated using an interpolation method that interpolates along multiple different directions, and a weight value corresponding to each original pixel, to obtain a second initial interpolation result; The second interpolation result is determined based on the second initial interpolation result, the edge severity, and the first interpolation result, to output the target image; or The input image is interpolated using an interpolation method that interpolates along the edge direction determined by the edge type, and a weight value corresponding to each original pixel to obtain a third initial interpolation result; The third interpolation result is determined based on the third initial interpolation result, the edge degree, and the first interpolation result, so as to output the target image.

8. The upsampling method according to claim 1, characterized in that, The target image includes multiple target pixels, and the corresponding position of each target pixel in the input image is a center pixel. The upsampling method further includes: Determine a first local region within a first predetermined range surrounding the center pixel; Perform pixel statistical analysis on the first local region to obtain pixel information; The step of determining the weight value corresponding to each original pixel of the input image within the interpolation range based on the detection result includes: The weight value corresponding to each original pixel is determined based on the detection results and the pixel information.

9. The upsampling method according to claim 5, characterized in that, The step of determining the weight value corresponding to each original pixel based on the first image gradient and the second image gradient further includes: After determining the weight value corresponding to the original pixel in each direction based on the first direction image gradient and the second direction image gradient corresponding to multiple different directions, the weight value corresponding to the original pixel in each direction is adjusted according to the pixel information of the first local region.

10. The upsampling method according to claim 7, characterized in that, The method of performing edge detection on the input image using various different edge detection methods to obtain detection results includes: The attenuation factor of the edge severity is determined based on the pixel information of the first local region; The edge severity is adjusted according to the attenuation factor; Determining the second interpolation result based on the second initial interpolation result, the edge degree, and the first interpolation result includes: The second interpolation result is determined based on the second initial interpolation result, the adjusted edge degree, and the first interpolation result; The step of determining the third interpolation result based on the third initial interpolation result, the edge degree, and the first interpolation result includes: The third interpolation result is determined based on the third initial interpolation result, the adjusted edge degree, and the first interpolation result.

11. The upsampling method according to any one of claims 8-10, characterized in that, The pixel information includes any one or more of the pixel value mean, pixel value variance, and pixel value difference of the original pixels in the first local region.

12. The upsampling method according to claim 1, characterized in that, The target image includes multiple target pixels, and the corresponding position of each target pixel in the input image is a center pixel. The step of performing edge detection on the input image using a first edge detection method to obtain the edge type includes: A third local region is determined within a third predetermined range surrounding the center pixel; wherein the third local region includes a central region, and the center pixel is located within the central region; Within the third local region, multiple reference regions are selected from the central region in multiple different directions; Calculate multiple absolute differences in multiple directions based on the pixel values ​​of multiple original pixels in the central region and the pixel values ​​of multiple original pixels in multiple reference regions; The edge type is determined based on multiple absolute differences in multiple different directions.

13. The upsampling method according to claim 1, characterized in that, The method of performing edge detection on the input image using various different edge detection methods to obtain detection results includes: The input image is subjected to edge detection based on brightness values ​​using various different edge detection methods to obtain a first detection result; The step of determining the weight value corresponding to each original pixel of the input image within the interpolation range based on the detection result includes: The weight value corresponding to each original pixel within the interpolation range is determined based on the first detection result.

14. The upsampling method according to claim 1, characterized in that, The method of performing edge detection on the input image using various different edge detection methods to obtain detection results includes: The input image is subjected to edge detection based on brightness values ​​using various different edge detection methods to obtain a first detection result; Edge detection is performed on the input image based on chroma values ​​to obtain a second detection result; The step of determining the weight value corresponding to each original pixel of the input image within the interpolation range based on the detection result includes: The weight value corresponding to each original pixel within the interpolation range is determined based on the first detection result and the second detection result.

15. An upsampling device based on multi-type edge detection, used to output a target image based on an input image, characterized in that, The upsampling device includes: An edge detection module is used to perform edge detection on the input image using multiple different edge detection methods to obtain detection results, including: performing edge detection on the input image using a first edge detection method to obtain an edge type; performing edge detection on the input image using a second edge detection method to obtain a first image gradient; and performing edge detection on the input image using a third edge detection method to obtain a second image gradient. The weight determination module is used to determine the weight value corresponding to each original pixel of the input image within the interpolation range based on the detection result, including determining the weight value corresponding to each original pixel based on the first image gradient and the second image gradient; An interpolation selection module is used to determine an interpolation method for interpolating the input image based on the detection result, including determining the interpolation method based on the edge type; An interpolation processing module is used to perform interpolation processing on the input image according to the interpolation method and the weight value corresponding to each original pixel to obtain the target image.

16. An electronic device, characterized in that, The electronic device includes one or more processors and a memory, the memory storing a computer program that, when executed by the processor, implements the upsampling method based on multi-type edge detection as described in any one of claims 1-14.

17. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the upsampling method based on multi-type edge detection as described in any one of claims 1-14.

Images