Image difference value method and device, electronic equipment and storage medium

Abstract

The invention provides an image interpolation method and device, electronic equipment and a storage medium, and the method comprises the steps: obtaining a coordinate position of a to-be-interpolated target interpolation point in an original image, determining a reference pixel point corresponding to the position of the target interpolation point from the original image according to the coordinate position of the target interpolation point in the original image, in response to the fact that the reference pixel point is an edge pixel, determining a main interpolation direction and a secondary interpolation direction corresponding to the reference pixel point, and in response to the fact that the reference pixel point has the main interpolation direction and the secondary interpolation direction, performing fusion interpolation on the target interpolation point according to the main interpolation direction and the secondary interpolation direction of the reference pixel point, according to the method, the target interpolation point can be subjected to interpolation calculation in different directions in the region where the target interpolation point is located during interpolation, so that the problem of discontinuous transition at the edge junction is relieved, the edge continuity is better, and the problems of artifacts and aliasing are avoided.

Description

Technical Field

[0001] This application relates to the field of image processing technologies, and in particular, to an image interpolation method, apparatus, electronic device, and storage medium. Background Art

[0002] In a video display system, due to the diverse resolutions of video sources and the increasing resolution of display devices, before a video input signal is sent to a flat panel display device for display, image super-resolution reconstruction is required, that is, an image processing technology that performs interpolation processing on a low-resolution image or image sequence to restore a high-resolution image and make it an image format supported by the display screen. With the continuous improvement of the size and resolution of the display panel, image interpolation technology is becoming increasingly important in the image processing system.

[0003] In related technologies, during the process of super-resolution reconstruction, obvious blurring or aliasing will occur in the image edge region, thus failing to meet actual requirements. Summary of the Invention

[0004] This application aims to at least solve one of the technical problems in related technologies to some extent.

[0005] To this end, this application proposes an image interpolation method, apparatus, electronic device, and storage medium to reduce the problem of discontinuous transition at the edge junction, make the edge continuity better, and avoid the problems of artifacts and aliasing.

[0006] An embodiment of one aspect of this application proposes an image interpolation method, including:

[0007] Obtain the coordinate position of a target interpolation point to be interpolated in the original image;

[0008] Determine a reference pixel point corresponding to the position of the target interpolation point in the original image according to the coordinate position of the target interpolation point in the original image;

[0009] In response to the reference pixel point being an edge pixel, determine the main interpolation direction and the secondary interpolation direction corresponding to the reference pixel point;

[0010] In response to determining that the reference pixel point has the main interpolation direction and the secondary interpolation direction, perform fusion interpolation on the target interpolation point according to the main interpolation direction and the secondary interpolation direction of the reference pixel point.

[0011] An embodiment of another aspect of this application proposes an image interpolation apparatus, including:

[0012] An acquisition module, configured to obtain the coordinate position of a target interpolation point to be interpolated in the original image;

[0013] A first determination module, configured to determine, according to the coordinate position of the target interpolation point in the original image, a reference pixel point corresponding to the position of the target interpolation point from the original image;

[0014] A second determination module, configured to determine a main interpolation direction and a secondary interpolation direction corresponding to the reference pixel point in response to the reference pixel point being an edge pixel;

[0015] An interpolation module, configured to perform fusion interpolation on the target interpolation point according to the main interpolation direction and the secondary interpolation direction of the reference pixel point in response to determining that the main interpolation direction and the secondary interpolation direction exist for the reference pixel point.

[0016] Another embodiment of this application provides an electronic device, including a memory, a processor, and a computer program stored on the memory and executable on the processor. When the processor executes the program, the method described in the foregoing aspect is implemented.

[0017] Another embodiment of this application provides a non-transitory computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the method described in the foregoing aspect is implemented.

[0018] Another embodiment of this application provides a computer program product, on which a computer program is stored. When the program is executed by a processor, the method described in the foregoing aspect is implemented.

[0019] The image interpolation method, apparatus, electronic device, and storage medium provided in this application obtain the coordinate position of a target interpolation point to be interpolated in an original image, determine, according to the coordinate position of the target interpolation point in the original image, a reference pixel point corresponding to the position of the target interpolation point from the original image, determine a main interpolation direction and a secondary interpolation direction corresponding to the reference pixel point in response to the reference pixel point being an edge pixel, and perform fusion interpolation on the target interpolation point according to the main interpolation direction and the secondary interpolation direction of the reference pixel point in response to determining that the main interpolation direction and the secondary interpolation direction exist for the reference pixel point, so that when interpolating the target interpolation point, interpolation calculations are performed comprehensively in different directions within the area where the target interpolation point is located, thereby alleviating the problem of discontinuous transition at the edge junction, making the edge continuity better, and avoiding the problems of artifacts and aliasing.

[0020] Additional aspects and advantages of this application will be given in part in the following description, become apparent in part from the following description, or be learned through the practice of this application. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] The above and / or additional aspects and advantages of this application will become apparent and be readily understood from the following description of the embodiments in conjunction with the drawings, where:

[0022] Figure 1 Schematic flowchart of another image interpolation method provided by an embodiment of the present application;

[0023] Figure 2 Schematic flowchart of another image interpolation method provided by an embodiment of the present application;

[0024] Figure 3 Schematic diagram of gradient operator matrices in all directions provided by an embodiment of the present application;

[0025] Figure 4 Schematic diagram of determining weights of pixel points participating in interpolation provided by an embodiment of the present application;

[0026] Figure 5 Schematic flowchart of another image interpolation method provided by an embodiment of the present application;

[0027] Figure 6A Schematic diagram of a pixel window for interpolation provided by an embodiment of the present application;

[0028] Figure 6B Schematic diagram of another pixel window for interpolation provided by an embodiment of the present application;

[0029] Figure 7 Schematic diagram of determining weights of pixel points participating in interpolation provided by an embodiment of the present application;

[0030] Figure 8 Schematic diagram of comparing interpolation results provided by an embodiment of the present application;

[0031] Figure 9 Schematic diagram of the structure of an image interpolation algorithm provided by an embodiment of the present application;

[0032] Figure 10 Schematic diagram of the structure of an electronic device provided by an embodiment of the present application. Detailed implementation manners

[0033] The embodiments of the present application will be described in detail below. Examples of the embodiments are shown in the accompanying drawings, where the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and are intended to explain the present application, and should not be construed as limiting the present application.

[0034] The image interpolation method, device, electronic device, and storage medium of the embodiments of the present application will be described below with reference to the accompanying drawings.

[0035] Figure 1 Schematic flowchart of an image interpolation method provided by an embodiment of the present application.

[0036] The execution subject of the image interpolation method in the embodiments of the present application is an image interpolation device, which can be set in an electronic device. The electronic device can be a smart phone, a tablet computer, etc., and is not limited in this embodiment.

[0037] As Figure 1 shown, the method may include the following steps:

[0038] Step 101, obtain the coordinate position of the target interpolation point to be interpolated in the original image.

[0039] In a scenario where image magnification is required, one scenario is to perform super-resolution reconstruction to adapt to the high resolution of the display device, such as the scenario of displaying low-resolution pictures on the high-resolution screens of mobile phones and tablets. Another scenario is to implement digital zoom by the user on the terminal device, such as the scenario of real-time magnification of the picture during mobile phone photography, video recording, and preview.

[0040] Among them, the original image is the input image, which is the image to be subjected to difference processing. According to the transformation ratio between the original image and the image after difference, the coordinate positions of multiple interpolation points to be pixel-interpolated in the original image can be determined. Among them, the target interpolation point is one of the multiple interpolation points to be interpolated.

[0041] Step 102, determine the reference pixel point corresponding to the target interpolation point position in the original image according to the coordinate position of the target interpolation point in the original image.

[0042] In an implementation manner of the embodiments of the present application, the target interpolation point position is rounded down to obtain a coordinate position in the original image, and the pixel in the original image corresponding to this coordinate position is the reference pixel point. Among them, the reference pixel point refers to the interpolation of the target interpolation point, and the corresponding interpolation method is determined based on the edge characteristics of the reference pixel point.

[0043] Step 103, in response to the reference pixel point being an edge pixel, determine the main interpolation direction and the secondary interpolation direction corresponding to the reference pixel point.

[0044] In the embodiments of the present application, in response to the reference pixel point being an edge pixel, the interpolation direction corresponding to the reference pixel point is determined. The interpolation direction includes the main interpolation direction and the secondary interpolation direction. By determining the interpolation direction corresponding to the reference pixel point, the interpolation directions of each reference pixel point in the image can be determined, and then interpolation processing can be performed based on the interpolation directions of each reference pixel point, so as to avoid the situation of only interpolating along a certain direction in the complex texture area of the image.

[0045] In an implementation of the embodiment of the present application, according to the position of the reference pixel point in the original image, a third pixel window including the reference pixel point is determined in the original image, where the third pixel window has a set size, for example, a size of 4*4. The number of edge pixels included in the third pixel window is counted as the first number. In response to the first number being greater than the set number threshold, according to the directions of the maximum gradient amplitudes corresponding to each edge pixel, the first set direction with the largest number and the second set direction with the second largest number are determined. The first set direction with the largest number is used as the main interpolation direction, and the second set direction with the second largest number is used as the secondary interpolation direction. That is to say, the main interpolation direction refers to the direction with the largest number of directions among the directions of the maximum gradient amplitudes corresponding to the edge pixels included in the third pixel window, and the secondary interpolation direction refers to the direction with the second largest number of directions among the directions of the maximum gradient amplitudes corresponding to the edge pixels included in the third pixel window. Step 104, in response to determining that the reference pixel point has a main interpolation direction and a secondary interpolation direction, perform fusion interpolation on the target interpolation point according to the main interpolation direction and the secondary interpolation direction of the reference pixel point.

[0046] In the embodiment of the present application, in response to determining that the reference pixel point has an interpolation direction, that is, determining that the reference pixel point needs to perform directional interpolation, fusion interpolation is performed on the target interpolation point according to the main interpolation direction and the secondary interpolation direction of the reference pixel point, so that the target interpolation point at the edge junction in different directions will perform interpolation calculations by synthesizing different directions in this area, thereby alleviating the problem of discontinuous transition at the edge junction. Especially in the scene of large magnification difference, such a fusion processing method can eliminate the fault problem caused by edge interpolation and improve the effect of edge difference.

[0047] In the image interpolation method of the embodiment of the present application, the coordinate position of the target interpolation point to be interpolated in the original image is obtained. According to the coordinate position of the target interpolation point in the original image, a reference pixel point corresponding to the position of the target interpolation point is determined from the original image. In response to the reference pixel point being an edge pixel, the main interpolation direction and the secondary interpolation direction corresponding to the reference pixel point are determined. In response to determining that the reference pixel point has a main interpolation direction and a secondary interpolation direction, fusion interpolation is performed on the target interpolation point according to the main interpolation direction and the secondary interpolation direction of the reference pixel point, so that the target interpolation point will perform interpolation calculations by synthesizing different directions in the area where the target interpolation point is located, thereby alleviating the problem of discontinuous transition at the edge junction, making the edge continuity better, and avoiding problems such as artifacts and aliasing.

[0048] Based on the above embodiments, Figure 2 is a schematic flowchart of another image difference method provided by the embodiment of the present application. As Figure 2 shown, the method includes the following steps:

[0049] Step 201: Obtain the coordinate position of the target interpolation point to be interpolated in the original image.

[0050] Step 202: Determine the reference pixel points corresponding to the target interpolation point in the original image according to the coordinate position of the target interpolation point in the original image.

[0051] Among them, Step 201 and Step 202 can refer to the explanations in the foregoing embodiments. The principles are the same and will not be elaborated here.

[0052] Step 203: Determine the first pixel window including the reference pixel points in the original image according to the position of the reference pixel points in the original image.

[0053] As an implementation manner, a first pixel window with a first set size can be determined centered on the reference pixel points. For example, the size of the first pixel window is 5*5, that is, the first pixel window is a 5*5 pixel matrix including 25 pixels.

[0054] It should be understood that the larger the pixel window, the greater the calculation complexity, and the smaller the pixel window, the lower the calculation complexity but also the lower the accuracy. Therefore, in the embodiments of the present application, the first pixel window is selected as a 5*5 pixel matrix based on a balanced consideration of meeting the accuracy requirements and calculation complexity.

[0055] Step 204: Obtain the gradient operator matrix in at least one set direction corresponding to the first pixel window.

[0056] In an implementation manner of the embodiments of the present application, the first pixel window includes the target difference point to be differentiated and the reference pixel points, and the gradient operator matrix in at least one set direction is obtained.

[0057] As an example, the pixels in the first pixel window are a 5*5 pixel matrix, and there are at least eight set directions. Each set direction is indicated by an angle, and the eight set directions correspond to 0°, 22.5°, 45°, 67.5°, 90°, 112.5°, 135° and 157.5° respectively. Figure 3 This is a schematic diagram of the gradient operator matrix in each direction provided in the embodiments of the present application. The gradient operator matrix is a 5*5 element matrix. Among them, each element in the gradient operator matrix is used to indicate the weight of each pixel in the first pixel window.

[0058] Step 205: For each set direction, add the pixels in the first pixel window and the weights indicating each pixel in the gradient operator matrix in the set direction by weighting to obtain the gradient amplitude in the set direction.

[0059] Step 206: Determine the total gradient amplitude of the reference pixel points according to the sum of the gradient amplitudes in at least one set direction.

[0060] As an example, there are at least 8 set directions. The gradient magnitudes in the 8 directions are added together to obtain the total gradient magnitude in the 8 directions, which is used as the total gradient magnitude of the reference pixel point.

[0061] Step 207: Determine the edge threshold corresponding to the reference pixel point, and compare the total gradient magnitude of the reference pixel point with the edge threshold corresponding to the reference pixel point.

[0062] In an implementation manner of this application, the edge threshold corresponding to the reference pixel point is a set edge threshold, that is, each pixel point in the image can determine whether it is an edge pixel based on the set edge threshold.

[0063] In another implementation manner of the embodiments of this application, in order to improve the accuracy of determining edge pixel points, an edge threshold matching the reference pixel point can be determined for each reference pixel point. As an implementation manner, obtain a set first threshold, where the first threshold is the threshold corresponding to each pixel point in the set original image. Furthermore, determine the other pixel points in the second pixel window centered on the reference pixel point except the reference pixel point, and determine the average total gradient magnitude of the other pixel points according to the total gradient magnitudes of the other pixel points in at least one set direction. Compare the average total gradient magnitude with the first threshold. In response to the average total gradient magnitude being greater than the first threshold, use the average total gradient magnitude as the edge threshold corresponding to the reference pixel point; in response to the average total gradient magnitude being less than or equal to the first threshold, use the first threshold as the edge threshold corresponding to the reference pixel point. Based on the average total gradient magnitudes of multiple pixel points related to the reference pixel point in the set window, the edge threshold corresponding to the reference pixel point is determined, which improves the accuracy of determining the edge threshold corresponding to the reference pixel point. At the same time, for different reference pixel points, an adaptive edge threshold corresponding to the reference pixel point can be determined, which improves the accuracy of determining the edge threshold.

[0064] As an example, the second pixel window is a window of a second set size, and the window of the second set size is smaller than the window of the first set size. For example, it is a 3*3 pixel matrix. Then, the total gradient magnitudes of the 8 pixel points except the reference pixel point are averaged to obtain the average total gradient magnitude corresponding to the 8 pixel points. Furthermore, compare the average total gradient magnitude with the first threshold, and take the larger value of the average total gradient magnitude and the first threshold as the edge threshold corresponding to the reference pixel point, which improves the accuracy of determining the edge threshold of each reference pixel point, and further improves the accuracy of determining edge pixels.

[0065] It should be understood that the larger the pixel window is, the greater the computational complexity is. When determining whether a reference pixel point is an edge pixel, it is not necessary to set a large pixel window to accurately identify the edge pixel, and at the same time, the requirement for the computing power of the hardware can be reduced.

[0066] Step 208: In response to the total gradient amplitude of the reference pixel point being greater than the edge threshold corresponding to the reference pixel point, determine that the reference pixel point is an edge pixel point.

[0067] In the embodiment of the present application, whether the reference pixel point is an edge pixel is determined by the edge threshold corresponding to the reference pixel point, that is, the total gradient amplitude of the reference pixel is compared with the edge threshold corresponding to the reference pixel point. If the total gradient amplitude of the reference pixel is greater than the edge threshold corresponding to the reference pixel point, it is determined that the reference pixel point is an edge pixel, and then the edge direction for interpolation of the edge pixel is determined.

[0068] Otherwise, in response to the total gradient amplitude of the reference pixel point being less than or equal to the edge threshold, determine that the reference pixel point is a non-edge pixel point, and then directly use the second interpolation method, such as the Lanczos3 interpolation method for interpolation. The specific interpolation method will be introduced in detail in the subsequent steps.

[0069] Step 209: In response to identifying that the reference pixel point is an edge pixel, determine a third pixel window including the reference pixel point in the original image according to the position of the reference pixel point in the original image.

[0070] As an example, in the original image, a third pixel window centered on the position of the reference pixel point is determined. For example, the third pixel window is a 4*4 pixel matrix.

[0071] Step 210: Determine the most numerous first set direction and the second most numerous second set direction according to the set directions corresponding to the maximum gradient amplitudes of the respective edge pixels included in the third pixel window.

[0072] Among them, the set quantity threshold can be set based on requirements.

[0073] As an example, the third pixel window includes 16 pixels, and the set quantity threshold is 9. Then the first quantity is at least 9, for example, 14, and they are numbered 1-14. Among them, the set directions corresponding to the maximum gradient amplitudes of the respective edge pixels are as shown in Table 1 below:

[0074] Table 1:

[0075]

[0076]

[0077] Among them, there are 9 edge pixels with the direction corresponding to the maximum gradient amplitude being 22.5 degrees, and 4 edge pixels with the direction corresponding to the maximum gradient amplitude being 45 degrees. Therefore, it is determined that the direction with the largest number among the directions corresponding to the maximum gradient amplitude is the direction corresponding to 22.5 degrees. Therefore, the direction corresponding to 22.5 degrees is the first set direction, and the direction with the second largest number is the direction corresponding to 45 degrees. Therefore, the direction corresponding to 45 degrees is the second set direction.

[0078] Step 211: Determine the main interpolation direction and the secondary interpolation direction corresponding to the reference pixel point according to the first set direction and the second set direction.

[0079] As an implementation method, the first set direction with the largest number is used as the main interpolation direction, and the second set direction with the second largest number is used as the secondary interpolation direction. For example, it is determined that the direction corresponding to 22.5 degrees is the main interpolation direction, and the direction corresponding to 45 degrees is the secondary interpolation direction.

[0080] As another implementation method, in order to improve the accuracy of determining the main interpolation direction and the secondary interpolation direction, it is also necessary to further screen based on the determined first set direction to determine whether the first set direction can be used as the main interpolation direction.

[0081] In the embodiments of the present application, for each pixel point in the original image, the method of determining the edge pixels in the above steps can be used to determine the set direction corresponding to the maximum gradient amplitude and whether it is an edge pixel. Thus, the pixel points included in the third pixel window can be counted to determine the number of edge pixels, denoted as the first number. And determine the second number of edge pixels corresponding to the first set direction, and determine the third set direction adjacent to the first set direction from the directions of the maximum gradient amplitudes corresponding to each edge pixel, and determine the total number of edge pixels corresponding to the third set direction as the third number, where there can be multiple third set directions adjacent to the first set direction. For example, there are 4 directions, including the two directions adjacent to the left of the first set direction and the two directions adjacent to the right of the first set direction. For example, if the first set direction is 45 degrees, the third set directions adjacent to the first set direction include 0 degrees, 22.5 degrees, 62.5 degrees, and 90 degrees.

[0082] Furthermore, in response to the first number being greater than or equal to the set number threshold, and the sum of the second number and the third number being greater than or equal to the first number, and the second number being greater than half of the first number, determine the first set direction as the main interpolation direction, and determine the second set direction as the secondary interpolation direction.

[0083] It should be understood that if the second quantity is greater than half of the first quantity, it indicates that the edge directions of half or more of the edge pixel points in the third pixel window are the same as the edge direction of the reference pixel point. Therefore, it shows that the interpolation directions of the edge pixels in the third pixel window are relatively consistent. Further, if the sum of the second quantity and the third quantity is greater than or equal to the first quantity, it means that the directions of the edge pixel points other than the edge pixel points in the first set direction in the third pixel window are all adjacent to the first set direction, that is, they are all relatively consistent with the first set direction. Thus, it is relatively accurate to use the first set direction as the main interpolation direction of the reference pixel point, and thus, it is also relatively accurate to use the second set direction as the secondary interpolation direction.

[0084] As an example, the third pixel window is a 4*4 pixel window. It is statistically determined that the number of edge pixels in the third pixel window is 14. The direction of the maximum gradient amplitude of the reference pixel point is the direction of 22.5 degrees, and it is statistically obtained that the number of edge pixel points included in the direction of 22.5 degrees is 9. The directions adjacent to 22.5 degrees include 45 degrees, 67.5 degrees, 0 degrees, and 157.5 degrees. By calculation, it is determined that the number of edge pixels in the 4 set directions of 45 degrees, 67.5 degrees, 0 degrees, and 157.5 degrees is 5. Then, the total number of edge pixels in the 5 set directions of 22.5 degrees, 45 degrees, 67.5 degrees, 0 degrees, and 157.5 degrees is 14, which is equal to the number of edge pixels in the first pixel window. Therefore, based on the main interpolation direction and the secondary interpolation direction corresponding to the reference pixel point, the main interpolation direction for direction interpolation of the target interpolation point is the direction of 22.5 degrees, and the secondary interpolation direction is the direction of 45 degrees, achieving accurate determination of the corresponding main interpolation direction and secondary interpolation direction for each reference pixel point, avoiding the situation of interpolation only along a certain direction in complex texture regions, and improving the accuracy and robustness of the determination of the interpolation direction of each pixel point.

[0085] Step 212, in response to determining that the reference pixel point has a main interpolation direction and a secondary interpolation direction, perform fusion interpolation on the target interpolation point according to the main interpolation direction and the secondary interpolation direction of the reference pixel point.

[0086] Among them, step 212 can refer to the explanation in the foregoing embodiments, with the same principle, and will not be elaborated here.

[0087] Step 213, in response to the first quantity being less than the set quantity threshold, or the sum of the second quantity and the third quantity being less than the first quantity, or the second quantity being less than or equal to half of the first quantity, determine that the reference pixel point does not have a main interpolation direction and a secondary interpolation direction.

[0088] Wherein, the first quantity is less than a set quantity threshold, the sum of the second quantity and the third quantity is less than the first quantity, and the second quantity is less than or equal to half of the first quantity. When one or more of these three conditions are met, it is determined that there is no main interpolation direction and sub-interpolation direction for the reference pixel point.

[0089] In the embodiments of the present application, if there is no main interpolation direction and sub-interpolation direction for the reference pixel point of the target interpolation point, an interpolation algorithm that does not consider the interpolation direction is used for interpolation. Among them, the interpolation algorithm that does not consider the interpolation direction, for example, is Lanczos3 interpolation. Then, all pixel points in the third pixel window except the target interpolation point are used as pixel points participating in interpolation, and interpolation calculation is performed on the target interpolation point. Taking the Lanczos3 interpolation algorithm as an example, the weights of the pixel points participating in interpolation are determined in the following manner:

[0090] Figure 4 It is a schematic diagram for determining the weights of pixel points participating in interpolation based on the Lanczos3 interpolation algorithm provided by the embodiments of the present application, as Figure 4 shown. The third pixel window is a pixel matrix of 6*6 size, including 36 pixel points. Among them, the large black dots indicate the reference pixel points in the original image, and the small black dots indicate the target interpolation points to be pixel-interpolated. Among them, all 36 pixel points are pixel points participating in interpolation calculation. The weights of each pixel point are determined in the following manner:

[0091] The weight of each pixel point participating in interpolation = horizontal direction weight * vertical direction weight. Taking the horizontal direction weight as an example, draw a perpendicular line through the pixel point participating in interpolation to intersect with the horizontal line, and the intersection point is represented by a golden triangle. The distances between the intersection point and the 6 pixel points participating in interpolation numbered 1'-6' on the horizontal line are represented by dx1~dx6 respectively. According to dx1~dx6 and the corresponding weight formula, the horizontal direction weights of the 6 pixel points participating in interpolation are calculated in turn.

[0092] If dx = 0, then w_lanczos3 = 1;

[0093] If 0 < dx < 3, then w_lanczos3 = (sin(π*dx)*sin(π / 3*dx) / (π*π / 3)*dx*dx);

[0094] If dx >= 3, then w_lanczos3 = 0;

[0095] As Figure 4 shown, the distance between adjacent two pixel points is 1. As can be seen from Figure 4 , the dx distances of the 6 pixel points participating in interpolation numbered 1'-6' are all values greater than 0 and less than 3. Therefore, the weight of each pixel point is determined by the following formula:

[0096] w_lanczos3 = (sin(π * dx) * sin(π / 3 * dx) / (π * π / 3) * dx * dx);

[0097] For example, if the distance corresponding to the pixel point labeled 1 is dx, then the corresponding weight is:

[0098] w1_lanczos3 = (sin(π * dx1) * sin(π / 3 * dx1) / (π * π / 3) * dx1 * dx1);

[0099] Furthermore, normalize the horizontal weights of all the pixel points participating in the interpolation to obtain the normalized horizontal weights.

[0100] Similarly, determine the weights of each pixel point participating in the interpolation in the vertical direction. The determination method of the weights in the vertical direction is similar to that in the horizontal direction and will not be elaborated here.

[0101] Furthermore, the weight of the pixel point participating in the interpolation = the normalized horizontal weight * the normalized vertical weight. Thus, the weight of each pixel point participating in the interpolation can be determined.

[0102] Furthermore, based on the Lanczos3 interpolation algorithm, determine the pixel value of the target interpolation point based on the weights and pixel values of the pixel points participating in the interpolation.

[0103] In the image interpolation method of the embodiment of the present application, edge pixel judgment and edge direction judgment are performed on the reference pixel points, which have high robustness and accuracy, and can avoid the situation of only interpolating along a certain direction in complex texture areas, improving the accuracy of interpolation.

[0104] Based on the above embodiments, the embodiment of the present application provides another image interpolation method. Figure 5 It is a schematic flowchart of another image interpolation method provided by the embodiment of the present application, as Figure 5 shown. This method includes the following steps:

[0105] Step 501, obtain the coordinate position of the target interpolation point to be interpolated in the original image.

[0106] Step 502, determine the reference pixel point corresponding to the position of the target interpolation point in the original image according to the coordinate position of the target interpolation point in the original image.

[0107] Step 503, in response to the reference pixel point being an edge pixel, determine the main interpolation direction and the secondary interpolation direction corresponding to the reference pixel point.

[0108] Among them, steps 501 to 503 can refer to the explanations in the foregoing embodiments. Since the principles are the same, they will not be elaborated here.

[0109] Step 504, in response to determining that there are a main interpolation direction and a secondary interpolation direction for the reference pixel point, perform interpolation on the target interpolation point according to the coordinate position of the target interpolation point, the coordinate position of the reference pixel point, and the corresponding main interpolation direction in the original image to obtain a first interpolation result.

[0110] In the embodiment of the present application, according to the offset of the coordinate position of the target interpolation point relative to the coordinate position of the reference pixel point and the main interpolation direction, multiple pixels participating in interpolation in the fourth pixel window are determined, where the multiple pixels participating in interpolation are all pixels in the original image.

[0111] As an example, as Figure 6A and 6B shown, the fourth pixel window is a 6*6 pixel matrix, including 36 pixels. Among them, the large black dot indicates the reference pixel point in the original image, and the small black dot indicates the target interpolation point to be pixel-interpolated. The figure respectively shows the multiple pixels participating in interpolation for the target interpolation point in the fourth pixel window when the main interpolation directions are 22.5 degrees, 45 degrees, 62.5 degrees, 112.5 degrees, 135 degrees, and 157.5 degrees. Among them, the light gray squares indicate the multiple pixels participating in interpolation. According to the pixel values of the multiple pixels participating in interpolation, the pixel value of the target interpolation point is determined based on cubic interpolation and linear interpolation. Figure 6A and 6B show different main interpolation directions. When the relative positions between the target interpolation point and the reference pixel point are different, the pixels participating in interpolation determined from the fourth pixel window are also different, specifically in terms of quantity and position. Among them, when the relative positions between the target interpolation point and the reference pixel point are different in different interpolation directions, the pixels participating in residual interpolation can be determined based on empirical values, which are pixels with a set quantity and position close to the position of the target interpolation point.

[0112] As an example, taking the main interpolation direction of 22.5 degrees as an example, where kx represents the horizontal coordinate offset between the target interpolation point and the window center pixel point (reference pixel point), and ky represents the vertical coordinate offset between the target interpolation point and the reference pixel point. If (2*ky + kx >= 1) and (2*ky + kx <= 2), then the multiple pixels participating in interpolation determined are as Figure 6A shown by the 8 light gray squares numbered 1-8 in

[0113] Furthermore, for each pixel point participating in interpolation, according to the position of the target interpolation point, the position of the pixel point participating in interpolation, and the main interpolation direction, determine the weight corresponding to the interpolation method adopted by the pixel point participating in interpolation. Then, perform interpolation on the target interpolation point according to the weights of each pixel point participating in interpolation and the corresponding interpolation method to obtain the first interpolation result.

[0114] In a scenario, if the interpolation methods adopted by the pixel points participating in interpolation are the cubic interpolation method and the linear interpolation method, the weights of the pixel points participating in interpolation are determined in the following manner:

[0115] Taking the main interpolation direction of 22.5 degrees as an example, when (2*ky + kx >= 1) and (2*ky + kx <= 2), the numbers of the pixel points participating in interpolation are 1 - 8. Among them, the pixel points participating in interpolation numbered 1 - 4 adopt the cubic interpolation method, and the pixel points participating in interpolation numbered 5 - 8 also adopt the cubic interpolation method. Among them, the cubic interpolation method needs to be based on 4 pixel points, and the linear interpolation method needs to be based on 2 pixel points.

[0116] As Figure 7 shown, taking the pixel points participating in interpolation numbered 1 - 4 as an example, the horizontal distance from the intersection point of the extension line of the target interpolation point along the corresponding main interpolation direction and the horizontal line where the 4 to - be - interpolated pixel points participating in interpolation calculation are located to each of these to - be - interpolated pixel points is dx. Among them, the dx from the intersection point to each pixel point participating in interpolation is different. Therefore, the weights of the 4 to - be - interpolated pixel points are also different. Among them, according to the distance of dx, the formula for determining the weight is as follows:

[0117] If dx <= 1, w = (1.25*dx – 2.25)*dx*dx + 1;

[0118] If 1 < dx < 2, w = ((-0.75*dx + 3.75)*dx - 6)*dx + 3;

[0119] If dx >= 2, w = 0.

[0120] When the main interpolation direction is 22.5°, draw an auxiliary line through the target interpolation point along the 22.5° direction and intersect with the horizontal line where the pixels participating in interpolation calculation (4 pixel points participating in interpolation numbered 1 - 4) are located. The intersection point is represented by a triangle. The distances from the intersection point to the 4 pixel points participating in interpolation numbered 1 - 4 on the horizontal line are represented by dx1, dx2, dx3, and dx4 respectively. Among them, the distance between adjacent two pixel points is 1. It can be seen from the figure that these distances satisfy the following conditions:

[0121] dx1 = dx2 + 1;

[0122] dx2 + dx3 = 1;

[0123] dx4 = dx3 + 1.

[0124] Wherein, dx1 = dx2 + 1, dx2 is less than 1, that is, 1 < dx1 < 2, then the weight w1 of the pixel point participating in interpolation numbered 1 is w1 = ((-0.75 * dx1 + 3.75) * dx2 - 6) * dx1 + 3;

[0125] Since dx2 is less than 1, the weight w2 of the pixel point participating in interpolation numbered 2 is w2 = (1.25 * dx2 – 2.25) * dx2 * dx2 + 1;

[0126] Since dx3 is less than 1, the weight w3 of the pixel point participating in interpolation numbered 3 is w3 = (1.25 * dx3 – 2.25) * dx3 * dx3 + 1;

[0127] Since 1 < dx4 < 2, the weight w4 of the pixel point participating in interpolation numbered 4 is w4 = ((-0.75 * dx4 + 3.75) * dx4 - 6) * dx4 + 3;

[0128] Furthermore, according to the weights and pixel values of the 4 pixel points participating in interpolation numbered 1 - 4, a cubic interpolation method is used for weighted calculation to determine the first sub - interpolation result of the cubic interpolation of the 4 pixel points participating in interpolation. Similarly, the weights and pixel values of the 4 pixel points participating in interpolation numbered 5 - 8 are determined, and a cubic interpolation method is used for weighted calculation to determine the second sub - interpolation result. Furthermore, the first sub - interpolation result and the second sub - interpolation result are used to determine the corresponding weights by linear interpolation. Among them, for linear interpolation, based on the positions of the triangles corresponding to the first interpolation result, that is, the triangle between the light - gray squares marked 2 and 3, and the positions of the triangles corresponding to the second interpolation result, that is, the triangle between the light - gray squares marked 6 and 7, the distance between them is used to determine the weights of the linear interpolation corresponding to the first interpolation result and the second interpolation result, and then weighted addition is performed to determine the first interpolation result.

[0129] It should be noted that if the main interpolation direction is other set directions except 0 degrees and 90 degrees, which can be called the first candidate set direction, the method for calculating the first interpolation result is the same as the method with the main interpolation direction of 22.5 degrees, and will not be elaborated here one by one. For the case of the second candidate set direction with the main interpolation directions of 0 degrees and 90 degrees, the interpolation method used is the Lanczos3 algorithm.

[0130] Step 505, determine the result of the fusion interpolation for the target interpolation point according to the number of sub - interpolation directions and the first interpolation result.

[0131] In the first implementation manner of the embodiment of the present application, in response to the number of secondary interpolation directions being one, the fourth quantity of edge pixel points in the first set direction and the fifth quantity of edge pixel points in the second set direction in the fourth pixel window are counted, and the result of fusion interpolation for the target interpolation point is determined according to the fourth quantity and the fifth quantity.

[0132] Among them, as an implementation manner, the set multiples of the fourth quantity and the fifth quantity are compared. In response to the fourth quantity being greater than or equal to the set multiple of the fifth quantity, for example, the set multiple is 8, max_ori_num >= sec_max_ori_num * 8, where max_ori_num is the fourth quantity and sec_max_ori_num is the fifth quantity. That is, the number of edge pixel points in the secondary interpolation direction is small and can be ignored. Therefore, the first interpolation result is used as the result of fusion interpolation for the target interpolation point.

[0133] As another implementation manner, the set multiples of the fourth quantity and the fifth quantity are compared. In response to the fourth quantity being less than the set multiple of the fifth quantity, interpolation is performed on the target interpolation point in the secondary interpolation direction to obtain a second interpolation result. The method of performing interpolation on the target interpolation point in the secondary interpolation direction to obtain the second interpolation result can refer to the method of the foregoing first interpolation result, and the principle is the same, so it will not be elaborated here. Since there are a certain number of edge pixels in the secondary interpolation direction, in order to improve accuracy, the first interpolation result and the second interpolation result are weighted and added to obtain the target interpolation result of the target interpolation point. Among them, the weight w1 of the first interpolation result is determined according to the following formula:

[0134] w1 = max_ori_num / (max_ori_num + sec_max_ori_num);

[0135] The weight of the second interpolation result is: 1 - w1.

[0136] Thus, the target interpolation result = w1 * the first interpolation result + the second interpolation result * (1 - w1).

[0137] Furthermore, the target interpolation result is used as the result of fusion interpolation for the target interpolation point.

[0138] In the second implementation manner of the embodiment of the present application, in response to the number of secondary interpolation directions being multiple, the Lanczos3 algorithm is used to perform interpolation on the target interpolation point to obtain a third interpolation result, and the result of fusion interpolation for the target interpolation point is determined according to the first interpolation result and the third interpolation result. As an implementation manner, the first interpolation result and the third interpolation result are averaged, and the average value is used as the result of fusion interpolation for the target interpolation point.

[0139] In the image interpolation method according to the embodiments of the present application, the interpolation results are fused by considering the primary interpolation direction and the secondary interpolation direction, and the fusion methods under different judgment conditions are given. This enables the target interpolation points to be interpolated at the edge junctions in different directions to comprehensively perform interpolation calculations in different directions within the region, thereby alleviating the problem of discontinuous transitions at the edge junctions. Especially at high magnification ratios, such a fusion processing method eliminates the fault problems caused by edge interpolation and improves the interpolation effect.

[0140] As an example, Figure 8 FIG. is a comparison schematic diagram of interpolation results provided by an embodiment of the present application. As Figure 8 shown, in the case of 4-fold magnification, the left side is the result obtained by interpolating using the image interpolation method of the present application, which avoids the fault and discontinuity problems caused by edge interpolation and improves the interpolation effect.

[0141] To implement the above embodiments, the embodiments of the present application also propose an image interpolation device.

[0142] Figure 9 FIG. is a schematic structural diagram of an image interpolation device provided by an embodiment of the present application.

[0143] As Figure 9 shown, the device may include:

[0144] An acquisition module 91, configured to acquire the coordinate position of a target interpolation point to be interpolated in the original image.

[0145] A first determination module 92, configured to determine, according to the coordinate position of the target interpolation point in the original image, a reference pixel point corresponding to the position of the target interpolation point from the original image.

[0146] A second determination module 93, configured to, in response to the reference pixel point being an edge pixel, determine the primary interpolation direction and the secondary interpolation direction corresponding to the reference pixel point.

[0147] An interpolation module 94, configured to, in response to determining that the reference pixel point has the primary interpolation direction and the secondary interpolation direction, perform fusion interpolation on the target interpolation point according to the primary interpolation direction and the secondary interpolation direction of the reference pixel point.

[0148] Further, in an implementation manner of the embodiments of the present application, the placement further includes:

[0149] A third determination module, configured to determine a first pixel window including the reference pixel point in the original image according to the position of the reference pixel point in the original image; obtain at least one gradient operator matrix in a set direction corresponding to the first pixel window; wherein each element in the gradient operator matrix is used to indicate the weight of each pixel in the first pixel window; for each set direction, perform weighted addition of each pixel in the first pixel window and the weights indicating each pixel in the gradient operator matrix in the set direction to obtain the gradient magnitude in the set direction; determine the total gradient magnitude of the reference pixel point according to the sum of the gradient magnitudes in the at least one set direction; determine the edge threshold corresponding to the reference pixel point; compare the total gradient magnitude of the reference pixel point with the edge threshold corresponding to the reference pixel point; and in response to the total gradient magnitude of the reference pixel point being greater than the edge threshold corresponding to the reference pixel point, determine that the reference pixel point is an edge pixel point.

[0150] In an implementation manner of the embodiment of the present application, the third determination module is further configured to determine a second pixel window centered on the reference pixel point in the original image, determine the average total gradient magnitude corresponding to the first pixel point according to the total gradient magnitudes of the first pixel points other than the reference pixel point in the second pixel window in at least one set direction, compare the average total gradient magnitude with a set first threshold, in response to the average total gradient magnitude being greater than the first threshold, use the average total gradient magnitude as the edge threshold corresponding to the reference pixel point, and in response to the average total gradient magnitude being less than or equal to the first threshold, use the first threshold as the edge threshold corresponding to the reference pixel point.

[0151] In an implementation manner of the embodiment of the present application, the second determination module 93 is further configured to:

[0152] Determine a third pixel window including the reference pixel point in the original image according to the position of the reference pixel point in the original image;

[0153] Determine a first set direction with the largest number and a second set direction with the second largest number according to the set direction corresponding to the maximum gradient magnitude of each edge pixel included in the third pixel window;

[0154] Determine the main interpolation direction and the secondary interpolation direction corresponding to the reference pixel point according to the first set direction and the second set direction.

[0155] In an implementation manner of the embodiment of the present application, the second determination module 93 is further configured to:

[0156] Determine a first number of edge pixels included in the third pixel window, and a second number of edge pixels corresponding to the first set direction;

[0157] Determine a third set direction adjacent to the first set direction from the directions of the maximum gradient magnitudes corresponding to the respective edge pixels;

[0158] Determine that the total number of edge pixels corresponding to the third set direction is the third quantity;

[0159] In response to the first quantity being greater than or equal to a set quantity threshold, and the sum of the second quantity and the third quantity being greater than or equal to the first quantity, and the second quantity being greater than half of the first quantity, determine the first set direction as the main interpolation direction and determine the second set direction as the secondary interpolation direction.

[0160] In an implementation manner of the embodiment of the present application, the second determination module 93 is further configured to:

[0161] In response to the first quantity being less than the set quantity threshold, or the sum of the second quantity and the third quantity being less than the first quantity, or the second quantity being less than or equal to half of the first quantity, determine that there is no main interpolation direction and secondary interpolation direction for the reference pixel point.

[0162] In an implementation manner of the embodiment of the present application, the interpolation module 94 is further configured to:

[0163] In response to determining that there are a main interpolation direction and a secondary interpolation direction for the reference pixel point, perform interpolation on the target interpolation point according to the coordinate position of the target interpolation point, the coordinate position of the reference pixel point, and the corresponding main interpolation direction in the original image to obtain a first interpolation result;

[0164] Determine the result of fusion interpolation for the target interpolation point according to the number of secondary interpolation directions and the first interpolation result.

[0165] In an implementation manner of the embodiment of the present application, the interpolation module 94 is further configured to:

[0166] In response to the number of secondary interpolation directions being one, determine a fourth pixel window corresponding to the reference pixel point in the original image according to the position of the reference pixel point in the original image;

[0167] Count the fourth quantity of edge pixel points in the main interpolation direction and the fifth quantity of edge pixel points in the secondary interpolation direction in the fourth pixel window;

[0168] Compare the set multiples of the fourth quantity and the fifth quantity;

[0169] In response to the fourth quantity being greater than or equal to a set multiple of the fifth quantity, use the first interpolation result as the result of fusion interpolation for the target interpolation point.

[0170] In one implementation manner of the embodiment of the present application, the interpolation module 94 is further configured to:

[0171] In response to the fourth quantity being less than the set multiple of the fifth quantity, perform interpolation on the target interpolation point in the secondary interpolation direction to obtain a second interpolation result;

[0172] Perform weighted addition on the first interpolation result and the second interpolation result to obtain the target interpolation result of the target interpolation point;

[0173] Use the target interpolation result as the result of fusion interpolation for the target interpolation point.

[0174] In one implementation manner of the embodiment of the present application, the interpolation module 94 is further configured to:

[0175] In response to the number of secondary interpolation directions being multiple, perform interpolation on the target interpolation point using an interpolation algorithm that does not consider the interpolation direction to obtain a third interpolation result;

[0176] Determine the result of fusion interpolation for the target interpolation point according to the first interpolation result and the third interpolation result.

[0177] In one implementation manner of the embodiment of the present application, the interpolation module 94 is further configured to:

[0178] According to the offset of the coordinate position of the target interpolation point relative to the coordinate position of the reference pixel point, and the primary interpolation direction, determine multiple pixel points participating in interpolation in the fourth pixel window;

[0179] For each pixel point participating in interpolation, determine the weight corresponding to the interpolation method used by the pixel point participating in interpolation according to the position of the target interpolation point, the position of the pixel point participating in interpolation, and the corresponding primary interpolation direction;

[0180] Perform interpolation on the target interpolation point according to the weights of the respective pixel points participating in interpolation and the corresponding interpolation methods to obtain a first interpolation result.

[0181] It should be noted that the foregoing explanation of the method embodiment is also applicable to the device of this embodiment, and will not be elaborated here.

[0182] In the image interpolation device according to the embodiment of the present application, the coordinate position of a target interpolation point to be interpolated in the original image is obtained. According to the coordinate position of the target interpolation point in the original image, reference pixel points corresponding to the position of the target interpolation point are determined from the original image. In response to the reference pixel points being edge pixels, the main interpolation direction and the secondary interpolation direction corresponding to the reference pixel points are determined. In response to determining that there are a main interpolation direction and a secondary interpolation direction for the reference pixel points, the target interpolation point is fused and interpolated according to the main interpolation direction and the secondary interpolation direction of the reference pixel points, so that when the target interpolation point is interpolated, interpolation calculations are performed comprehensively in different directions within the area where the target interpolation point is located, thereby alleviating the problem of discontinuous transition at the edge junction, making the edge continuity better, and avoiding the problems of artifacts and aliasing.

[0183] To implement the above embodiment, the present application also proposes an electronic device, including a memory, a processor, and a computer program stored on the memory and executable on the processor. When the processor executes the program, the method described in the foregoing method embodiment is implemented.

[0184] To implement the above embodiment, the present application also proposes a non-temporary computer-readable storage medium, on which a computer program is stored. When the program is executed by a processor, the method described in the foregoing method embodiment is implemented.

[0185] To implement the above embodiment, the present application also proposes a computer program product, on which a computer program is stored. When the computer program is executed by a processor, the method described in the foregoing method embodiment is implemented.

[0186] Figure 10 It is a block diagram of an electronic device provided by an embodiment of the present application. For example, the electronic device 800 may be a mobile phone, a computer, a digital broadcast terminal, a messaging device, a game console, a tablet device, a medical device, a fitness device, a personal digital assistant, etc.

[0187] Refer to Figure 10 , the electronic device 800 may include one or more of the following components: a processing component 802, a memory 804, a power component 806, a multimedia component 808, an audio component 810, an input / output (I / O) interface 812, a sensor component 814, and a communication component 816.

[0188] The processing component 802 generally controls the overall operation of the electronic device 800, such as operations associated with display, telephone calls, data communication, camera operations, and recording operations. The processing component 802 may include one or more processors 820 to execute instructions to complete all or part of the steps of the above-described methods. In addition, the processing component 802 may include one or more modules to facilitate the interaction between the processing component 802 and other components. For example, the processing component 802 may include a multimedia module to facilitate the interaction between the multimedia component 808 and the processing component 802.

[0189] The memory 804 is configured to store various types of data to support the operation of the electronic device 800. Examples of such data include instructions for any application or method operating on the electronic device 800, contact data, phone book data, messages, pictures, videos, etc. The memory 804 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk, or optical disk.

[0190] The power component 806 provides power to various components of the electronic device 800. The power component 806 may include a power management system, one or more power sources, and other components associated with generating, managing, and distributing power for the electronic device 800.

[0191] The multimedia component 808 includes a screen that provides an output interface between the electronic device 800 and the user. In some embodiments, the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen can be implemented as a touch screen to receive input signals from the user. The touch panel includes one or more touch sensors to sense touches, swipes, and gestures on the touch panel. The touch sensors can not only sense the boundaries of touch or swipe actions, but also detect the duration and pressure associated with the touch or swipe operations. In some embodiments, the multimedia component 808 includes a front camera and / or a rear camera. When the electronic device 800 is in an operating mode, such as a shooting mode or a video mode, the front camera and / or the rear camera can receive external multimedia data. Each of the front camera and the rear camera can be a fixed optical lens system or have a focal length and optical zoom capabilities.

[0192] The audio component 810 is configured to output and / or input audio signals. For example, the audio component 810 includes a microphone (MIC), which is configured to receive external audio signals when the electronic device 800 is in an operating mode, such as a call mode, a recording mode, and a voice recognition mode. The received audio signals can be further stored in the memory 804 or transmitted via the communication component 816. In some embodiments, the audio component 810 further includes a speaker for outputting audio signals.

[0193] The I / O interface 812 provides an interface between the processing component 802 and a peripheral interface module, and the peripheral interface module can be a keyboard, a click wheel, buttons, etc. These buttons can include but are not limited to: a home button, a volume button, a power button, and a lock button.

[0194] The sensor component 814 includes one or more sensors for providing status assessments of various aspects of the electronic device 800. For example, the sensor component 814 can detect the on / off state of the electronic device 800, the relative positioning of components, such as the display and keypad of the electronic device 800. The sensor component 814 can also detect a change in the position of the electronic device 800 or a component of the electronic device 800, the presence or absence of user contact with the electronic device 800, the orientation or acceleration / deceleration of the electronic device 800, and a change in the temperature of the electronic device 800. The sensor component 814 can include a proximity sensor configured to detect the presence of nearby objects without any physical contact. The sensor component 814 can also include a light sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, the sensor component 814 can further include an acceleration sensor, a gyroscope sensor, a magnetic sensor, a pressure sensor, or a temperature sensor.

[0195] The communication component 816 is configured to facilitate communication between the electronic device 800 and other devices in a wired or wireless manner. The electronic device 800 can access a wireless network based on a communication standard, such as WiFi, 4G, or 5G, or a combination thereof. In an exemplary embodiment, the communication component 816 receives a broadcast signal or broadcast-related information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communication component 816 further includes a near field communication (NFC) module to facilitate short-range communication. For example, the NFC module can be implemented based on radio frequency identification (RFID) technology, infrared data association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology, and other technologies.

[0196] In an exemplary embodiment, the electronic device 800 may be implemented by one or more application specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components for performing the above method.

[0197] In an exemplary embodiment, a non-transitory computer-readable storage medium including instructions is also provided, such as a memory 804 including instructions, and the above instructions can be executed by a processor 820 of the electronic device 800 to complete the above method. For example, the non-transitory computer-readable storage medium may be a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, etc.

[0198] In the description of this specification, the descriptions referring to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples", etc. mean that the specific features, structures, materials, or characteristics described in connection with the embodiment or example are included in at least one embodiment or example of the present application. In this specification, the schematic representations of the above terms are not necessarily directed to the same embodiment or example. Moreover, the specific features, structures, materials, or characteristics described may be combined in any one or more embodiments or examples in a suitable manner. In addition, without conflict, those skilled in the art may combine and combine the different embodiments or examples described in this specification and the features of the different embodiments or examples.

[0199] In addition, the terms "first" and "second" are used for descriptive purposes only and cannot be construed as indicating or implying relative importance or implicitly indicating the quantity of the indicated technical features. Thus, the features defined with "first" and "second" may explicitly or implicitly include at least one of the features. In the description of the present application, "a plurality of" means at least two, such as two, three, etc., unless otherwise specifically defined.

[0200] Any process or method description in a flowchart or described in other ways herein may be understood to represent a module, segment, or portion of code including one or more executable instructions for implementing a customized logic function or process, and the scope of the preferred embodiments of the present application includes additional implementations, where the functions may be executed in a substantially simultaneous manner or in a reverse order according to the involved functions, rather than in the order shown or discussed, which should be understood by those skilled in the art to which the embodiments of the present application belong.

[0201] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as a definable sequence list of executable instructions for implementing logical functions, and can be specifically implemented in any computer-readable medium for use by an instruction execution system, apparatus, or device (such as a computer-based system, a system including a processor, or other systems that can fetch and execute instructions from the instruction execution system, apparatus, or device), or used in conjunction with these instruction execution systems, apparatus, or devices. For the purposes of this specification, a "computer-readable medium" can be any device that can contain, store, communicate, propagate, or transport a program for use by or in conjunction with an instruction execution system, apparatus, or device. More specific examples (non-exhaustive list) of computer-readable media include the following: an electrical connection portion with one or more wirings (electronic device), a portable computer disk cartridge (magnetic device), a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber device, and a portable compact disc read-only memory (CDROM). Additionally, the computer-readable medium can even be paper or other suitable media on which the program can be printed, because the program can be obtained electronically, for example, by optically scanning the paper or other media, followed by editing, interpretation, or other appropriate processing as necessary, and then storing it in a computer memory.

[0202] It should be understood that various parts of the present application can be implemented using hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented using software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented using hardware, as in another embodiment, any one or a combination of the following techniques well known in the art can be used: discrete logic circuits having logic gate circuits for implementing logical functions on data signals, application-specific integrated circuits having appropriate combinational logic gate circuits, programmable gate arrays (PGAs), field programmable gate arrays (FPGAs), etc.

[0203] Those of ordinary skill in the art of this technology can understand that all or part of the steps carried by the method of the above embodiments can be completed by instructing relevant hardware through a program, and the program can be stored in a computer-readable storage medium. When the program is executed, it includes one or a combination of the steps of the method embodiments.

[0204] In addition, each functional unit in various embodiments of the present application may be integrated into a processing module, may exist physically alone for each unit, or two or more units may be integrated into one module. The above-mentioned integrated module may be implemented in the form of hardware or in the form of a software functional module. When the integrated module is implemented in the form of a software functional module and sold or used as an independent product, it may also be stored in a computer-readable storage medium.

[0205] The above-mentioned storage medium may be a read-only memory, a magnetic disk or an optical disc, etc. Although the embodiments of the present application have been shown and described above, it can be understood that the above embodiments are exemplary and should not be construed as limiting the present application. Those of ordinary skill in the art can make changes, modifications, substitutions and variations to the above embodiments within the scope of the present application.

Claims

1. An image interpolation method, characterized in that, it includes: Obtain the coordinate position of the target interpolation point to be interpolated in the original image; According to the coordinate position of the target interpolation point in the original image, determine the reference pixel points corresponding to the position of the target interpolation point from the original image; In response to the reference pixel point being an edge pixel, determine the main interpolation direction and the secondary interpolation direction corresponding to the reference pixel point; In response to determining that the main interpolation direction and the secondary interpolation direction exist for the reference pixel point, perform fusion interpolation on the target interpolation point according to the main interpolation direction and the secondary interpolation direction of the reference pixel point.

2. The method according to claim 1, characterized in that, before the step of, in response to identifying that the reference pixel point is an edge pixel, determining the main interpolation direction and the secondary interpolation direction corresponding to the reference pixel point, further includes: According to the position of the reference pixel point in the original image, determine a first pixel window including the reference pixel point in the original image; Obtain at least one gradient operator matrix in a set direction corresponding to the first pixel window; wherein, each element in the gradient operator matrix is used to indicate the weight of each pixel in the first pixel window; For each set direction, add the pixels in the first pixel window and the weights indicating the pixels in the gradient operator matrix in the set direction weighted, to obtain the gradient amplitude in the set direction; According to the sum of the gradient amplitudes in the at least one set direction, determine the total gradient amplitude of the reference pixel point; Determine the edge threshold corresponding to the reference pixel point; Compare the total gradient amplitude of the reference pixel point with the edge threshold corresponding to the reference pixel point; In response to the total gradient amplitude of the reference pixel point being greater than the edge threshold corresponding to the reference pixel point, determine that the reference pixel point is an edge pixel point.

3. The method according to claim 2, characterized in that, the step of determining the edge threshold corresponding to the reference pixel point includes: Determine a second pixel window centered on the reference pixel point in the original image; According to the total gradient amplitude of the first pixel points other than the reference pixel point in the second pixel window in at least one set direction, determine the average total gradient amplitude corresponding to the first pixel points; Compare the average total gradient amplitude with a set first threshold; In response to the average total gradient amplitude being greater than the first threshold, use the average total gradient amplitude as the edge threshold corresponding to the reference pixel point; In response to the average total gradient amplitude being less than or equal to the first threshold, use the first threshold as the edge threshold corresponding to the reference pixel point.

4. The method according to claim 1, characterized in that, the step of determining the main interpolation direction and the secondary interpolation direction corresponding to the reference pixel point includes: According to the position of the reference pixel point in the original image, determine a third pixel window including the reference pixel point in the original image; Determine the most numerous first set direction and the second most numerous second set direction according to the set directions corresponding to the maximum gradient magnitudes of the respective edge pixels included in the third pixel window; Determine the main interpolation direction and the secondary interpolation direction corresponding to the reference pixel point according to the first set direction and the second set direction.

5. The method according to claim 4, wherein, the determining the main interpolation direction and the secondary interpolation direction corresponding to the reference pixel point according to the first set direction and the second set direction includes: Determine the first quantity of the edge pixels included in the third pixel window, and the second quantity of the edge pixels corresponding to the first set direction; Determine a third set direction adjacent to the first set direction from the directions of the maximum gradient magnitudes corresponding to the respective edge pixels; Determine that the total quantity of the edge pixels corresponding to the third set direction is the third quantity; In response to the first quantity being greater than or equal to a set quantity threshold, and the sum of the second quantity and the third quantity being greater than or equal to the first quantity, and the second quantity being greater than half of the first quantity, determine the first set direction as the main interpolation direction and determine the second set direction as the secondary interpolation direction.

6. The method according to claim 5, wherein, the method further includes: In response to the first quantity being less than the set quantity threshold, or the sum of the second quantity and the third quantity being less than the first quantity, or the second quantity being less than or equal to half of the first quantity, determine that the reference pixel point has no main interpolation direction and secondary interpolation direction.

7. The method according to any one of claims 1-5, wherein, the fusing and interpolating the target interpolation point according to the main interpolation direction and the secondary interpolation direction of the reference pixel point includes: Interpolate the target interpolation point according to the coordinate position of the target interpolation point in the original image, the coordinate position of the reference pixel point, and the corresponding main interpolation direction to obtain a first interpolation result; Determine the result of fusing and interpolating the target interpolation point according to the quantity of the secondary interpolation direction and the first interpolation result.

8. The method according to claim 7, wherein, the determining the result of fusing and interpolating the target interpolation point according to the quantity of the secondary interpolation direction and the first interpolation result includes: In response to the quantity of the secondary interpolation direction being one, determine a fourth pixel window corresponding to the reference pixel point in the original image according to the position of the reference pixel point in the original image; Count the fourth quantity of the edge pixel points in the main interpolation direction and the fifth quantity of the edge pixel points in the secondary interpolation direction in the fourth pixel window; Compare the fourth quantity with a set multiple of the fifth quantity; In response to the fourth quantity being greater than or equal to the set multiple of the fifth quantity, use the first interpolation result as the result of fusing and interpolating the target interpolation point.

9. The method according to claim 8, wherein, the method further includes: In response to the fourth quantity being less than a set multiple of the fifth quantity, perform interpolation on the target interpolation point in the secondary interpolation direction to obtain a second interpolation result; Perform weighted addition on the first interpolation result and the second interpolation result to obtain the target interpolation result of the target interpolation point; Use the target interpolation result as the result of fusion interpolation for the target interpolation point.

10. The method according to claim 7, wherein, the determining the result of fusion interpolation for the target interpolation point according to the quantity of the secondary interpolation direction and the first interpolation result further includes: In response to the quantity of the secondary interpolation directions being multiple, perform interpolation on the target interpolation point using an interpolation algorithm that does not consider the interpolation direction to obtain a third interpolation result; Determine the result of fusion interpolation for the target interpolation point according to the first interpolation result and the third interpolation result.

11. The method according to claim 8, wherein, the performing interpolation on the target interpolation point according to the coordinate position of the target interpolation point in the original image, the coordinate position of the reference pixel point, and the corresponding primary interpolation direction to obtain a first interpolation result includes: Determine multiple pixels participating in interpolation in the fourth pixel window according to the offset of the coordinate position of the target interpolation point relative to the coordinate position of the reference pixel point and the primary interpolation direction; For each pixel participating in interpolation, determine the weight corresponding to the interpolation method used by the pixel participating in interpolation according to the position of the target interpolation point, the position of the pixel participating in interpolation, and the corresponding primary interpolation direction; Perform interpolation on the target interpolation point according to the weights of the pixels participating in interpolation and the corresponding interpolation methods to obtain a first interpolation result.

12. An image interpolation device, wherein, it includes: An acquisition module for acquiring the coordinate position of a target interpolation point to be interpolated in the original image; A first determination module for determining a reference pixel point corresponding to the position of the target interpolation point in the original image according to the coordinate position of the target interpolation point in the original image; A second determination module for determining the primary interpolation direction and the secondary interpolation direction corresponding to the reference pixel point in response to the reference pixel point being an edge pixel; An interpolation module for performing fusion interpolation on the target interpolation point according to the primary interpolation direction and the secondary interpolation direction of the reference pixel point in response to determining that the reference pixel point has the primary interpolation direction and the secondary interpolation direction.

13. An electronic device, wherein, it includes a memory, a processor, and a computer program stored on the memory and executable on the processor. When the processor executes the program, the method according to any one of claims 1-11 is implemented.

14. A non-transitory computer-readable storage medium, on which a computer program is stored, wherein, when the computer program is executed by a processor, the method according to any one of claims 1-11 is implemented.

Images