Image processing apparatus, method, electronic device, computer equipment, and storage medium

Abstract

This disclosure provides an image processing apparatus, method, electronic device, computer device, and storage medium. The image processing apparatus is used to magnify an input image to be optimized according to a preset magnification ratio to obtain an output image; determine target pixels in the input image that correspond one-to-one with each second pixel based on the position information of each first pixel in the input image, the position information of each second pixel in the output image, and the preset magnification ratio; determine gradient direction information and a reference interpolation result based on the position information and color gamut information of the target pixels, as well as the position information and color gamut information of other target pixels surrounding the target pixels; determine a first interpolation result of the second pixels in the gradient direction based on the gradient direction information and the reference interpolation result; fuse the first interpolation result of the second pixels in the gradient direction to obtain a first interpolation fusion result; and obtain an optimized target image based on the first interpolation fusion result.

Description

Technical Field

[0001] This disclosure belongs to the field of image processing technology, and specifically relates to an image processing apparatus, method, electronic device, computer device, and storage medium. Background Technology

[0002] With the rapid development of display technology, the forms of image content display are becoming increasingly diversified. Today's display products mostly pursue higher resolution, larger displays, and clearer image quality. If the original input signal source itself has a low resolution, or if the video is downsampled during transmission to save data, it is necessary to enlarge the original low-resolution image source in order to restore and adapt it to high-resolution display devices.

[0003] However, magnified images often suffer from problems such as jagged edges or blurriness at edges with large gradients. Summary of the Invention

[0004] This disclosure aims to at least solve one of the technical problems existing in the prior art, and to provide an image processing apparatus, device, electronic device, computer device, and storage medium.

[0005] Firstly, the technical solution adopted to solve the technical problem of this disclosure is an image processing apparatus, comprising:

[0006] The magnification module is used to magnify the input image to be optimized according to a preset magnification ratio to obtain the output image;

[0007] The pixel determination module is used to determine the target pixel in the input image that corresponds one-to-one with each of the second pixels based on the position information of each first pixel in the input image, the position information of each second pixel in the output image, and the preset magnification.

[0008] The gradient direction determination module is used to determine the gradient direction information of the second pixel based on the position information and color gamut information of the target pixel, as well as the position information and color gamut information of other target pixels around the target pixel.

[0009] The first processing module is used to determine the reference interpolation result of the second pixel in a preset reference direction based on the position information and color gamut information of the target pixel, as well as the position information and color gamut information of other target pixels around the target pixel.

[0010] The second processing module is used to determine the first interpolation result of the second pixel in the gradient direction based on the gradient direction information of the second pixel and the reference interpolation result of the second pixel in the preset reference direction.

[0011] The fusion module is used to fuse the first interpolation result of the second pixel in the gradient direction to obtain the first interpolation fusion result;

[0012] An image optimization module is used to obtain an optimized target image based at least on the first interpolation fusion result.

[0013] In some embodiments, the gradient direction determination module includes a pixel block determination unit, a sub-pixel block determination unit, and a gradient direction determination unit;

[0014] The pixel block determination unit is used to determine, based on the position information of the target pixel and the position information of other target pixels around the target pixel, the pixel block corresponding to the second pixel in the input image.

[0015] The sub-pixel block determining unit is used to divide the pixel block into multiple sub-pixel blocks according to preset division conditions;

[0016] The gradient direction determination unit is used to determine the gradient direction information of the second pixel point for each of the sub-pixel blocks in the plurality of pixel blocks, based on the color gamut information of each target pixel point in the sub-pixel block; different sub-pixel blocks correspond to different gradient direction information of the second pixel point.

[0017] In some embodiments, the gradient direction determination unit is specifically used to determine the value of the second pixel in the horizontal direction and the value in the vertical direction based on the color gamut information of each target pixel in the sub-pixel block; and to determine the gradient direction information of the second pixel based on the value of the second pixel in the horizontal direction and the value in the vertical direction.

[0018] In some embodiments, the image processing apparatus further includes a phase determination module;

[0019] The phase determination module is used to determine the phase information of the second pixel relative to the target pixel based on the position information of the target pixel, the position information of the second pixel, and the preset magnification.

[0020] The first processing module is specifically used to determine the reference interpolation result of the second pixel in a preset reference direction based on the phase information, the color gamut information of the target pixel, and the position information and color gamut information of other target pixels around the target pixel.

[0021] In some embodiments, the preset reference direction includes a first reference direction; the phase information of the second pixel relative to the target pixel includes horizontal phase and vertical phase;

[0022] The first processing module includes a first processing unit, a second processing unit, and a third processing unit;

[0023] The first processing unit is configured to determine a plurality of first associated pixels corresponding to the target pixel based on the position information of other target pixels around the target pixel and a first association condition pre-configured for the first reference direction;

[0024] The second processing unit is configured to determine a first intermediate interpolation result of the second pixel based on the horizontal phase, the color gamut information of the target pixel, and the color gamut information of a plurality of first associated pixels.

[0025] The third processing unit is used to determine the reference interpolation result of the second pixel in the first reference direction based on the first intermediate interpolation result of the second pixel and the vertical phase.

[0026] In some embodiments, the preset reference direction includes a second reference direction; the phase information of the second pixel relative to the target pixel includes horizontal phase and vertical phase;

[0027] The first processing module includes a first processing unit, a second processing unit, a third processing unit, and a fourth processing unit;

[0028] The fourth processing unit is used to determine the first angular phase, the first left phase, and the first right phase of the second pixel point based on the fact that the horizontal phase and the vertical phase are perpendicular.

[0029] The first processing unit is configured to determine a plurality of second associated pixels corresponding to the target pixel based on the comparison result of the horizontal phase and the vertical phase, the position information of other target pixels around the target pixel, and the second association conditions pre-configured for the second reference direction;

[0030] The second processing unit is configured to determine the second intermediate interpolation result of the second pixel based on the first left phase, the first right phase, the color gamut information of the target pixel, and the color gamut information of a plurality of second associated pixels;

[0031] The third processing unit is used to determine the reference interpolation result of the second pixel in the second reference direction based on the second intermediate interpolation result of the second pixel and the first angle phase.

[0032] In some embodiments, the preset reference direction includes a fourth reference direction; the phase information of the second pixel relative to the target pixel includes horizontal phase and vertical phase;

[0033] The first processing module includes a first processing unit, a second processing unit, a third processing unit, and a fourth processing unit;

[0034] The fourth processing unit is used to determine the second angular phase, the second left phase, and the second right phase of the second pixel point based on the fact that the horizontal phase and the vertical phase are perpendicular.

[0035] The first processing unit is configured to determine a plurality of fourth associated pixels corresponding to the target pixel based on the horizontal phase, the vertical phase, the position information of other target pixels around the target pixel and the fourth association conditions pre-configured for the fourth reference direction;

[0036] The second processing unit is configured to determine the fourth intermediate interpolation result of the second pixel based on the second left phase, the second right phase, the color gamut information of the target pixel, and the color gamut information of the plurality of fourth associated pixels;

[0037] The third processing unit is used to determine the reference interpolation result of the second pixel in the fourth reference direction based on the fourth intermediate interpolation result of the second pixel and the second angle phase.

[0038] In some embodiments, the image processing apparatus further includes a phase determination module;

[0039] The phase determination module is used to determine the phase information of the second pixel relative to the target pixel based on the position information of the target pixel, the position information of the second pixel, and the preset magnification.

[0040] The fusion module is used to fuse the first interpolation result of the second pixel point in the gradient direction based on the phase information to obtain a first interpolation fusion result.

[0041] In some embodiments, the image processing apparatus further includes a phase determination module;

[0042] The phase determination module is used to determine the phase information of the second pixel relative to the target pixel based on the position information of the target pixel, the position information of the second pixel, and the preset magnification.

[0043] The image optimization module includes a first optimization unit, a second optimization unit, and a third optimization unit;

[0044] The first optimization unit is used to determine a plurality of fifth associated pixels corresponding to the target pixel based on the position information of other target pixels around the target pixel and a pre-set fifth association condition;

[0045] The second optimization unit is used to determine the second interpolation result of the second pixel based on the phase information, the color gamut information of the target pixel, and the color gamut information of the plurality of fifth associated pixels;

[0046] The third optimization unit is used to obtain the optimized target image based on the first interpolation fusion result and the second interpolation result.

[0047] In some embodiments, the third optimization unit is specifically used to determine interpolation fusion information for each of the sub-pixel blocks in the plurality of pixel blocks based on the color gamut information of each of the target pixels in the sub-pixel blocks; to fuse the first interpolation fusion result and the second interpolation result based on at least a portion of the interpolation fusion information to obtain a second interpolation fusion result; and to obtain an optimized target image based on the second interpolation fusion result.

[0048] In some embodiments, the third optimization unit is specifically configured to: determine the gradient in the horizontal direction and the gradient in the vertical direction of the sub-pixel block based on the color gamut information of each target pixel in the sub-pixel block; determine the autocorrelation information of the horizontal gradient based on the gradient in the horizontal direction of the sub-pixel block; determine the autocorrelation information of the vertical gradient based on the gradient in the vertical direction of the sub-pixel block; determine the cross-correlation information of the horizontal gradient and the vertical gradient based on the gradient in the horizontal direction and the gradient in the vertical direction of the sub-pixel block; and determine the interpolation fusion information based on the autocorrelation information of the horizontal gradient, the autocorrelation information of the vertical gradient, and the cross-correlation information of the horizontal gradient and the vertical gradient.

[0049] In some embodiments, the image processing apparatus further includes a spatial conversion module;

[0050] The space conversion module is used to perform color gamut space conversion on the input image to be optimized, so as to obtain the color gamut information of the input image;

[0051] The third optimization unit is specifically used to perform reverse color gamut space conversion on the second interpolation fusion result to obtain the optimized target image.

[0052] Secondly, embodiments of this disclosure also provide an image processing method, including:

[0053] The input image to be optimized is magnified according to the preset magnification ratio to obtain the output image;

[0054] Based on the position information of each first pixel in the input image, the position information of each second pixel in the output image, and the preset magnification, the target pixel in the input image that corresponds one-to-one with each second pixel is determined;

[0055] Based on the position information and color gamut information of the target pixel, as well as the position information and color gamut information of other target pixels around the target pixel, the gradient direction information of the second pixel is determined;

[0056] Based on the position information and color gamut information of the target pixel, as well as the position information and color gamut information of other target pixels around the target pixel, the reference interpolation result of the second pixel in the preset reference direction is determined;

[0057] Based on the gradient direction information of the second pixel and the reference interpolation result of the second pixel in the preset reference direction, the first interpolation result of the second pixel in the gradient direction is determined;

[0058] The first interpolation result of the second pixel point in the gradient direction is fused to obtain the first interpolation fusion result;

[0059] Based at least on the first interpolation fusion result, an optimized target image is obtained.

[0060] Thirdly, embodiments of this disclosure also provide an electronic device, including an image processing apparatus as described in any one of the first aspects.

[0061] Fourthly, embodiments of this disclosure also provide a computer device, including: a processor, a memory, and a bus, wherein the memory stores machine-readable instructions executable by the processor, and when the computer device is running, the processor communicates with the memory via the bus, and when the machine-readable instructions are executed by the processor, the steps of the image processing method as described in the second aspect are performed.

[0062] Fifthly, embodiments of this disclosure also provide a computer non-transient readable storage medium storing a computer program that, when executed by a processor, performs the steps of the image processing method as described in the second aspect. Attached Figure Description

[0063] Figure 1 A schematic diagram of an image processing apparatus provided in an embodiment of this disclosure;

[0064] Figure 2 A flowchart of an image processing method provided in an embodiment of this disclosure;

[0065] Figure 3 A schematic diagram of an exemplary pixel block provided for an embodiment of this disclosure;

[0066] Figure 4 A schematic diagram of an exemplary plurality of sub-pixel blocks provided for an embodiment of this disclosure;

[0067] Figure 5 A detailed flowchart of an image processing method provided in this embodiment of the present disclosure;

[0068] Figure 6 This is a schematic diagram of the structure of a computer device provided in an embodiment of this disclosure. Detailed Implementation

[0069] To make the objectives, technical solutions, and advantages of the embodiments of this disclosure clearer, the technical solutions of the embodiments of this disclosure will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this disclosure, and not all of them. The components of the embodiments of this disclosure described and shown in the accompanying drawings can generally be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of this disclosure provided in the accompanying drawings is not intended to limit the scope of the claimed disclosure, but merely represents selected embodiments of this disclosure. All other embodiments obtained by those skilled in the art based on the embodiments of this disclosure without inventive effort are within the scope of protection of this disclosure.

[0070] Unless otherwise defined, the technical or scientific terms used in this disclosure shall have the ordinary meaning understood by one of ordinary skill in the art to which this disclosure pertains. The terms “first,” “second,” and similar terms used in this disclosure do not indicate any order, quantity, or importance, but are merely used to distinguish different components. Similarly, the terms “an,” “a,” or “the,” and similar terms do not indicate a quantity limitation, but rather indicate the presence of at least one. The terms “including,” “comprising,” or “containing,” and similar terms mean that the element or object preceding the word encompasses the elements or objects listed following the word and their equivalents, without excluding other elements or objects. The terms “connected,” “linked,” or similar terms are not limited to physical or mechanical connections, but can include electrical connections, whether direct or indirect. The terms “upper,” “lower,” “left,” and “right,” etc., are used only to indicate relative positional relationships, and these relative positional relationships may change accordingly when the absolute position of the described objects changes.

[0071] In this disclosure, "multiple or several" refers to two or more. "And / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, or B alone. The character " / " generally indicates that the preceding and following related objects have an "or" relationship.

[0072] In related technologies, commonly used image scaling methods include interpolation scaling methods based on traditional techniques or AI-based methods. Traditional interpolation scaling methods include: 1) Nearest Neighbor Interpolation Scaling: This scaling method does not consider image texture and details. In scale-down applications, image texture loss is severe, and in scale-up applications, severe jagged edges exist. 2) Traditional linear interpolation methods, such as bilinear interpolation and bi-cubic interpolation. This method does not optimize for the texture of high-resolution images, resulting in severe texture loss and jagged edges. AI-based methods, such as AI Scaling, are currently a research direction in academia. However, the performance of such methods has not yet completely surpassed traditional methods, and the cost is very high. For example, a fixed set of parameters and models typically supports a fixed scaling factor, making it difficult to implement in chip manufacturing.

[0073] In view of this, embodiments of the present disclosure provide an image processing apparatus that can solve the problem of jagged edges with large gradients, while preserving details in the image and reducing edge blurring.

[0074] Figure 1 This is a schematic diagram of an image processing apparatus provided in an embodiment of the present disclosure, such as... Figure 1 As shown, the image processing device includes a magnification module 1, a pixel determination module 2, a gradient direction determination module 3, a first processing module 4, a second processing module 5, a fusion module 6, and an image optimization module 7, wherein:

[0075] The magnification module 1 is used to magnify the input image to be optimized according to a preset magnification ratio to obtain the output image.

[0076] The pixel determination module 2 is used to determine the target pixel in the input image that corresponds one-to-one with each of the second pixels based on the position information of each first pixel in the input image, the position information of each second pixel in the output image, and the preset magnification.

[0077] The gradient direction determination module 3 is used to determine the gradient direction information of the second pixel based on the position information and color gamut information of the target pixel, as well as the position information and color gamut information of other target pixels around the target pixel.

[0078] The first processing module 4 is used to determine the reference interpolation result of the second pixel in the preset reference direction based on the position information and color gamut information of the target pixel, as well as the position information and color gamut information of other target pixels around the target pixel.

[0079] The second processing module 5 is used to determine the first interpolation result of the second pixel in the gradient direction based on the gradient direction information of the second pixel and the reference interpolation result of the second pixel in the preset reference direction.

[0080] The fusion module 6 is used to fuse the first interpolation result of the second pixel in the gradient direction to obtain the first interpolation fusion result.

[0081] The image optimization module 7 is used to obtain an optimized target image based at least on the first interpolation fusion result.

[0082] First, the image processing apparatus provided in this disclosure is applied to scenarios involving high-resolution image optimization. To save data transmission power during transmission, the image processing apparatus selects a low-resolution input image for transmission. During transmission, it utilizes a magnification module to magnify the input image to be optimized according to a preset magnification ratio, thereby obtaining a high-resolution output image.

[0083] Secondly, the gradient direction determination module 3 adds information about other target pixels around the target pixel to improve the accuracy of the interpolation results. The first and second processing modules are used respectively to accurately calculate the texture gradient of the second pixel. Combining the gradient direction information and the reference interpolation results in the preset baseline direction, the gradient can be calculated and the first interpolation result determined even for texture details with small gradients. This preserves high-resolution details in the image, enabling the fusion module to obtain a better interpolation effect for the second pixel in the gradient direction during the fusion process, thus achieving a target image with superior display quality.

[0084] To facilitate understanding of the processing procedures of each module in the image processing device, the functions of each module will be explained in detail below through the specific execution flow of the image processing method.

[0085] Figure 2 A flowchart of an image processing method provided in this disclosure embodiment, such as... Figure 2 As shown, steps S11 to S17 are included, wherein:

[0086] S11. The magnification module 1 magnifies the input image to be optimized according to the preset magnification ratio to obtain the output image.

[0087] Here, the preset magnification can be an image magnification factor that is pre-set based on the application scenario of high resolution.

[0088] The resolution of the input image to be optimized is adjusted according to the preset magnification to obtain a high-resolution output image. Gradient optimization processing will then be performed on the high-resolution output image.

[0089] S12. The pixel determination module 2 determines the target pixel in the input image that corresponds one-to-one with each second pixel based on the position information of each first pixel in the input image, the position information of each second pixel in the output image, and the preset magnification.

[0090] The location information represents the coordinates of a pixel in the image.

[0091] This disclosure performs gradient optimization for each second pixel, which requires finding the first pixel that corresponds one-to-one with each second pixel in the output image, and taking the first pixel that uniquely corresponds to the second pixel as the target pixel.

[0092] For example, such as Figure 2 As shown, given the preset magnification factor is rate, and the position information of a certain second pixel is (p_o_xf, p_o_yf), the process of determining the target pixel that uniquely corresponds to this second pixel is as follows: Formula (I)

[0093]

[0094]

[0095] Where p_i_xf represents the x-coordinate of the target pixel; p_i_yf represents the y-coordinate of the target pixel; This indicates rounding down to the nearest integer.

[0096] S13. The gradient direction determination module 3 determines the gradient direction information of the second pixel based on the position information and color gamut information of the target pixel, as well as the position information and color gamut information of other target pixels around the target pixel.

[0097] The color gamut information of the target pixel is based on the color gamut information obtained after the input image has undergone color gamut space transformation. Specifically, the input image to be optimized is subjected to color gamut space transformation to obtain the color gamut information of the input image. For example, the input image is transformed from RGB pixels to YUV color gamut. The color gamut information includes components Y, U, and V, where Y represents luminance, and U and V represent chrominance.

[0098] For example, the conversion from RGB pixels to the YUV color gamut can be seen in the following formula (II):

[0099]

[0100] Here, >>6 represents a shift operation, specifically meaning division by 64; Indicates the color gamut conversion parameters; The parameters are those used for standardization after color gamut conversion; R1, G1, and B1 represent the sub-pixel values ​​of the red, green, and blue channels of the first pixel of the input image, respectively; Y represents luminance; U represents the first chromaticity value; and V represents the second chromaticity value.

[0101] In step S13, the other target pixels surrounding the target pixel can be any first pixel in the input image other than the target pixel. Alternatively, they can be the area surrounding the target pixel planned according to a pre-set algorithm. Any other first pixels falling within this area and excluding the target pixel are denoted as other target pixels surrounding the target pixel.

[0102] In some embodiments, the gradient direction determination module 3 includes a pixel block determination unit, a sub-pixel block determination unit, and a gradient direction determination unit; the above step S13 specifically includes the following steps S13-1-1 to S13-1-3, wherein the pixel block determination unit is used to perform the following step S13-1-1, the sub-pixel block determination unit is used to perform the following step S13-1-2, and the gradient direction determination unit is used to perform the following step S13-1-3, specifically:

[0103] S13-1-1 The pixel block determination unit determines the pixel block in the input image corresponding to the second pixel point based on the position information of the target pixel point and the position information of other target pixels around the target pixel point.

[0104] Specifically, based on the location information of the target pixel and the location information of other target pixels around it, a pre-set pixel block division algorithm is used to select M×N first pixels, centered on the target pixel, as a pixel block B0. Here, M and N are both the number of first pixels. The pixel block information includes multiple target pixels and their location information.

[0105] Figure 3 A schematic diagram of an exemplary pixel block provided for an embodiment of this disclosure, such as... Figure 3 As shown, given that the target pixel is determined, taking the target pixel (p_i_xf, p_i_yf) located at position (1,1) in the central region as an example, that is, taking the target pixel located in the second row and second column of the pixel block as an example, 4×4 first pixel points are divided into a pixel block.

[0106] The above defines the pixel block size using a 4×4 size as an example; of course, other pixel blocks of different sizes can also be selected, and this embodiment does not impose specific limitations on this.

[0107] For ease of understanding, this disclosure uses the target pixel being located at position (1,1) as a reference and the pixel block size being 4×4 as an example for explanation.

[0108] S13-1-2, The sub-pixel block determination unit divides the pixel block into multiple sub-pixel blocks according to the preset division conditions.

[0109] Here, the preset division conditions can be set based on application scenarios and experience. A pixel block can be divided into multiple different pixel blocks.

[0110] For example, Figure 4 This disclosure provides an exemplary schematic diagram of multiple sub-pixel blocks, such as... Figure 4 As shown, taking pixel block B0 as an example, it is divided into four 3×3 sub-pixel blocks, denoted as B_sub0, B_sub1, B_sub2, and B_sub3 respectively.

[0111] S13-1-3, Gradient Direction Determination Unit: For each sub-pixel block in multiple pixel blocks, the gradient direction information of the second pixel is determined based on the color gamut information of each target pixel in the sub-pixel block.

[0112] Among them, different sub-pixel blocks correspond to different gradient direction information of the second pixel point.

[0113] Step S13-1-3 uses a single sub-pixel block as an example to determine the gradient direction information of the second pixel. Different gradient direction information of the second pixel can be calculated for different sub-pixel blocks.

[0114] Given the color gamut information of each first pixel in the input image, the color gamut information of each target pixel in the sub-pixel block can be determined based on the position information of each target pixel in the sub-pixel block.

[0115] Color gamut information includes luminance Y, first chromaticity value U, and second chromaticity value V.

[0116] This embodiment improves the accuracy of the interpolation results by adding information about other target pixels around the target pixel.

[0117] For example, the gradient direction determination unit can determine the gradient direction information of the second pixel based on the brightness Y of each target pixel in the sub-pixel block. Specifically, step S13-1-3 includes steps S13-1-31 and S13-1-32, wherein:

[0118] S13-1-31. Based on the brightness Y of each target pixel in the sub-pixel block, determine the cost value of the second pixel in the horizontal direction and the cost value in the vertical direction.

[0119] Taking sub-pixel block B_sub0 as an example, the brightness Y of each target pixel includes Y(0,0), Y(0,1), Y(0,2), Y(1,0), Y(1,1), Y(1,2), Y(2,0), Y(2,1) and Y(2,2).

[0120] The process of determining the horizontal cost_hor of the second pixel is shown in Formula (III) below:

[0121] Cost_hor_0=ABS(Y(0,1)×2-Y(0,0)-Y(0,2))

[0122] Cost_hor_1=ABS(Y(1,1)×2-Y(1,0)-Y(1,2))

[0123] Cost_hor_2=ABS(Y(2,1)×2-Y(2,0)-Y(2,2))

[0124] Cost_hor=Cost_hor_0+Cost_hor_1+Cost_hor_2....Formula (3)

[0125] Here, ABS represents taking the absolute value.

[0126] The process of determining the cost_ver of the second pixel in the vertical direction is shown in the following formula (IV):

[0127] Cost_ver_0=ABS(Y(1,0)×2-Y(0,0)-Y(2,0))

[0128] Cost_ver_1=ABS(Y(1,1)×2-Y(0,1)-Y(2,1))

[0129] Cost_ver_2=ABS(Y(1,2)×2-Y(0,2)-Y(2,2))

[0130] Cost_ver=Cost_ver_0+Cost_ver_1+Cost_ver_2....Formula (4)

[0131] Similarly, the cost values ​​of sub-pixel blocks B_sub1, B_sub2, and B_sub3 in the horizontal direction and in the vertical direction are the same as those of sub-pixel block B_sub0 in the above-mentioned processing of cost values ​​in the horizontal and vertical directions (i.e., formulas (III) and (IV)). Repeated parts will not be repeated.

[0132] S13-1-32. Determine the gradient direction information of the second pixel based on the cost value of the second pixel in the horizontal direction and the cost value in the vertical direction.

[0133] The process for determining the gradient direction information of the second pixel is shown in the following formula (V):

[0134]

[0135] Wherein, D0 corresponds to the gradient direction information determined based on sub-pixel block B_sub0; D1 corresponds to the gradient direction information determined based on sub-pixel block B_sub1; D2 corresponds to the gradient direction information determined based on sub-pixel block B_sub2; and D3 represents the gradient direction information determined based on sub-pixel block B_sub3.

[0136] For example, the gradient direction determination unit can also determine the gradient direction information of the second pixel based on the luminance Y, the first chromaticity value U, and the second chromaticity value V of each target pixel in the sub-pixel block. For example, taking sub-pixel block B_sub0 as an example, based on the luminance Y of each target pixel in sub-pixel block B_sub0, a first intermediate result is determined according to the above methods S13-1-31 and S13-1-32; based on the first chromaticity value U of each target pixel in sub-pixel block B_sub0, a second intermediate result is determined according to the above methods S13-1-31 and S13-1-32; based on the second chromaticity value V of each target pixel in sub-pixel block B_sub0, a third intermediate result is determined according to the above methods S13-1-31 and S13-1-32; finally, the first intermediate result, the first intermediate result, and the first intermediate result are fused to obtain the gradient direction information of the second pixel determined based on sub-pixel block B_sub0. The calculation process for the gradient direction information of other sub-pixel blocks B_sub1, B_sub2 and B_sub3 is similar, and the repeated parts will not be repeated.

[0137] In the above embodiments, calculating the gradient direction information solely based on the luminance Y of the target pixel improves optimization efficiency compared to combining the first chromaticity value U and the second chromaticity value V. Furthermore, luminance Y is a major factor affecting human visual perception; therefore, it is a crucial parameter influencing gradient information. Calculating the gradient direction information based on luminance Y ensures effective gradient optimization. Combining the luminance Y, the first chromaticity value U, and the second chromaticity value V of the target pixel to calculate the gradient direction information further enhances the gradient optimization effect, albeit at the cost of some computational efficiency. Therefore, a suitable optimization scheme can be selected based on the specific application scenario.

[0138] Of course, in addition to using the cost calculation process of S13-1-31 and S13-1-32 described above to determine the gradient direction information, other operators can also be used to obtain the gradient direction information. This embodiment of the present disclosure does not specifically limit this.

[0139] In some embodiments, the image processing apparatus further includes a phase determination module 8. Before determining the reference interpolation result of the second pixel in the preset reference direction, the phase determination module 8 determines the phase information of the second pixel relative to the target pixel in the input image after determining the target pixel in each of the second pixels. The phase determination module 8 can determine the phase information of the second pixel relative to the target pixel based on the position information of the target pixel, the position information of the second pixel, and the preset magnification.

[0140] For example, given that the preset magnification is rate, the position information of the second pixel is (p_o_xf, p_o_yf), and the position information of the target pixel corresponding to the second pixel is (p_i_xf, p_i_yf), the process of determining the phase information of the second pixel relative to the target pixel is shown in the following formula (VI):

[0141] px_phase = p_o_x / rate - p_i_xf

[0142] ln_phase=p_o_y / rate-p_i_yf...............Formula (VI)

[0143] Wherein, px_phase represents the relative displacement of the second pixel relative to the target pixel in the row direction, that is, the horizontal phase; ln_phase represents the relative displacement of the second pixel relative to the target pixel in the column direction, that is, the vertical phase.

[0144] S14. The first processing module 4 determines the reference interpolation result of the second pixel in the preset reference direction based on the position information and color gamut information of the target pixel, as well as the position information and color gamut information of other target pixels around the target pixel.

[0145] Specifically, the first processing module 4 can determine the reference interpolation result of the second pixel in the preset reference direction based on the phase information, the color gamut information of the target pixel, and the position information and color gamut information of other target pixels around the target pixel.

[0146] Here, the preset reference direction can be a pre-set reference direction, which may include multiple directions. For example, it may include at least two of a first reference direction, a second reference direction, a third reference direction, and a fourth reference direction. The first reference direction may be, for example, 0°; the second reference direction may be, for example, 45°; the third reference direction may be, for example, 90°; and the fourth reference direction may be, for example, 135°.

[0147] It should be noted that, in order to solve the technical problem to a greater extent, the reference directions can be expanded. The more reference directions there are, the better the final magnification effect will be. However, the overall image processing load will also be higher. Therefore, considering the complexity of the entire processing flow, four reference directions are selected for processing.

[0148] In this embodiment, the interpolation result of the second pixel point in a preset reference direction is predicted in advance. This interpolation result is a reference interpolation result and is not the actual interpolation result in the gradient direction indicated by the gradient direction information.

[0149] The first processing module 4 includes a first processing unit, a second processing unit, and a third processing unit. Taking a preset reference direction as the first reference direction as an example, it determines the reference interpolation result of the second pixel point at 0° in the first reference direction, specifically including steps S14-1-11 to S14-1-13, wherein the first processing unit is used to execute S14-1-11, the second processing unit is used to execute S14-1-12, and the third processing unit is used to execute S14-1-13, specifically:

[0150] S14-1-11, The first processing unit determines multiple first associated pixels corresponding to the target pixel based on the position information of other target pixels around the target pixel and the first association conditions pre-configured for the first reference direction.

[0151] Following a general linear interpolation algorithm, multiple first-associative pixels corresponding to the target pixel are selected from other target pixels surrounding the target pixel. Taking the target pixel (1,1) as an example, if a bilinear interpolation algorithm is used, the first association condition pre-configured for the 0° direction is to determine a 2×2 pixel group with the target pixel as the top-left corner. This pixel group includes the target pixels (1,1), (1,2), (1,3), and (1,4), where (1,2), (1,3), and (1,4) are the first-associative pixels associated with the target pixel (1,1). Here, by increasing the number of first-associative pixels associated with the target pixel (1,1), the accuracy of the interpolation result is improved.

[0152] S14-1-12. The second processing unit determines the first intermediate interpolation result of the second pixel based on the horizontal phase, the color gamut information of the target pixel, and the color gamut information of multiple first associated pixels.

[0153] Color gamut information includes luminance Y, first chromaticity value U, and second chromaticity value V.

[0154] For example, the first intermediate interpolation result of the second pixel in the Y channel can be determined based on the horizontal phase, the brightness Y of the target pixel, and the brightness Y of each of the multiple first associated pixels. The specific determination process is shown in the following formula (VII):

[0155] y_ang_t_1=(Y(1,1)×(1-px_phase)+Y(1,2)×px_phase)

[0156] y_ang_b_1=(Y(2,1)×(1-px_phase)+Y(2,2)×px_phase) formula (7)

[0157] Similarly, to determine the first intermediate interpolation results u_ang_t_1 and u_ang_b_1 of the second pixel in the U channel, and to determine the first intermediate interpolation results v_ang_t_1 and v_ang_b_1 of the second pixel in the V channel, please refer to the above process for determining the first intermediate interpolation result of the second pixel in the Y channel (i.e., formula (VII)). Repeated parts will not be repeated.

[0158] S14-1-13. The third processing unit determines the reference interpolation result of the second pixel point in the first reference direction based on the first intermediate interpolation result and the vertical phase of the second pixel point.

[0159] For example, based on the first intermediate interpolation result of the second pixel in the Y channel and the vertical phase, the reference interpolation result y_ang[0] of the second pixel in the 0° direction is determined. For details, please refer to the following formula (VIII):

[0160] y_ang[0]=(y_ang_t_1×(1-ln_phase)+y_ang_b_1×ln_phase) formula (8)

[0161] Similarly, to determine the reference interpolation result u_ang[0] of the second pixel in the 0° direction, and to determine the reference interpolation result u_ang[0] of the second pixel in the 0° direction, please refer to the above process of determining the reference interpolation result y_ang[0] of the second pixel in the 0° direction (that is, formula (eight)). The repeated parts will not be repeated.

[0162] The first processing module 4 includes a first processing unit, a second processing unit, a third processing unit, and a fourth processing unit. Taking a preset reference direction as the second reference direction as an example, it determines the reference interpolation result of the second pixel point at 45° in the second reference direction, specifically including steps S14-1-21 to S14-1-24. Specifically, the fourth processing unit executes S14-1-21, the first processing unit executes S14-1-22, the second processing unit executes S14-1-23, and the third processing unit executes S14-1-24.

[0163] S14-1-21. The fourth processing unit determines the first angular phase, the first left phase, and the first right phase of the second pixel based on the horizontal phase and the vertical phase.

[0164] Specifically, the first processing unit can determine the first angular phase, the first left phase, and the first right phase of the second pixel based on the horizontal phase, the vertical phase, and the numerical comparison results of the horizontal phase and the vertical phase.

[0165] Case 1: When ln_phase > px_phase, the process of determining the first angular phase phase_line_1, the first left phase phase_l_1, and the first right phase phase_r_1 of the second pixel is shown in the following formula (IX):

[0166] phase_line_1=a1+px_phase-ln_phase

[0167] phase_l_1=phase_r+a2

[0168] phase_r_1=(px_phase+ln_phase) / a3...........Formula (IX)

[0169] Wherein, a1 represents the first preset parameter, a2 represents the second preset parameter, and a3 represents the third preset parameter. The first, second, and third preset parameters are set based on actual scenarios and experience, and are not specifically limited in this embodiment.

[0170] Case 2: When ln_phase ≤ px_phase, the process of determining the first angular phase phase_line, the first left phase phase_l, and the first right phase phase_r of the second pixel is shown in the following formula (x):

[0171] phase_line_1=a4+px_phase-ln_phase

[0172] phase_l_1=phase_r+a5

[0173] phase_r_1=(px_phase+ln_phase) / a3...........Formula (10)

[0174] Where a4 represents the fourth preset parameter, and a4 < a1; a5 represents the fifth preset parameter, and a5 < a2.

[0175] S14-1-22, the first processing unit determines a plurality of second associated pixels corresponding to the target pixel based on the comparison results of the horizontal phase and the vertical phase, the position information of other target pixels around the target pixel, and the second association conditions pre-configured for the second reference direction.

[0176] Following a standard linear interpolation algorithm, multiple second-related pixels corresponding to the target pixel are selected from other target pixels surrounding the target pixel. Here, by increasing the number of second-related pixels associated with the target pixel, the accuracy of the interpolation result is improved.

[0177] Scenario 1: When ln_phase > px_phase, taking the target pixel (1, 1) as an example, if the bilinear interpolation algorithm is used, the second association condition pre-configured for the 45° direction is: the pixel with the target pixel as its upper left corner, corresponding to other target pixels (2, 2) diagonally downwards in the 45° direction; the other target pixel (1, 0) adjacent to the left of the target pixel (1, 1); and the other target pixel (2, 1) diagonally downwards in the 45° direction corresponding to the other target pixel (1, 0). Here, (2, 2), (1, 0), and (2, 1) are the second associated pixels related to the target pixel.

[0178] Case 2: When ln_phase ≤ px_phase, taking the target pixel (1, 1) as an example, if the bilinear interpolation algorithm is used, the second association condition pre-configured for the 45° direction is: the pixel with the target pixel as its upper left corner, corresponding to other target pixels (2, 2) diagonally downwards in the 45° direction; other target pixels (1, 2) adjacent to the right of the target pixel (1, 1); and other target pixels (2, 3) diagonally downwards in the 45° direction corresponding to the other target pixel (1, 2). Here, (2, 2), (1, 2), and (2, 3) are the second associated pixels related to the target pixel.

[0179] S14-1-23, The second processing unit determines the second intermediate interpolation result of the second pixel based on the first left phase, the first right phase, the color gamut information of the target pixel, and the color gamut information of multiple second associated pixels.

[0180] Color gamut information includes luminance Y, first chromaticity value U, and second chromaticity value V.

[0181] For example, the second intermediate interpolation result of the second pixel in the Y channel can be determined based on the first left phase, the first right phase, the brightness Y of the target pixel, and the brightness Y of each of the multiple second associated pixels.

[0182] Case 1: When ln_phase > px_phase, determine the second intermediate interpolation result of the second pixel in the Y channel. For the specific determination process, please refer to the following formula (XI):

[0183] y_ang_l_2=(Y(1,0)×(1-phase_l_1)+Y(2,1)×phase_l_1)

[0184] y_ang_r_2=(Y(1,1)×(1-phase_r_1)+Y(2,2)×phase_r_1) Formula (XI)

[0185] Similarly, the second intermediate interpolation results u_ang_l_2 and u_ang_r_2 of the second pixel in the U channel can also be determined according to the above algorithm; and the second intermediate interpolation results v_ang_l_2 and v_ang_r_2 of the second pixel in the V channel can also be determined.

[0186] Scenario 2, in ln phase When ≤px_phase, determine the second intermediate interpolation result of the second pixel in the Y channel. For the specific determination process, please refer to the following formula (XII):

[0187] y_ang_l_2=(Y(1,1)×(1-phase_l_1)+Y(2,2)×phase_l_1)

[0188] y_ang_r_2=(Y(1,2)×(1-phase_F_1)+Y(2,3)×phase_r_1) Formula (XII)

[0189] Similarly, the second intermediate interpolation results u_ang_l_2 and u_ang_r_2 of the second pixel in the U channel can also be determined according to the above algorithm; and the second intermediate interpolation results v_ang_l_2 and v_ang_r_2 of the second pixel in the V channel can also be determined.

[0190] S14-1-24. The third processing unit determines the reference interpolation result of the second pixel in the second reference direction based on the second intermediate interpolation result of the second pixel and the first angle phase.

[0191] For example, based on the second intermediate interpolation result of the second pixel in the Y channel and the first angle phase, the reference interpolation result y_ang[1] of the second pixel in the 45° direction is determined. The specific process is as follows:

[0192] y_ang[1]=(y_ang_l_2×(1-phase_line_1)+y_ang_r_2×phatse_line_1) formula (13)

[0193] Similarly, to determine the reference interpolation result u_ang[1] of the second pixel in the 45° direction and the reference interpolation result v_ang[1] of the second pixel in the 45° direction, you can refer to the above process of determining the reference interpolation result y_ang[1] of the second pixel in the 45° direction (that is, formula (thirteen)). The repeated parts will not be repeated.

[0194] Taking the preset reference direction as the third reference direction as an example, the reference interpolation result of the second pixel point at 90° in the third reference direction is determined. The implementation process is the same as that of determining the reference interpolation result of the second pixel point at 0° in the first reference direction. The repeated parts will not be described again.

[0195] For example, based on the third intermediate interpolation result of the second pixel in the Y channel and the vertical phase, the reference interpolation result y_ang[2] of the second pixel in the 90° direction is determined. The specific process is as follows:

[0196] y_ang[2]=(y_ang_t_3×(1-ln_phase)+y_ang_b×ln_phase) formula (14)

[0197] Where y_ang_t_3 represents the third intermediate interpolation result of the second pixel in the Y channel, and y_ang_t_3 = y_ang_t_1.

[0198] Similarly, to determine the reference interpolation result u_ang[2] of the second pixel in the 90° direction, you can refer to the above process of determining the reference interpolation result y_ang[2] of the second pixel in the 90° direction (that is, formula (fourteen)). The repeated parts will not be repeated.

[0199] The first processing module 4 includes a first processing unit, a second processing unit, a third processing unit, and a fourth processing unit. Taking the preset reference direction as the fourth reference direction as an example, it determines the reference interpolation result of the second pixel point in the fourth reference direction, specifically including steps S14-1-31 to S14-1-34, wherein the fourth processing unit is used to execute S14-1-31, the first processing unit is used to execute S14-1-32, the second processing unit is used to execute S14-1-33, and the third processing unit is used to execute S14-1-34, specifically:

[0200] S14-1-31, The fourth processing unit determines the second angle phase, the second left phase, and the second right phase of the second pixel based on the horizontal phase and the vertical phase.

[0201] Case 3: When (ln_phase + px_phase) > 1, the process of determining the second angular phase phase_line_2, the second left phase phase_l_2, and the second right phase phase_r_2 of the second pixel is shown in the following formula (XV):

[0202] phase_line_2=px_phase+ln_phase-a6

[0203] phase_l_2=phase_r+a7

[0204] phase_r_2=(px_phase+ln_phase) / a8.......Formula (XV)

[0205] Wherein, a6 represents the sixth preset parameter, a7 represents the seventh preset parameter, and a8 represents the eighth preset parameter. The sixth, seventh, and eighth preset parameters are set based on actual scenarios and experience, and are not specifically limited in this embodiment.

[0206] Case 4: When (ln_phase + px_phase) ≤ 1, the process of determining the first angular phase phase_line, the first left phase phase_l, and the first right phase phase_r of the second pixel is shown in the following formula (XVI):

[0207] phase_line_2=px_phase+ln_phase-a9

[0208] phase_l_2=(ln_phase-px_phase) / a10

[0209] phase_r_2=(ln_phase+px_phase) / a10+a11.......Formula (Sixteen)

[0210] Wherein, a9 represents the ninth preset parameter, and a9 < a6; a10 represents the tenth preset parameter; and a11 represents the seventh preset parameter. The ninth, tenth, and eleventh preset parameters are set according to actual scenarios and experience, and are not specifically limited in this embodiment.

[0211] S14-1-32, The first processing unit determines multiple fourth associated pixels corresponding to the target pixel based on the horizontal phase, vertical phase, position information of other target pixels around the target pixel, and the fourth association conditions pre-configured for the fourth reference direction.

[0212] Following a standard linear interpolation algorithm, multiple fourth-related pixels corresponding to the target pixel are selected from other target pixels surrounding the target pixel. Here, by increasing the number of fourth-related pixels associated with the target pixel, the accuracy of the interpolation result is improved.

[0213] Case 3: When (ln_phase+px_phase)>1, taking the target pixel (1,1) as an example, if the bilinear interpolation algorithm is used, the fourth association condition configured in advance for the 135° direction is to determine a 2×2 pixel group with the other target pixel (1,2) adjacent to the right of the target pixel as the top left pixel. This pixel group includes the target pixels (1,2), (2,3), (1,3) and (2,4), where (1,2), (2,3), (1,3) and (2,4) are all fourth association pixels associated with the target pixel (1,1).

[0214] Case 4: When (ln_phase + px_phase) ≤ 1, taking the target pixel (1, 1) as an example, if the bilinear interpolation algorithm is used, the fourth association condition pre-configured for the 135° direction is: the pixel with the target pixel as its upper left corner, corresponding to other target pixels (2, 2) diagonally downwards in the 45° direction; other target pixels (1, 2) adjacent to the right of the target pixel (1, 1); and other target pixels (2, 3) diagonally downwards in the 45° direction corresponding to the other target pixel (1, 2). Among them, (2, 2), (1, 2), and (2, 3) are the fourth association pixels associated with the target pixel.

[0215] S14-1-33. The second processing unit determines the fourth intermediate interpolation result of the second pixel based on the second left phase, the second right phase, the color gamut information of the target pixel, and the color gamut information of multiple fourth associated pixels.

[0216] Color gamut information includes luminance Y, first chromaticity value U, and second chromaticity value V.

[0217] For example, the fourth intermediate interpolation result of the second pixel in the Y channel can be determined based on the second left phase, the second right phase, the brightness Y of the target pixel, and the brightness Y of each of the multiple fourth associated pixels.

[0218] Case 3: When (ln_phase+px_phase)>1, determine the fourth intermediate interpolation result of the second pixel in the Y channel. For the specific determination process, please refer to the following formula (XVII):

[0219] y_ang_l_4=(Y(1,2)×(1-phase_l_2)+Y(2,3)×phase_l_2)

[0220] y_ang_r_4=(Y(1,3)×(1-phase_r_2)+Y(2,4)×phase_r_2) Formula (XVII)

[0221] Similarly, according to the above formula (17), the fourth intermediate interpolation results u_ang_l_4 and u_ang_r_4 of the second pixel point in the U channel can also be determined; and the fourth intermediate interpolation results v_ang_l_4 and v_ang_r_4 of the second pixel point in the V channel can also be determined.

[0222] Case 4: When (ln_phase + px_phase) ≤ 1, determine the fourth intermediate interpolation result of the second pixel in the Y channel. The specific calculation process is as follows:

[0223] y_ang_l_4=(Y(1,1)×(1-phase_l_2)+Y(2,2)×phase_l_2)

[0224] y_ang_r_4=(Y(1,2)×(1-phase_r_2)+Y(2,3)×phase_r_2) Formula (XVIII)

[0225] Similarly, according to the above formula (18), the fourth intermediate interpolation results u_ang_l_4 and u_ang_r_4 of the second pixel point in the U channel can also be determined; and the fourth intermediate interpolation results v_ang_l-4 and v_ang_r_4 of the second pixel point in the V channel can also be determined.

[0226] S14-1-34, The third processing unit determines the reference interpolation result of the second pixel in the fourth reference direction based on the fourth intermediate interpolation result and the second angle phase of the second pixel.

[0227] For example, based on the fourth intermediate interpolation result of the second pixel in the Y channel and the second angle phase, the reference interpolation result y_ang[3] of the second pixel in the 135° direction is determined. The specific process is as follows: y_ang[3]=(y_ang_l-4×(1-phase_line_2)+y_ang_r_4×phase_line_2) Formula (XIX)

[0228] Similarly, to determine the reference interpolation result u_ang[3] of the second pixel in the 135° direction and the reference interpolation result v_ang[3] of the second pixel in the 135° direction, you can refer to the above process of determining the reference interpolation result y_ang[3] of the second pixel in the 135° direction (that is, formula (19)). The repeated parts will not be repeated.

[0229] S15, the second processing module 5 determines the first interpolation result of the second pixel in the gradient direction based on the gradient direction information of the second pixel and the reference interpolation result of the second pixel in the preset reference direction.

[0230] After the above process S13, the gradient direction information D0, D1, D2 and D3 of the second pixel point are obtained; and after the above process S14, the reference interpolation results y_ang[0], u_ang[0], v_ang[0] in the 0° direction are obtained; the reference interpolation results y_ang[1], u_ang[1], v_ang[1] in the 45° direction are obtained; the reference interpolation results y_ang[2], u_ang[2], u_ang[2] in the 135° direction are obtained; and the reference interpolation results y_ang[4], u_ang[4], v_ang[4] in the 90° direction are obtained.

[0231] Based on the above results, the first interpolation result of the second pixel point in the gradient direction is determined. Here, "gradient direction" can be understood as the directions D0, D1, D2, and D3 indicated by the gradient direction information.

[0232] Any direction D0, D1, D2, or D3 calculated based on the surrounding four sub-pixel blocks B_sub0, B_sub1, B_sub2, and B_sub3 can be formed by combining four preset reference directions: 0°, 45°, 90°, and 135°. For example, if D0 is 22°, located between 0° and 45°, then D0 can be obtained by combining 0° and 45°. The actual interpolation result of the second pixel in the D0 direction (i.e., the first interpolation result) can also be obtained by combining the reference interpolation results corresponding to 0° and 45° respectively.

[0233] For example, the first interpolation result of the second pixel point in the D0 direction is determined, and the specific process is shown in the following formula (20):

[0234]

[0235] index1 = index0 + 1

[0236] angle_phase=D0-idex0×45

[0237]

[0238]

[0239]

[0240] Where y_dir[0] represents the first interpolation result of the Y channel of the second pixel in the D0 direction; u_dir[0] represents the first interpolation result of the U channel of the second pixel in the D0 direction; v_dir[0] represents the first interpolation result of the V channel of the second pixel in the D0 direction; This indicates rounding down to the nearest integer.

[0241] Similarly, the first interpolation results y_dir[1], u_dir[1], and v_dir[1] of the second pixel in the D1 direction, the first interpolation results y_dir[2], u_dir[2], and v_dir[2] of the second pixel in the D2 direction, and the first interpolation results y_dir[3], u_dir[3], and v_dir[3] of the second pixel in the D3 direction are processed in the same way as the first interpolation results y_dir[0], u_dir[0], and v_dir[0] of the second pixel in the D0 direction (i.e., formula (20)). The repeated parts will not be repeated.

[0242] The processes S11 to S15 described above can calculate the texture gradient relatively accurately, and have good interpolation effects (first interpolation result) for gradients in various directions (D0 to D3). Specifically, by combining gradient direction information and reference interpolation results in a preset reference direction, this process can calculate the gradient and determine the first interpolation result at texture details with small gradients, preserving high-resolution details in the image, so that the second pixel has a good interpolation effect in the gradient direction.

[0243] S16, the fusion module 6 fuses the first interpolation result of the second pixel point in the gradient direction to obtain the first interpolation fusion result.

[0244] The image processing device further includes a phase determination module 8; the phase determination module 8 is used to determine the phase information of the second pixel relative to the target pixel based on the position information of the target pixel, the position information of the second pixel, and the preset magnification. The fusion module 6 is used to fuse the first interpolation result of the second pixel in the gradient direction based on the phase information to obtain a first interpolation fusion result.

[0245] The phase information of the second pixel relative to the target pixel is px_phase and ln_phase. The first interpolation result of the second pixel in the gradient direction includes y_dir[0], u_dir[0], v_dir[0], y_dir[1], u_dir[1], v_dir[1], y_dir[2], u_dir[2], v_dir[2], y_dir[3], u_dir[3], v_dir[3].

[0246] For example, bilinear interpolation can be used to determine the first interpolation fusion result y_r0 of the second pixel in the Y channel. The specific process is shown in the following formula (XXI):

[0247] y_dir_t=y_dir[0]×(1-px_phase)+y_dir[1]×px_phase

[0248] y_dir_b=y_dir[2]×(1-px_phase)+y_dir[3]×px_phase

[0249] Formula (21): y_r0 = y_dir_t × (1 - ln_phase) + y_dir_b × ln_phase

[0250] Similarly, using bilinear interpolation, the first interpolation fusion result u_r0 of the second pixel in the U channel and the first interpolation fusion result v_r0 of the second pixel in the V channel are determined. See the process of determining the first interpolation fusion result y_r0 of the second pixel in the Y channel (i.e., formula (21)). Repeated parts will not be repeated.

[0251] S17. The image optimization module 7 obtains the optimized target image based at least on the first interpolation fusion result.

[0252] For example, the image optimization module 7 can perform inverse color gamut space conversion on the first interpolation fusion results y_r0, u_r0 and v_r0 to obtain the optimized target image. For details, please refer to the following formula (22):

[0253]

[0254] Here, >>10 represents a shift operation, specifically meaning division by 1024; The conversion parameters represent the inverted color gamut space; The parameters are those used for normalization after the color gamut space is inverted; R2, G2, and B2 represent the sub-pixel values ​​of the red, green, and blue channels of the target image pixels, respectively, after optimization.

[0255] In some embodiments, the image processing apparatus further includes a phase determination module 8; before determining the reference interpolation result of the second pixel in the preset reference direction, the phase determination module 8 determines the phase information of the second pixel relative to the target pixel in the input image after determining the target pixel in the input image that corresponds one-to-one with each second pixel, based on the position information of the target pixel, the position information of the second pixel and the preset magnification.

[0256] For example, given that the preset magnification is rate, the position information of the second pixel is (p_o_xf, p_o_yf), and the position information of the target pixel corresponding to the second pixel is (p_i_xf, p_i_yf), the process of determining the phase information of the second pixel relative to the target pixel is shown in the following formula (VI).

[0257] Subsequently, for the process of obtaining the optimized target image based at least on the first interpolation fusion result, specifically, the image optimization module 7 can use the bilinear interpolation method to determine the second interpolation result of the second pixel point; then, based on the first interpolation fusion result and the second interpolation result, the optimized target image is obtained.

[0258] Image optimization module 7 includes a first optimization unit, a second optimization unit, and a third optimization unit; wherein, the first optimization unit is used to execute the following S17-1, the second optimization unit is used to execute the following S17-2, and the third optimization unit is used to execute the following S17-3, specifically including steps S17-1 to S17-3, wherein:

[0259] S17-1, The first optimization unit determines multiple fifth associated pixels corresponding to the target pixel based on the position information of other target pixels around the target pixel and the pre-set fifth association conditions.

[0260] Here, the target location refers to the location of the target pixel.

[0261] Following a general linear interpolation algorithm, multiple fifth-associative pixels corresponding to the target pixel are selected from other target pixels surrounding the target pixel. Taking the target pixel (1,1) as an example, if a bilinear interpolation algorithm is used, the pre-set fifth-associative condition is to determine a 2×2 pixel group with the target pixel as the top-left corner pixel. This pixel group includes the target pixels (1,1), (1,2), (1,3), and (1,4), where (1,2), (1,3), and (1,4) are the fifth-associative pixels associated with the target pixel (1,1). Here, by increasing the number of fifth-associative pixels associated with the target pixel, the accuracy of the interpolation result is improved.

[0262] S17-2, The second optimization unit determines the second interpolation result of the second pixel based on the phase information, the color gamut information of the target pixel, and the color gamut information of multiple fifth associated pixels.

[0263] Color gamut information includes luminance Y, first chromaticity value U, and second chromaticity value V. Phase information includes horizontal phase px_phase and vertical phase ln_phase.

[0264] For example, the second interpolation result y_r1 of the second pixel in the Y channel can be determined based on the horizontal phase, the brightness Y of the target pixel, and the brightness Y of each of the multiple fifth associated pixels. For the specific determination process, please refer to the following formula (XXIII):

[0265] y1_t=(Y(1,1)×(1-px_phase)+Y(1,2)×px_phase)

[0266] y1_b=(Y(2,1)×(1-px_phase)+Y(2,2)×px_phase)

[0267] Formula (XXIII) is given by: y_r1 = (y1_t × (1 - ln_phase) + y1_b × ln_phase)

[0268] Similarly, to determine the second interpolation result u_r1 of the second pixel point in the U channel and the second interpolation result v_r1 of the second pixel point in the V channel, please refer to the above process of determining the second interpolation result y_r1 of the second pixel point in the Y channel (i.e., formula (23)). Repeated parts will not be repeated.

[0269] S17-3, the third optimization unit obtains the optimized target image based on the first interpolation fusion result and the second interpolation result.

[0270] Step S17-3 above specifically includes S17-3-1 to S17-3-3, wherein:

[0271] S17-3-1. For each sub-pixel block in multiple pixel blocks, determine the interpolation fusion information based on the color gamut information of each target pixel in the sub-pixel block.

[0272] The interpolation fusion information here can be determined after S13-1-2, based on the multiple sub-pixel blocks obtained from the division, specifically including steps S17-3-11 to S17-3-15, wherein:

[0273] S17-3-11. Based on the color gamut information of each target pixel in the sub-pixel block, determine the gradient of the sub-pixel block in the horizontal direction and the gradient in the vertical direction.

[0274] Color gamut information includes luminance Y, first chromaticity value U, and second chromaticity value V.

[0275] For example, the gradients g_h[0], g_h[1], g_h[2] and g_h[3] of the sub-pixel block in the horizontal direction can be determined based on the brightness Y of each target pixel in the sub-pixel block B_sub0. For details, please refer to the following formula (24):

[0276] g_h[0]=(Y(0,0)-Y(0,1))+(Y(1,0)-Y(1,1))

[0277] g_h[1]=(Y(0,1)-Y(0,2))+(Y(1,1)-Y(1,2))

[0278] g_h[2]=(Y(1,0)-Y(1,1))+(Y(2,0)-Y(2,1))

[0279] g_h[3]=(Y(1,1)-Y(1,2))+(Y(2,1)-Y(2,2)) Formula (Twenty-four)

[0280] The calculation process for the gradients in the horizontal direction for the other sub-pixel blocks B_sub1, B_sub2, and B_sub3 is similar, and the repeated parts will not be repeated.

[0281] For example, the gradients g_v[0], g_v[1], g_v[2] and g_v[3] of the sub-pixel block in the vertical direction can be determined based on the brightness Y of each target pixel in the sub-pixel block B_sub0. For details, please refer to the following formula (25):

[0282] g_v[0]=(Y(0,0)-Y(1,0))+(Y(0,1)-Y(1,1))

[0283] g_v[1]=(Y(0,1)-Y(1,1))+(Y(0,2)-Y(1,2))

[0284] g_v[2]=(Y(1,0)-Y(2,0))+(Y(1,1)-Y(2,1))

[0285] g_v[3]=(Y(1,1)-Y(2,1))+(Y(1,2)-Y(2,2)) Formula (Twenty-five)

[0286] The calculation process for the gradients in the vertical direction of other sub-pixel blocks B_sub1, B_sub2, and B_sub3 is similar, and the repeated parts will not be repeated.

[0287] S17-3-12. Determine the autocorrelation information of the horizontal gradient based on the gradient of the sub-pixel block in the horizontal direction.

[0288] For example, the self-correlation information g_hh of the horizontal gradient can be determined based on the gradients g_h[0], g_h[1], g_h[2] and g_h[3] of the sub-pixel block B_sub0 in the horizontal direction. For details, please refer to the following formula (26):

[0289] g_hh=g_h[0]×g_h[0]+g_h[1]×g_h[1]+g_h[2]×g_h[2]+g_h[3]×g_h[3] Formula (Twenty-six)

[0290] The calculation process for the autocorrelation information of the horizontal gradients corresponding to other sub-pixel blocks B_sub1, B_sub2, and B_sub3 is similar, and the repeated parts will not be repeated.

[0291] S17-3-13. Determine the autocorrelation information of the vertical gradient based on the gradient of the sub-pixel block in the vertical direction.

[0292] For example, the self-related information g_vv of the vertical gradient can be determined based on the gradients g_v[0], g_v[1], g_v[2] and g_v[3] of the sub-pixel block B_sub0 in the vertical direction. The specific process is as follows: Formula (XXVII): g_vv=g_v[0]×g_v[0]+g_v[1]×g_v[1]+g_v[2]×g_v[2]+g_v[3]×g_v[3] Formula (XXVII)

[0293] The calculation process for the autocorrelation information of the vertical gradients of other sub-pixel blocks B_sub1, B_sub2, and B_sub3 is similar, and the repeated parts will not be repeated.

[0294] S17-3-14. Determine the cross-correlation between the horizontal and vertical gradients based on the gradients of the sub-pixel blocks in the horizontal and vertical directions.

[0295] For example, the mutual information g_hv of the horizontal and vertical gradients can be determined based on the gradients g_h[0], g_h[1], g_h[2] and g_h[3] of the sub-pixel block B_sub0 in the horizontal direction and the gradients g_v[0], g_v[1], g_v[2] and g_v[3] in the vertical direction. The specific process is as follows: Formula (XXVIII)

[0296] g_hv=g_h[0]×g_v[0]+g_h[1]×g_v[1]+g_h[2]×g_v[2]+g_h[3]×g_v[3] Formula (Twenty-eight)

[0297] The calculation process for the cross-correlation information of the horizontal and vertical gradients of other sub-pixel blocks B_sub1, B_sub2 and B_sub3 is similar, and the repeated parts will not be repeated.

[0298] S17-3-15. Based on the autocorrelation information of the horizontal gradient, the autocorrelation information of the vertical gradient, and the cross-correlation information of the horizontal and vertical gradients, determine the interpolation fusion information.

[0299] For example, the interpolation fusion information alpha_1 determined based on sub-pixel block B_sub0 can be determined according to the autocorrelation information g_hh of the horizontal gradient, the autocorrelation information g_vv of the vertical gradient, and the mutual correlation information g_hv of the horizontal and vertical gradients. The specific process is as follows: Formula (XXIX)

[0300]

[0301] Similarly, the interpolation fusion information determined based on sub-pixel block B_sub1 is alpha_1; the interpolation fusion information determined based on sub-pixel block B_sub2 is alpha_2; and the interpolation fusion information determined based on sub-pixel block B_sub3 is alpha_3.

[0302] S17-3-2. Based on at least some interpolation fusion information, the first interpolation fusion result and the second interpolation result are fused to obtain the second interpolation fusion result.

[0303] In specific implementation, a target interpolation fusion information alpha_target is determined based on at least some interpolation fusion information; the target interpolation fusion information alpha_target is used to fuse the first interpolation fusion result and the second interpolation result to obtain the second interpolation fusion result.

[0304] For example, the fused mean of alpha_0, alpha_1, alpha_2, and alpha_3 can be calculated, and this fused mean can be used as the target interpolation fusion information alpha_target. The mean calculation process involves less computation compared to complex algorithms, so this example can improve optimization efficiency.

[0305] For example, based on alpha_0, alpha_1, alpha_2, and alpha_3, a bilinear interpolation algorithm is used to obtain the interpolation fusion information target alpha_target. The specific process is as follows: Formula (XXX)

[0306] alpha_t=(alpha_0×(1-px_phase)+alpha_1×px_phase)

[0307] alpha_b=(alpha_2×(1-px_phase)+alpha_3×px_phase)

[0308] alpha_target=(alpha_t×(1-ln_phase)+alpha_b×ln_phase) formula (30)

[0309] Of course, in addition to using the gradient calculation process from S17-3-1 to S17-3-5 described above to determine the interpolation fusion information, other gradient calculation methods can also be used to obtain similar fusion information. This embodiment of the present disclosure does not specifically limit this method.

[0310] The first interpolation fusion result y_r0, u_r0, v_r0 and the second interpolation result y_r1, u_r1, v_r1 are fused using alpha_target to obtain the second interpolation fusion result y_r1, u_r1 and v_r1. The specific process is as follows: Formula (31)

[0311] y_r1=y_r0×alpha_target+y_r1×(1-alpha_target)

[0312] u_r1=u_r0×alpha_target+u_r1×(1-alpha_target)

[0313] Formula (31) u_r1=v_r0×alpha_target+v_r1×(1-alpha_target)

[0314] S17-3-3. Based on the second interpolation fusion result, the optimized target image is obtained.

[0315] For example, the second interpolation fusion results y_r1, u_r1, and v_r1 can be subjected to inverse color gamut space transformation to obtain the optimized target image. The specific process is as follows: Formula (XXXII):

[0316]

[0317] Here, >>10 represents a shift operation, specifically meaning division by 1024; The conversion parameters represent the inverted color gamut space; The parameters are those used for normalization after the color gamut space is inverted; R2, G2, and B2 represent the sub-pixel values ​​of the red, green, and blue channels of the target image pixels, respectively, after optimization.

[0318] The present disclosure describes the entire S17-3 processing procedure, which integrates the results of gradient direction interpolation and the interpolation magnification results obtained by general interpolation algorithms. By adaptively calculating based on the strength of gradient and directionality, this method can preserve the intensity of texture and contour in the magnified image (output image) to a certain extent.

[0319] To further understand the overall image optimization process of this disclosure, a complete embodiment is described in detail below. Figure 5 A detailed flowchart of an image processing method provided in this disclosure embodiment is shown below. Figure 5 As shown, it includes steps S21 to S210, wherein:

[0320] S21. The space conversion module 9 performs color space conversion on the input image to be optimized to obtain the color space information of the input image; then, steps S22 and S28 are executed synchronously.

[0321] S22. The magnification module 1 magnifies the input image to be optimized according to the preset magnification ratio to obtain the output image.

[0322] S23, the pixel determination module 2 determines the target pixel in the input image to map each second pixel of the output image, determines the position information of the target pixel, and the phase determination module 8 determines the phase information of the second pixel relative to the target pixel.

[0323] S24, the gradient direction determination module 3 determines the gradient direction information of the second pixel, and the third optimization unit determines the interpolation fusion information.

[0324] S25. The first processing module 4 determines the reference interpolation result of the second pixel point in the preset reference direction.

[0325] S26. The second processing module 5 determines the first interpolation result of the second pixel in the gradient direction based on the gradient direction information of the second pixel and the reference interpolation result in the preset reference direction.

[0326] S27. The fusion module 6 uses phase information to fuse the first interpolation result of the second pixel in the gradient direction to obtain the first interpolation fusion result, and then executes step S29.

[0327] S28, the first optimization unit and the second optimization unit work together to determine the second interpolation result of the second pixel using a general interpolation algorithm.

[0328] S29. The third optimization unit uses the interpolation fusion information to fuse the first interpolation fusion result and the second interpolation result to obtain the second interpolation fusion result.

[0329] S210 and the third optimization unit perform inverse color gamut space conversion on the second interpolation fusion result to obtain the optimized target image.

[0330] For details on the specific implementation process of steps S21 to S210, please refer to the detailed descriptions of the above embodiments. Repeated parts will not be repeated.

[0331] The above is a complete description of the image processing apparatus provided in this disclosure, and the image processing methods corresponding to the functions of each module. This disclosure uses a gradient direction interpolation fusion-based image processing method. It maps the second pixel of the output image to the target pixel in the input image to be optimized, extracts other target pixels near the target pixel in the input image to form a pixel block, and divides this into multiple sub-pixel blocks. Difference is calculated based on the gradient directions (D0~D3) obtained from the surrounding sub-pixel blocks. This method can ensure the smoothness of the texture and transition areas of the output image during magnification, greatly reducing edge jaggedness and blurring. Of course, to improve the optimization effect, more gradient directions (D0~Dn) can be extended; the more directions, the better the final fusion effect. Considering both the computational load and optimization efficiency, a suitable gradient direction (D0~Dm) is extended to achieve high optimization efficiency and good optimization results.

[0332] Those skilled in the art will understand that, in the above-described method of the specific implementation, the order in which each step is written does not imply a strict execution order and does not constitute any limitation on the implementation process. The specific execution order of each step should be determined by its function and possible internal logic.

[0333] In addition, this disclosure also provides an electronic device that includes the image processing apparatus described in any of the above embodiments. This electronic device includes, for example, mobile phones, tablet computers, televisions, monitors, laptops, digital photo frames, in-vehicle devices, and other electronic products.

[0334] In some embodiments, the image processing device is an integrated chip.

[0335] Figure 6 This is a schematic diagram of the structure of a computer device provided in an embodiment of the present disclosure, such as... Figure 6 As shown, this disclosure provides a computer device including: one or more processors 601, a memory 602, and one or more I / O interfaces 603. The memory 602 stores one or more programs, which, when executed by the one or more processors, cause the one or more processors to implement any of the image processing devices described in the above embodiments; the one or more I / O interfaces 603 are connected between the processor and the memory, configured to enable information interaction between the processor and the memory.

[0336] The processor 601 is a device with data processing capabilities, including but not limited to a central processing unit (CPU); the memory 602 is a device with data storage capabilities, including but not limited to random access memory (RAM, more specifically SDRAM, DDR, etc.), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), and flash memory (FLASH); the I / O interface (read / write interface) 603 is connected between the processor 601 and the memory 602, enabling information exchange between the processor 601 and the memory 602, including but not limited to a data bus (Bus).

[0337] In some embodiments, the processor 601, memory 602, and I / O interface 603 are interconnected via bus 604, and thus connected to other components of the computing device.

[0338] According to embodiments of this disclosure, a computer non-transient readable storage medium is also provided. This computer non-transient readable storage medium stores a computer program, which, when executed by a processor, implements the steps in any of the image processing apparatuses described in the above embodiments.

[0339] In particular, according to embodiments of this disclosure, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of this disclosure include a computer program product comprising a computer program carried on a machine-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via a communication component, and / or installed from a removable medium. When the computer program is executed by a central processing unit (CPU), it performs the functions defined above in the system of this disclosure.

[0340] It should be noted that the computer-readable non-transient readable medium disclosed herein may be a computer-readable signal medium or a computer-readable storage medium, or any combination thereof. Computer-readable storage media may be, for example, but not limited to, electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatuses, or devices, or any combination thereof. More specific examples of computer-readable storage media may include, but are not limited to: electrical connections having one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disc read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof. In this disclosure, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. In this disclosure, a computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. The transmitted data signal can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. The computer-readable signal medium can also be any non-transient readable computer storage medium other than a computer-readable storage medium, which can transmit, propagate, or transfer a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the non-transient readable computer storage medium can be transmitted using any suitable medium, including but not limited to: wireless, wire, optical fiber, RF, etc., or any suitable combination thereof.

[0341] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of apparatus, methods, and computer program products according to various embodiments of the present disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions marked in the blocks may occur in a different order than those shown in the drawings. For example, two adjacent blocks may actually represent substantially parallel execution, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0342] It is understood that the above embodiments are merely exemplary embodiments used to illustrate the principles of this disclosure, and this disclosure is not limited thereto. For those skilled in the art, various modifications and improvements can be made without departing from the spirit and substance of this disclosure, and these modifications and improvements are also considered to be within the scope of protection of this disclosure.

Claims

1. An image processing apparatus, wherein, include: The magnification module is used to magnify the input image to be optimized according to a preset magnification ratio to obtain the output image; The pixel determination module is used to determine the target pixel in the input image that corresponds one-to-one with each of the second pixels based on the position information of each first pixel in the input image, the position information of each second pixel in the output image, and the preset magnification. The gradient direction determination module is used to determine, based on the position information of the target pixel and the position information of other target pixels around the target pixel, that the second pixel corresponds to multiple sub-pixel blocks in the input image; for each sub-pixel block, the gradient direction information of the second pixel is determined based on the color gamut information of each target pixel in the sub-pixel block; different sub-pixel blocks correspond to different gradient direction information for the second pixel; The first processing module is used to determine the reference interpolation result of the second pixel in a preset reference direction based on the position information and color gamut information of the target pixel, as well as the position information and color gamut information of other target pixels around the target pixel. The second processing module is used to determine the weight of the reference interpolation result for each sub-pixel block based on the gradient direction indicated by the gradient direction information of the second pixel and the preset reference direction. Based on the weight of the reference interpolation result and the reference interpolation result of the second pixel in the preset reference direction, the first interpolation result of the second pixel in the gradient direction is determined; The fusion module is used to fuse the first interpolation results of the second pixel points corresponding to the multiple sub-pixel blocks in the gradient direction to obtain the first interpolation fusion result; An image optimization module is used to obtain an optimized target image based at least on the first interpolation fusion result.

2. The image processing apparatus according to claim 1, wherein, The gradient direction determination module includes a pixel block determination unit, a sub-pixel block determination unit, and a gradient direction determination unit. The pixel block determination unit is used to determine, based on the position information of the target pixel and the position information of other target pixels around the target pixel, the pixel block corresponding to the second pixel in the input image. The sub-pixel block determining unit is used to divide the pixel block into multiple sub-pixel blocks according to preset division conditions; The gradient direction determination unit is used to determine the gradient direction information of the second pixel point for each of the sub-pixel blocks in the plurality of pixel blocks, based on the color gamut information of each target pixel point in the sub-pixel block.

3. The image processing apparatus according to claim 2, wherein, The gradient direction determination unit is specifically used to determine the value of the second pixel in the horizontal direction and the value in the vertical direction based on the color gamut information of each target pixel in the sub-pixel block; and to determine the gradient direction information of the second pixel based on the value of the second pixel in the horizontal direction and the value in the vertical direction.

4. The image processing apparatus according to claim 1, wherein, The image processing device further includes a phase determination module; The phase determination module is used to determine the phase information of the second pixel relative to the target pixel based on the position information of the target pixel, the position information of the second pixel, and the preset magnification. The first processing module is specifically used to determine the reference interpolation result of the second pixel in a preset reference direction based on the phase information, the color gamut information of the target pixel, and the position information and color gamut information of other target pixels around the target pixel.

5. The image processing apparatus according to claim 4, wherein, The preset reference direction includes a first reference direction; the phase information of the second pixel relative to the target pixel includes horizontal phase and vertical phase; The first processing module includes a first processing unit, a second processing unit, and a third processing unit; The first processing unit is configured to determine a plurality of first associated pixels corresponding to the target pixel based on the position information of other target pixels around the target pixel and a first association condition pre-configured for the first reference direction; The second processing unit is configured to determine a first intermediate interpolation result of the second pixel based on the horizontal phase, the color gamut information of the target pixel, and the color gamut information of a plurality of first associated pixels. The third processing unit is used to determine the reference interpolation result of the second pixel in the first reference direction based on the first intermediate interpolation result of the second pixel and the vertical phase.

6. The image processing apparatus according to claim 4, wherein, The preset reference direction includes a second reference direction; the phase information of the second pixel relative to the target pixel includes horizontal phase and vertical phase; The first processing module includes a first processing unit, a second processing unit, a third processing unit, and a fourth processing unit; The fourth processing unit is used to determine the first angular phase, the first left phase, and the first right phase of the second pixel based on the horizontal phase and the vertical phase. The first processing unit is configured to determine a plurality of second associated pixels corresponding to the target pixel based on the comparison result of the horizontal phase and the vertical phase, the position information of other target pixels around the target pixel, and the second association conditions pre-configured for the second reference direction; The second processing unit is configured to determine the second intermediate interpolation result of the second pixel based on the first left phase, the first right phase, the color gamut information of the target pixel, and the color gamut information of a plurality of second associated pixels; The third processing unit is used to determine the reference interpolation result of the second pixel in the second reference direction based on the second intermediate interpolation result of the second pixel and the first angle phase.

7. The image processing apparatus according to claim 4, wherein, The preset reference direction includes a fourth reference direction; the phase information of the second pixel relative to the target pixel includes horizontal phase and vertical phase; The first processing module includes a first processing unit, a second processing unit, a third processing unit, and a fourth processing unit; The fourth processing unit is used to determine the second angle phase, the second left phase, and the second right phase of the second pixel based on the horizontal phase and the vertical phase. The first processing unit is configured to determine a plurality of fourth associated pixels corresponding to the target pixel based on the horizontal phase, the vertical phase, the position information of other target pixels around the target pixel and the fourth association conditions pre-configured for the fourth reference direction; The second processing unit is configured to determine the fourth intermediate interpolation result of the second pixel based on the second left phase, the second right phase, the color gamut information of the target pixel, and the color gamut information of the plurality of fourth associated pixels; The third processing unit is used to determine the reference interpolation result of the second pixel in the fourth reference direction based on the fourth intermediate interpolation result of the second pixel and the second angle phase.

8. The image processing apparatus according to claim 1, wherein, The image processing device further includes a phase determination module; The phase determination module is used to determine the phase information of the second pixel relative to the target pixel based on the position information of the target pixel, the position information of the second pixel, and the preset magnification. The fusion module is used to fuse the first interpolation result of the second pixel point in the gradient direction based on the phase information to obtain a first interpolation fusion result.

9. The image processing apparatus according to claim 2, wherein, The image processing device further includes a phase determination module; The phase determination module is used to determine the phase information of the second pixel relative to the target pixel based on the position information of the target pixel, the position information of the second pixel, and the preset magnification. The image optimization module includes a first optimization unit, a second optimization unit, and a third optimization unit; The first optimization unit is used to determine a plurality of fifth associated pixels corresponding to the target pixel based on the position information of other target pixels around the target pixel and a pre-set fifth association condition; The second optimization unit is used to determine the second interpolation result of the second pixel based on the phase information, the color gamut information of the target pixel, and the color gamut information of the plurality of fifth associated pixels; The third optimization unit is used to obtain the optimized target image based on the first interpolation fusion result and the second interpolation result.

10. The image processing apparatus according to claim 9, wherein, The third optimization unit is specifically used to determine interpolation fusion information for each of the sub-pixel blocks in the plurality of pixel blocks based on the color gamut information of each target pixel in the sub-pixel block; to fuse the first interpolation fusion result and the second interpolation result based on at least part of the interpolation fusion information to obtain a second interpolation fusion result; and to obtain an optimized target image based on the second interpolation fusion result.

11. The image processing apparatus according to claim 10, wherein, The third optimization unit is specifically used to determine the gradient in the horizontal direction and the gradient in the vertical direction of the sub-pixel block based on the color gamut information of each target pixel in the sub-pixel block; determine the autocorrelation information of the horizontal gradient based on the gradient in the horizontal direction of the sub-pixel block; determine the autocorrelation information of the vertical gradient based on the gradient in the vertical direction of the sub-pixel block; determine the cross-correlation information of the horizontal gradient and the vertical gradient based on the gradient in the horizontal direction and the gradient in the vertical direction of the sub-pixel block; and determine the interpolation fusion information based on the autocorrelation information of the horizontal gradient, the autocorrelation information of the vertical gradient, and the cross-correlation information of the horizontal gradient and the vertical gradient.

12. The image processing apparatus according to claim 10, wherein, The image processing device also includes a spatial conversion module; The space conversion module is used to perform color gamut space conversion on the input image to be optimized, so as to obtain the color gamut information of the input image; The third optimization unit is specifically used to perform reverse color gamut space conversion on the second interpolation fusion result to obtain the optimized target image.

13. An image processing method, wherein, include: The input image to be optimized is magnified according to the preset magnification ratio to obtain the output image; Based on the position information of each first pixel in the input image, the position information of each second pixel in the output image, and the preset magnification, the target pixel in the input image that corresponds one-to-one with each second pixel is determined; Based on the position information of the target pixel and the position information of other target pixels around the target pixel, it is determined that the second pixel corresponds to multiple sub-pixel blocks in the input image; for each sub-pixel block, the gradient direction information of the second pixel is determined based on the color gamut information of each target pixel in the sub-pixel block; different sub-pixel blocks correspond to different gradient direction information of the second pixel; Based on the position information and color gamut information of the target pixel, as well as the position information and color gamut information of other target pixels around the target pixel, the reference interpolation result of the second pixel in the preset reference direction is determined; For each sub-pixel block, the weight of the reference interpolation result is determined based on the gradient direction indicated by the gradient direction information of the second pixel and the preset reference direction; Based on the weight of the reference interpolation result and the reference interpolation result of the second pixel in the preset reference direction, the first interpolation result of the second pixel in the gradient direction is determined; The first interpolation results of the second pixel points corresponding to the multiple sub-pixel blocks in the gradient direction are fused to obtain the first interpolation fusion result; Based at least on the first interpolation fusion result, an optimized target image is obtained.

14. An electronic device comprising an image processing apparatus as claimed in any one of claims 1 to 12.

15. A computer device, wherein, include: The computer device includes a processor, a memory, and a bus. The memory stores machine-readable instructions executable by the processor. When the computer device is running, the processor communicates with the memory via the bus. When the machine-readable instructions are executed by the processor, the steps of the image processing method as described in claim 13 are performed.

16. A computer-defined non-transient readable storage medium, wherein, The computer non-transient readable storage medium stores a computer program that, when executed by a processor, performs the steps of the image processing method as described in claim 13.

Images